Chatbot

Quick start

First time only — build the app image and download the model:

docker compose build app
docker compose up -d ollama app
docker network connect chatbot_template_default ollama || true
docker compose exec ollama ollama pull qwen2.5:7b
docker network disconnect chatbot_template_default ollama || true

Every time — start services and run the chatbot:

docker compose up -d ollama app
docker compose exec app uv run streamlit run scripts/chatbot.py --server.address 0.0.0.0 --server.port 8501

Then open http://localhost:8501 in your browser. Replace the localhost with the IP address of the server if needed.

---

Stop

docker compose down

Download additional models

MODEL=qwen2.5-coder:7b
docker network connect chatbot_template_default ollama || true
docker compose exec ollama ollama pull "$MODEL"
docker network disconnect chatbot_template_default ollama || true

List installed models:

docker compose exec ollama ollama list

Switching the active model

The chatbot reads the model name from the OLLAMA_MODEL environment variable (default: qwen2.5:7b). To switch to a different model you've already pulled, set it before starting app:

export OLLAMA_MODEL=qwen2.5-coder:7b
docker compose up -d ollama app

Or add it to a .env file in the project root:

OLLAMA_MODEL=qwen2.5-coder:7b

Services

Service	Description	Port
ollama	LLM runtime, isolated from internet at runtime	11434
app	Application container, workspace at /workspaces/app	—

Network isolation

Ollama has no outbound internet during normal runtime. To download models, temporarily connect it to the default network as shown above.

GPU support

By default, Ollama runs on CPU. To use an NVIDIA GPU instead, add -f docker-compose.gpu.yml to your docker compose commands, e.g.:

docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d ollama app

This requires the NVIDIA Container Toolkit to be installed on the host — without it, the GPU-enabled ollama container will fail to start with could not select device driver "nvidia".

Tip: set export COMPOSE_FILE=docker-compose.yml:docker-compose.gpu.yml in your shell to make the GPU overlay apply automatically without adding -f flags each time.