Docker strategy

The organization goal is to run APIs in Docker. The server is not using Docker yet. This document explains what changes, what stays the same, and how to migrate without redoing Nginx or secrets layout.

Short answer

The edge of the stack stays the same. Nginx, TLS, firewall, subdomain routing, and secret files under /etc/<api-name>/ do not change.

What moves inside the container: Python runtime, virtualenv, and application dependencies. Instead of uvicorn/gunicorn on the host, a container process runs the same command.

What you can drop on the host (Docker mode): /opt/venvs/<api-name>/ — dependencies live in the image.

What you may keep on the host: /opt/apis/<api-name>/ as a git checkout only if you build images on the server. Prefer building in CI and pulling from a registry.

Side-by-side: native vs Docker

Concern	Native (current plan)	Docker (target)
Application code	`/opt/apis/<api-name>/` git checkout	Baked into image (from CI build)
Dependencies	`/opt/venvs/<api-name>/`	Inside image
Secrets	`/etc/<api-name>/<api-name>.env`	Same path; mounted via `env_file` or `--env-file`
Process manager	systemd runs gunicorn/uvicorn	systemd runs `docker compose` or Docker `restart: unless-stopped`
App listen address	`127.0.0.1:<port>` on host	Container listens on `8080`; host maps `127.0.0.1:<port>:8080`
Nginx upstream	`127.0.0.1:8001`	Unchanged — still `127.0.0.1:8001`
UFW	22, 80, 443 only	Unchanged — do not publish app ports on `0.0.0.0`
Logs	`journalctl -u <api-name>`	`docker logs <container>` or journald if systemd wraps compose
Deploy	`git pull` + `pip install` + restart	`docker pull` + recreate container (or compose up)

                    ┌─────────────────────────────────────┐
  Internet ────────►│ Nginx :443                          │
                    │   proxy_pass → 127.0.0.1:8001       │
                    └──────────────────┬──────────────────┘
                                       │
         ┌─────────────────────────────┴─────────────────────────────┐
         │ Native                         Docker (target)              │
         │ gunicorn on host :8001         127.0.0.1:8001 → container   │
         │ /opt/venvs/...                 :8080                        │
         └─────────────────────────────────────────────────────────────┘

Recommended rollout phases

Phase 1 — Native (now)

Use 02-adding-a-new-api.md: venv + systemd. Gets the first API live on the laptop server quickly without installing Docker.

Phase 2 — Dockerfile in each repo (next)

Add to every API repository:

Dockerfile
.dockerignore
docker-compose.yml (for local dev and optional server use)

Build and test on a developer workstation. No server changes yet.

Phase 3 — Docker on server

Install and configure users — full copy-paste: 08-docker-install-and-users.md.

sudo apt install -y docker.io docker-compose-v2
sudo systemctl enable --now docker
sudo usermod -aG docker gqc   # admin only — not apisvc

Replace the systemd unit from “run gunicorn” to “run compose” (see templates/docker/api.service.docker.template).

Secrets remain at /etc/<api-name>/<api-name>.env.

Phase 4 — CI builds and registry (optional but standard)

GitHub Actions or Azure DevOps builds the image on push, pushes to GHCR or Azure Container Registry, server pulls tagged images. Avoid building heavy images on the laptop.

Host layout with Docker

Minimal production layout once Docker is enabled:

/etc/<api-name>/
    <api-name>.env              # secrets (unchanged)
    docker-compose.yml          # service definition, image tag, port map

/var/lib/docker/                # Docker data (default)

# Optional — only if building on server instead of pulling:
/opt/apis/<api-name>/           # git checkout + Dockerfile

You do not need /opt/venvs/ in Docker mode.

Port allocation in README.md is unchanged: first API 8001, second 8002, etc.

Container conventions (all APIs)

Setting	Value	Notes
Base image	`python:3.11-slim-bookworm`	Match `runtime.txt` in each repo
Internal port	`8080`	Set `PORT=8080`; map to host `<port>`
Process	`gunicorn --config gunicorn.conf.py main:app`	Same as Azure App Service today
User	non-root `appuser` inside container	Do not run as root in container
Host bind	`127.0.0.1:<port>:8080`	Never `0.0.0.0:<port>:8080` on the host
Health check	`GET /health`	Use Docker `HEALTHCHECK`
Logs	stdout/stderr	`gunicorn.conf.py` already logs to `-` when `PORT` is set

Compose file naming (standard pattern)

Use two files in the git repo plus one canonical file on the server:

File	Location	Purpose
`docker-compose.yml`	Git repo	Local dev — relative paths, `.env` in project dir
`docker-compose.prod.yml`	Git repo	Production reference — absolute server paths
`docker-compose.yml`	`/etc/<api-name>/` on server	Live production compose (systemd points here)

Why not run prod directly from /opt/apis/?

Production binds 127.0.0.1:8001, reads /etc/.../secrets.env — server-specific, not developer-laptop-specific.
Keeps secrets and port maps out of the git checkout (git pull does not overwrite /etc/).
systemd always uses one stable path: /etc/<api-name>/docker-compose.yml.

Deploy flow:

sudo cp /opt/apis/dwd-api-fastapi/docker-compose.prod.yml \
        /etc/dwd-api-fastapi/docker-compose.yml
sudo docker compose -f /etc/dwd-api-fastapi/docker-compose.yml up -d --build

You may edit /etc/.../docker-compose.yml on the server for one-off changes (image tag); re-copy from docker-compose.prod.yml when you want to reset to repo defaults.

Other naming you will see (all fine):

Convention	Notes
`compose.yaml` + `compose.override.yaml`	Newer Docker default; override file gitignored for dev
`docker-compose.staging.yml`	Multi-environment repos
`-f docker-compose.yml -f docker-compose.prod.yml`	Merge two files; less common for single-server ops

For this host:

Layout	Compose source	Server path
DWD API stack (3 services)	`hydrotrek-dwd-suite/docker/docker-compose.prod.yml`	`/etc/dwd-stack/docker-compose.yml`
Single API (generic)	`<api-repo>/` — Dockerfile only; extract one service from stack or write per-api compose	`/etc/<api-name>/docker-compose.yml`

Per-repo docker-compose.yml / docker-compose.prod.yml are not used for the three DWD Python/Node APIs — stack orchestration is centralized in hydrotrek-dwd-suite.

Multi-service stack (`dwd-stack`)

When APIs call each other inside Docker (e.g. Copilot → FastAPI /agent, FastAPI → MCP), use one Compose file in the hydrotrek-dwd-suite super-repo:

hydrotrek-dwd-suite/docker/
    docker-compose.yml          # local dev (submodule build contexts)
    docker-compose.prod.yml     # server → /etc/dwd-stack/docker-compose.yml
    env/*.env.example           # per-service secrets templates
    nginx/dwd-stack-lan.conf    # path-based Nginx for all three APIs

Per-service Dockerfile and .dockerignore remain in each submodule (dwd-api-fastapi, dwd-mcp-fastmcp, dwd-copilot-server). Do not put cross-service compose files in those repos.

Use Compose service names in env vars (http://dwd-api-fastapi:8080/agent), not the host LAN IP or 127.0.0.1 (which refers to the container itself).

Sprint reference: 14-sprint-copilot-mcp-docker.md.

systemd + Docker

Two valid patterns:

A — systemd manages Compose (recommended for consistency with current ops)

ExecStart=/usr/bin/docker compose -f /etc/<api-name>/docker-compose.yml up --remove-orphans
ExecStop=/usr/bin/docker compose -f /etc/<api-name>/docker-compose.yml down

B — Compose restart: unless-stopped only

Simpler, but boot order and systemctl status per API are less uniform.

Use pattern A if you want systemctl restart dwd-api-fastapi to keep working.

Image variants (ML vs slim)

Some APIs (including dwd-api-fastapi) split dependencies:

File	Contents	Image
`requirements.txt`	Core + sklearn fallback	slim — production default
`requirements-ml.txt`	tsai, wandb, PyTorch	ml — larger; use only if TSAI required in prod

Azure already deploys slim-only. Match that unless product requires tsai in production.

Build with Docker multi-stage or build args:

docker build --target slim -t dwd-api-fastapi:slim .
docker build --target ml   -t dwd-api-fastapi:ml .

Local `repos/` folder on this workstation

repos/<git-repo-name>/    # local clone for review, Docker testing, and docs

This is not the server path. The server uses /opt/apis/ (native) or pulls images (Docker). Keep clones here to iterate on Dockerfiles before pushing to git.

Checklist: add Docker to an existing API

First API walkthrough: 06-dockerizing-dwd-api-fastapi.md
Ready-to-copy files: templates/docker/dwd-api-fastapi/ (Dockerfile, .dockerignore)
Native deployment (interim): 02-adding-a-new-api.md

Docker strategy

Short answer​

Side-by-side: native vs Docker​

Recommended rollout phases​

Phase 1 — Native (now)​

Phase 2 — Dockerfile in each repo (next)​

Phase 3 — Docker on server​

Phase 4 — CI builds and registry (optional but standard)​

Host layout with Docker​

Container conventions (all APIs)​

Compose file naming (standard pattern)​

Multi-service stack (dwd-stack)​

systemd + Docker​

Image variants (ML vs slim)​

Local repos/ folder on this workstation​

Checklist: add Docker to an existing API​

Related docs​