Posted on
• Artificial Intelligence

Artificial Intelligence Docker Automation on Linux

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence Docker Automation on Linux (with Bash)

If your AI experiments only work on one machine, break after every OS update, or require a 300‑line “setup notes” file, this post is for you. Docker plus a pinch of Bash can turn fragile AI workflows into repeatable, portable, and schedulable pipelines you can run on any Linux box—from your laptop to a headless GPU server.

This article explains why containerizing AI is worth it and gives you a practical, Linux‑first playbook to automate training and inference jobs. You’ll install the right tooling (apt, dnf, zypper covered), scaffold a minimal AI container, wire it up with Bash and Docker Compose, schedule it with systemd timers, and harden/optimize it for real workloads.

Why Docker for AI on Linux?

  • Reproducibility: Pin Python, CUDA, cuDNN, and library versions in a Dockerfile. No more “works on my machine.”

  • Portability: Run the same container on dev laptops, CI agents, or GPU servers without re‑installing toolchains.

  • Isolation: Avoid polluting system Python or clobbering GPU drivers. Containers keep dependencies scoped.

  • Automation: Bash scripts + Docker Compose profiles + systemd timers = hands‑off, reliable AI jobs.

  • Performance: Use NVIDIA Container Toolkit to pass GPUs to containers with near‑native speed.

1) Install Docker, Compose, and optional NVIDIA GPU support

Install Docker Engine and the Docker Compose plugin. Then (optionally) enable GPU passthrough.

Verify your distro/package manager and run the matching block.

Docker Engine + Compose

Apt (Ubuntu/Debian):

sudo apt update
sudo apt install -y ca-certificates curl gnupg lsb-release
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo $ID)/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/$(. /etc/os-release; echo $ID) $(
    . /etc/os-release; echo $VERSION_CODENAME
  ) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker $USER
newgrp docker
sudo systemctl enable --now docker
docker run --rm hello-world

DNF (Fedora/RHEL derivatives with DNF):

sudo dnf -y install dnf-plugins-core
sudo dnf config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo
sudo dnf -y install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker $USER
newgrp docker
sudo systemctl enable --now docker
docker run --rm hello-world

Zypper (openSUSE Leap/Tumbleweed or SLES):

sudo zypper refresh
# Prefer the distro packages for simplicity:
sudo zypper install -y docker docker-compose
sudo usermod -aG docker $USER
newgrp docker
sudo systemctl enable --now docker
docker run --rm hello-world
# If your distro provides the v2 plugin, this may also work:
sudo zypper install -y docker-compose-plugin || true

Check Docker Compose:

docker compose version
# Fallback (older distros):
docker-compose version

Optional: NVIDIA Container Toolkit (GPU)

Prerequisites: install the proprietary NVIDIA driver on the host first and reboot. Then:

Apt (Ubuntu/Debian):

distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -fsSL https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update
sudo apt install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
# Test:
docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

DNF (Fedora/RHEL derivatives with DNF):

distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /etc/pki/rpm-gpg/NVIDIA-CONTAINER-TOOLKIT.gpg
curl -fsSL https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.repo | \
  sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
sudo dnf -y install nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubi8 nvidia-smi

Zypper (openSUSE/SLES):

distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -fsSL https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.repo | \
  sudo tee /etc/zypp/repos.d/nvidia-container-toolkit.repo
sudo zypper refresh
sudo zypper install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

2) Scaffold a minimal AI container (CPU or GPU)

Create a project layout:

ai-docker/
├─ Dockerfile
├─ requirements.txt
├─ src/
│  └─ train.py
├─ scripts/
│  ├─ train.sh
│  └─ infer.sh
└─ data/          # bind-mounted datasets

requirements.txt (example: transformers + PyTorch CPU)

torch==2.3.1
transformers==4.42.4
datasets==2.20.0
accelerate==0.33.0

Dockerfile (CPU baseline; switch base image to GPU later):

# CPU base
FROM python:3.11-slim AS base

# System deps
RUN apt-get update && apt-get install -y --no-install-recommends \
    git build-essential curl ca-certificates && \
    rm -rf /var/lib/apt/lists/*

# Non-root user for safety
ARG UID=1000
ARG GID=1000
RUN groupadd -g $GID app && useradd -m -u $UID -g $GID app
WORKDIR /app

# Python deps
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Cache models/datasets in a volume for speed
ENV HF_HOME=/models \
    HF_DATASETS_CACHE=/datasets \
    PYTHONUNBUFFERED=1

COPY src/ ./src/
COPY scripts/ ./scripts/
RUN chmod +x scripts/*.sh

USER app
ENTRYPOINT ["bash"]

GPU variant (replace the first line if you have NVIDIA set up):

FROM pytorch/pytorch:2.3.1-cuda11.8-cudnn8-runtime
# then keep the same steps as above (install git/build tools via apt, add user, etc.)

Example train.py:

import os
from datasets import load_dataset
from transformers import AutoTokenizer, AutoModelForSequenceClassification, Trainer, TrainingArguments

model_name = os.environ.get("MODEL_NAME", "distilbert-base-uncased")
dataset_name = os.environ.get("HF_DATASET", "imdb")

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)
ds = load_dataset(dataset_name)

def tok(batch):
    return tokenizer(batch["text"], truncation=True, padding="max_length", max_length=128)

tok_ds = ds.map(tok, batched=True)
args = TrainingArguments(
    output_dir="/artifacts",
    per_device_train_batch_size=int(os.environ.get("BATCH", "8")),
    num_train_epochs=float(os.environ.get("EPOCHS", "1")),
    evaluation_strategy="epoch",
    logging_steps=50,
    save_strategy="epoch",
)

trainer = Trainer(model=model, args=args, train_dataset=tok_ds["train"], eval_dataset=tok_ds["test"])
trainer.train()
trainer.save_model("/artifacts/model")
print("Training complete.")

scripts/train.sh:

#!/usr/bin/env bash
set -euo pipefail
export MODEL_NAME="${MODEL_NAME:-distilbert-base-uncased}"
export HF_DATASET="${HF_DATASET:-imdb}"
export BATCH="${BATCH:-8}"
export EPOCHS="${EPOCHS:-1}"
python -u src/train.py

Build and test:

docker build -t ai:latest .
docker run --rm -it \
  -v "$(pwd)/data:/data" \
  -v "$(pwd)/artifacts:/artifacts" \
  -v "models:/models" \
  -v "datasets:/datasets" \
  ai:latest -lc "scripts/train.sh"

  • Named volumes “models” and “datasets” keep Hugging Face caches outside the container so repeated runs are fast.

  • Bind mounts “data” and “artifacts” give you local access to inputs/outputs.

GPU test (if enabled):

docker run --rm -it --gpus all \
  -v "$(pwd)/artifacts:/artifacts" \
  -v "models:/models" -v "datasets:/datasets" \
  ai:latest -lc "nvidia-smi && scripts/train.sh"

3) Automate with Docker Compose, Bash, and Make

Compose makes multi‑step or multi‑service jobs straightforward.

docker-compose.yml:

name: ai-stack
services:
  trainer:
    image: ai:latest
    build:
      context: .
    command: ["bash", "-lc", "scripts/train.sh"]
    env_file: [.env]
    environment:
      - MODEL_NAME=${MODEL_NAME:-distilbert-base-uncased}
      - HF_DATASET=${HF_DATASET:-imdb}
      - BATCH=${BATCH:-8}
      - EPOCHS=${EPOCHS:-1}
    volumes:
      - ./artifacts:/artifacts
      - models:/models
      - datasets:/datasets
    deploy:
      resources:
        limits:
          cpus: "4"
          memory: 8g
    profiles: ["train"]
    # For GPU, uncomment:
    # deploy: { resources: { reservations: { devices: [ { capabilities: ["gpu"] } ] } } }
    # Or use `--gpus all` via compose extensions if supported.

  infer:
    image: ai:latest
    command: ["bash", "-lc", "scripts/infer.sh"]
    env_file: [.env]
    volumes:
      - ./inbox:/inbox
      - ./out:/out
      - models:/models
      - datasets:/datasets
    profiles: ["infer"]

volumes:
  models:
  datasets:

.env:

MODEL_NAME=distilbert-base-uncased
HF_DATASET=imdb
BATCH=8
EPOCHS=1

scripts/infer.sh (example stub):

#!/usr/bin/env bash
set -euo pipefail
echo "Running batch inference..."
# Your inference script would read from /inbox and write to /out

Makefile for convenience:

.PHONY: build train infer logs clean

build:
    docker compose build

train:
    docker compose --profile train up --build --abort-on-container-exit

infer:
    docker compose --profile infer up --build --abort-on-container-exit

logs:
    docker compose logs -f

clean:
    docker compose down -v --remove-orphans

Run:

make build
make train

Real‑world examples:

  • Nightly fine‑tune: Set EPOCHS=2, profile=train, store artifacts in ./artifacts, schedule with systemd (next section).

  • Batch inference: Drop files into ./inbox, run make infer, results land in ./out.

  • Multi‑host dev/prod: docker context create prod --docker "host=ssh://user@server" then docker --context prod compose up to run remotely.

4) Schedule and monitor with systemd timers

Automating periodic jobs on headless servers is easy with systemd.

ai-train.service:

[Unit]
Description=AI Training (Docker Compose)

[Service]
Type=oneshot
WorkingDirectory=/path/to/ai-docker
ExecStart=/usr/bin/docker compose --profile train up --build --abort-on-container-exit
ExecStop=/usr/bin/docker compose down
TimeoutStartSec=0

[Install]
WantedBy=multi-user.target

ai-train.timer:

[Unit]
Description=Run AI training nightly

[Timer]
OnCalendar=02:00
Persistent=true
Unit=ai-train.service

[Install]
WantedBy=timers.target

Enable and test:

sudo cp ai-train.service /etc/systemd/system/
sudo cp ai-train.timer /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now ai-train.timer
systemctl list-timers | grep ai-train
# On-demand:
sudo systemctl start ai-train.service
# Logs:
journalctl -u ai-train.service -f

Tip: Rotate Docker logs and prune regularly:

docker system prune -f --volumes

For retention, prefer:

docker image prune -f
docker volume prune -f

5) Secure and optimize your AI containers

  • Non‑root by default: The Dockerfile creates and uses a non‑root “app” user; keep it that way.

  • Pin versions: Lock base images, CUDA versions, and pip packages to prevent unplanned upgrades.

  • Resource limits: Use Compose deploy.resources to avoid overcommitting CPU/RAM.

  • BuildKit and caching: Speed builds and use a registry mirror.

mkdir -p ~/.docker
cat <<'EOF' > ~/.docker/config.json
{
  "features": { "buildkit": true },
  "proxies": {},
  "credsStore": "desktop" 
}
EOF
export DOCKER_BUILDKIT=1
  • Secrets and envs: Keep credentials out of images. Use --env-file .env or Docker secrets.
# .env (non-secret)
HF_HOME=/models

# secrets/api_key.txt (git-ignored)
# then in compose:
# secrets:
#   api_key:
#     file: ./secrets/api_key.txt
# services:
#   trainer:
#     secrets: [api_key]
  • GPU safety: Limit device access if multiple GPUs:
docker run --rm --gpus '"device=1,2"' ai:latest nvidia-smi

  • Persist caches: Named volumes for model/data caches massively cut re‑download time.

  • Healthchecks: Add to Compose for long‑running services.

healthcheck:
  test: ["CMD", "python", "-c", "import torch; print('ok')"]
  interval: 30s
  timeout: 5s
  retries: 3

Conclusion and next steps

You now have a reliable pattern to containerize, automate, and schedule AI workloads on Linux using nothing more than Docker, Bash, and systemd. This setup eliminates environment drift, accelerates iteration, and makes production rollouts boring—in a good way.

Your next steps:

  • Clone your current AI project into this scaffold and containerize it.

  • Turn your most frequent manual steps into scripts/*.sh and Compose profiles.

  • Add a systemd timer for nightly training or weekly inference sweeps.

  • If you have GPUs, enable NVIDIA Container Toolkit and switch to a CUDA base image.

Want more? Extend this with a model API (FastAPI + Uvicorn), a metrics stack (Prometheus + Grafana), or a CI pipeline that builds and pushes your images on every commit. Your AI pipeline just graduated from “works on my laptop” to “works anywhere.”

Related posts