The AI Deployment Checklist (Linux + Bash Edition)

Deploying an AI model to production isn’t just “pip install” and vibes. It’s a sequence of repeatable checks that keep latency low, errors rare, and 3 a.m. pages off your phone. This guide gives you a practical, Bash-first checklist to package, secure, observe, and roll out AI services on Linux—complete with commands you can paste today.

What you’ll get:

• Why deployment checklists matter for AI workloads

• A preflight Bash script to validate your host

• A minimal, containerized AI service (FastAPI) you can harden and ship

• Secure defaults (non-root, firewalls, resource limits)

• Observability, service management, and a safe rollout pattern

---

Why a checklist for AI deploys?

AI workloads compound typical web-service risks:

• Heavy dependencies and native libs (breakage risk)

• Large models that change performance and memory behavior

• GPU and CPU heterogeneity across hosts

• Tight SLOs, plus security and data governance requirements

A simple, consistent checklist makes deploys boring—in a good way.

---

0) Install the essentials

We’ll use these baseline tools: git, curl, jq, Python 3 + pip/venv, Podman (container runtime), skopeo (image inspection/copy), nginx (edge proxy), plus a few ops-friendly goodies.

• Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y git curl jq python3 python3-venv python3-pip podman skopeo nginx htop tmux ufw

• Fedora/RHEL family (dnf):

sudo dnf install -y git curl jq python3 python3-pip podman skopeo nginx htop tmux firewalld

• openSUSE/SLES (zypper):

sudo zypper refresh
sudo zypper install -y git curl jq python3 python3-pip podman skopeo nginx htop tmux firewalld

Tip: If “nginx” isn’t in your repo (some RHEL/SLES variants), enable your distribution’s optional repos (e.g., EPEL on RHEL) first.

---

1) Preflight: baseline your host

Run this Bash script to verify OS, kernel, CPU flags, memory, disk, container runtime, cgroups, and port availability. Save as ai-preflight.sh and execute.

#!/usr/bin/env bash
set -euo pipefail

need() { command -v "$1" >/dev/null 2>&1 || { echo "Missing dependency: $1" >&2; exit 1; }; }

echo "=== AI Deployment Preflight ==="

# OS and kernel
if [ -f /etc/os-release ]; then
  . /etc/os-release
  echo "OS: $PRETTY_NAME"
else
  echo "OS: unknown"
fi
echo "Kernel: $(uname -r)"
echo "Arch: $(uname -m)"

# CPU and memory
echo "CPUs: $(nproc)"
mem_kb=$(awk '/MemTotal/ {print $2}' /proc/meminfo)
printf "Memory: %.2f GiB\n" "$(echo "$mem_kb/1024/1024" | bc -l)"

# Disk space (root)
echo "Disk (root):"
df -hT / | awk 'NR==1 || NR==2 {print}'

# Cgroups v2 check
if [ -f /sys/fs/cgroup/cgroup.controllers ]; then
  echo "Cgroups: v2 (OK)"
else
  echo "Cgroups: v1 (consider migrating to v2 for better isolation)"
fi

# Container runtime
if command -v podman >/dev/null 2>&1; then
  echo "Podman: $(podman --version)"
else
  echo "Podman: not installed"
fi

# GPU presence (optional)
if command -v nvidia-smi >/dev/null 2>&1; then
  echo "GPU: NVIDIA detected"
  nvidia-smi -L || true
else
  echo "GPU: none detected or driver not installed"
fi

# Python
if command -v python3 >/dev/null 2>&1; then
  echo "Python: $(python3 --version)"
  echo "Pip: $(python3 -m pip --version || echo 'pip missing')"
else
  echo "Python: not installed"
fi

# Ports
check_port() {
  local port=$1
  if ss -ltn "( sport = :$port )" | grep -q ":$port"; then
    echo "Port $port: in use"
  else
    echo "Port $port: available"
  fi
}
check_port 8000
check_port 80
check_port 443

echo "Preflight complete."

Run it:

chmod +x ai-preflight.sh
./ai-preflight.sh

Checklist:

• [ ] Kernel and libc are consistent across nodes

• [ ] Cgroups v2 for modern limits/isolation

• [ ] Port plan (e.g., app on 127.0.0.1:8000; nginx exposes 80/443)

• [ ] GPU availability matches model/runtime requirements

---

2) Package the model into a minimal service

We’ll ship a simple FastAPI app that can load a model and expose:

• GET /healthz (liveness)

• POST /predict (inference)

Project layout:

ai-service/
  ├─ requirements.txt
  ├─ server.py
  └─ Dockerfile

requirements.txt:

fastapi==0.111.0
uvicorn[standard]==0.30.0
onnxruntime==1.18.0

server.py (replace the “fake inference” with your real model):

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import os

app = FastAPI(title="AI Service", version="1.0.0")

class InferenceRequest(BaseModel):
    text: str

@app.get("/healthz")
def healthz():
    return {"status": "ok"}

@app.post("/predict")
def predict(req: InferenceRequest):
    # Example: replace with ONNXRuntime session inference
    # import onnxruntime as ort
    # session = ort.InferenceSession(os.getenv("MODEL_PATH", "model.onnx"))
    # result = session.run(None, {"input": preprocess(req.text)})
    # return {"result": postprocess(result)}

    if not req.text:
        raise HTTPException(status_code=400, detail="No text")
    score = float(len(req.text))  # stub
    return {"score": score}

Dockerfile (builds a non-root image):

FROM python:3.11-slim

# Non-root user
RUN useradd -m -u 1000 app
WORKDIR /app

# System deps (optional minimal tools)
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates && rm -rf /var/lib/apt/lists/*

# Python deps
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# App
COPY server.py .

# Healthcheck (expecting 200)
HEALTHCHECK --interval=30s --timeout=3s --retries=3 CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/healthz').read()"

EXPOSE 8000
USER app
CMD ["python", "-m", "uvicorn", "server:app", "--host", "0.0.0.0", "--port", "8000"]

Build and run with Podman:

cd ai-service
podman build -t localhost/ai-service:1.0 .
podman run --rm -p 8000:8000 --read-only --tmpfs /tmp:rw,size=64m,uid=1000,gid=1000 \
  --memory=4g --cpus=2 \
  --user 1000:1000 \
  localhost/ai-service:1.0

Check it:

curl -s localhost:8000/healthz
curl -s -X POST localhost:8000/predict -H 'content-type: application/json' -d '{"text":"hello world"}'

Inspect the image (signature/metadata) with skopeo:

skopeo inspect containers-storage:localhost/ai-service:1.0 | jq .

Checklist:

• [ ] Container runs as non-root (UID/GID 1000)

• [ ] Read-only root FS, temp space via tmpfs

• [ ] Resource limits set (CPU, memory)

• [ ] Healthcheck baked into image

---

3) Security and isolation basics

Run as non-root and lock down the runtime:

podman run --rm --name ai-service \
  --read-only --tmpfs /tmp:rw,size=64m,uid=1000,gid=1000 \
  --cap-drop=ALL --pids-limit=512 \
  --user 1000:1000 \
  -e MODEL_PATH=/models/model.onnx \
  -v /srv/ai/models:/models:ro \
  -p 127.0.0.1:8000:8000 \
  localhost/ai-service:1.0

Firewall the host:

• Debian/Ubuntu (ufw):

sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status

• Fedora/RHEL/openSUSE (firewalld):

sudo systemctl enable --now firewalld
sudo firewall-cmd --add-service=http --permanent
sudo firewall-cmd --add-service=https --permanent
sudo firewall-cmd --reload
sudo firewall-cmd --list-all

Minimum file permissions:

sudo install -d -o 1000 -g 1000 -m 0750 /srv/ai/models
sudo chown -R 1000:1000 /srv/ai

Checklist:

• [ ] Non-root container, minimal Linux capabilities

• [ ] Root filesystem read-only; only explicit volumes mounted

• [ ] Firewall exposes only required ports

• [ ] Secrets via env files or mounted files, not baked into images

---

4) Observability and service management

Run the service under systemd for auto-restart and lifecycle control, and put nginx in front for TLS/edge.

Systemd unit for Podman (save as /etc/systemd/system/ai-service.service):

[Unit]
Description=AI Service (Podman)
After=network-online.target
Wants=network-online.target

[Service]
Restart=always
TimeoutStopSec=15
ExecStartPre=-/usr/bin/podman rm -f ai-service
ExecStart=/usr/bin/podman run --name ai-service \
  --log-driver=journald \
  --read-only --tmpfs /tmp:rw,size=64m,uid=1000,gid=1000 \
  --cap-drop=ALL --pids-limit=512 \
  --user 1000:1000 \
  --memory=4g --cpus=2 \
  -e MODEL_PATH=/models/model.onnx \
  -v /srv/ai/models:/models:ro \
  -p 127.0.0.1:8000:8000 \
  localhost/ai-service:1.0
ExecStop=/usr/bin/podman stop -t 10 ai-service
ExecStopPost=/usr/bin/podman rm -f ai-service

[Install]
WantedBy=multi-user.target

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable --now ai-service
sudo systemctl status ai-service
journalctl -u ai-service -f

Nginx reverse proxy (save as /etc/nginx/conf.d/ai.conf):

upstream ai_backend {
    server 127.0.0.1:8000;
}

server {
    listen 80;
    server_name _;

    location / {
        proxy_pass http://ai_backend;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Request-Id $request_id;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_read_timeout 60s;
    }
}

Apply and verify:

sudo nginx -t
sudo systemctl reload nginx
curl -I http://127.0.0.1/healthz

Baseline SLO checks:

• [ ] p50/p95 latency at expected QPS (benchmark with wrk or hey)

• [ ] Error rate < target threshold

• [ ] CPU/memory headroom ≥ 30% under peak test

• [ ] Logs include request IDs and status codes

---

5) Rollout, canary, and rollback

Use tags and indirection to make switches safe.

• Pull new version:

podman pull registry.example.com/ai-service:1.1

• Run canary on a different port via systemd or a second unit (e.g., ai-service-canary.service on 127.0.0.1:8001), then weight traffic in nginx:

upstream ai_backend {
    server 127.0.0.1:8000 weight=90;
    server 127.0.0.1:8001 weight=10;
}

sudo nginx -t && sudo systemctl reload nginx

• Promote or rollback by changing weights or stopping the canary unit:

sudo systemctl stop ai-service-canary
# or swap weights to 100/0 and reload nginx

• Keep artifacts immutable and versioned:

skopeo copy docker://registry.example.com/ai-service:1.1 dir:/var/cache/images/ai-service-1.1

Checklist:

• [ ] Immutable image tags mapped to change tickets

• [ ] Canary receives real traffic before promotion

• [ ] One-command rollback documented and tested

• [ ] Config and model files are versioned and backed up

---

Real-world tips

• Warm-up: Load the model and run a few inference calls at startup so the first user doesn’t pay JIT/cold costs.

• Pin CPU/GPU: Ensure the same instruction set/GPU compute capability across nodes for consistent performance.

• Small base images: Prefer slim Python + wheels built for your arch; avoid building from scratch on prod nodes.

• Healthchecks: Distinguish liveness (/healthz) vs readiness (/readyz that verifies the model is loaded).

---

Conclusion / Call to Action

You now have a pragmatic, Bash-first AI deployment checklist: preflight your host, containerize with secure defaults, put a stable proxy in front, instrument for SLOs, and roll out safely with a canary and a rollback plan.

Next steps: 1) Copy this article into your repo as DEPLOYMENT.md.
2) Adapt the preflight script and systemd unit to your environment.
3) Run a canary deploy of your next model using the nginx weight pattern above.

Ship boring. Sleep better. And if you found this useful, share it with your team and make checklists non-negotiable for AI deploys.