Posted on
• Artificial Intelligence

Artificial Intelligence Deployment Automation

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Automating AI Deployment with Bash: From Notebook to Service in Minutes

If your best models live and die in notebooks, you’re leaving value on the table. The real win is getting AI into production reliably—over and over—without the 2 a.m. “it works on my machine” surprises. In this guide, you’ll learn how to turn an AI model into a containerized, self-healing service with a couple of Bash scripts, plus how to schedule batch inference jobs. No heavy platforms required—just Linux, Bash, and containers.

Why automate AI deployment?

  • Consistency and reproducibility: The same container runs on laptops, servers, or the edge. No missing libraries or mismatched drivers.

  • Speed and confidence: One command to build, test, and roll out. Roll back just as fast.

  • Cost control: Batch jobs can be scheduled and turned off; services auto-restart on failure.

  • Governance: Declarative scripts and images are auditable and versioned.

Below you’ll find a minimalist, production-friendly pattern: a containerized FastAPI model server, a one-command deploy script, a systemd service, and a batch inference job.

Prerequisites (Linux)

You can use either Podman (rootless by default) or Docker. Install the basics:

  • Debian/Ubuntu (apt)

    sudo apt update
sudo apt install -y python3 python3-pip git curl jq podman docker.io
# Optional: enable Docker daemon and allow your user to run it
sudo systemctl enable --now docker
sudo usermod -aG docker "$USER"  # re-login or run: newgrp docker

  • Fedora/RHEL/CentOS (dnf)

    sudo dnf -y install python3 python3-pip git curl jq podman moby-engine
# Optional: enable Docker daemon
sudo systemctl enable --now docker
sudo usermod -aG docker "$USER"

  • openSUSE/SLE (zypper)

    sudo zypper refresh
sudo zypper install -y python3 python3-pip git curl jq podman docker
# Optional: enable Docker daemon
sudo systemctl enable --now docker
sudo usermod -aG docker "$USER"

Tip: This article defaults to Podman when available, then falls back to Docker.

1) Containerize your model server

Create a tiny FastAPI app that exposes a /predict endpoint. Replace the “toy model” with your real model load and predict logic.

Project layout:

ai-app/
  Dockerfile
  requirements.txt
  server.py
  .env
  deploy.sh
  batch_infer.sh
  ai-app.service  # optional systemd unit

requirements.txt

fastapi==0.110.*
uvicorn[standard]==0.29.*

server.py

from fastapi import FastAPI
from pydantic import BaseModel
import os

app = FastAPI(title="AI App")

class Item(BaseModel):
    text: str

MODEL_VARIANT = os.getenv("MODEL_VARIANT", "small")

@app.get("/healthz")
def health():
    return {"status": "ok", "variant": MODEL_VARIANT}

@app.post("/predict")
def predict(item: Item):
    # Replace this with your real model inference
    reversed_text = item.text[::-1]
    return {"variant": MODEL_VARIANT, "result": {"reverse": reversed_text, "len": len(item.text)}}

Dockerfile

FROM python:3.11-slim
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
WORKDIR /app

# Non-root user
RUN adduser --disabled-password --gecos "" appuser

# Dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# App
COPY server.py .
USER appuser
EXPOSE 8000

# Healthcheck without curl/wget
HEALTHCHECK --interval=10s --timeout=2s --retries=12 \
  CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/healthz')" || exit 1

CMD ["uvicorn", "server:app", "--host", "0.0.0.0", "--port", "8000"]

.env (configurable at deploy time)

APP_NAME=ai-app
APP_PORT=8000
MODEL_VARIANT=small
IMAGE=ai-app
REGISTRY=

Notes:

  • MODEL_VARIANT demonstrates runtime config via environment variables.

  • For heavy models, bake them into the image or mount them at runtime.

2) One-command build and deploy with Bash

This script chooses Podman if it’s available; otherwise, it uses Docker. It builds, tags, and runs the container with sensible defaults.

deploy.sh

#!/usr/bin/env bash
set -euo pipefail

# Load .env if present
if [ -f ".env" ]; then
  set -a
  . ./.env
  set +a
fi

APP_NAME="${APP_NAME:-ai-app}"
APP_PORT="${APP_PORT:-8000}"
MODEL_VARIANT="${MODEL_VARIANT:-small}"
IMAGE="${IMAGE:-ai-app}"
REGISTRY="${REGISTRY:-}"   # e.g., "registry.example.com/myteam"
CONTAINER_NAME="${APP_NAME}"

# Pick container engine: prefer podman if available
if command -v podman >/dev/null 2>&1; then
  ENG="podman"
elif command -v docker >/dev/null 2>&1; then
  ENG="docker"
else
  echo "Neither podman nor docker found. Install one and retry." >&2
  exit 1
fi

# Generate a tag: git short SHA or timestamp
if command -v git >/dev/null 2>&1 && git rev-parse --is-inside-work-tree >/dev/null 2>&1; then
  TAG="$(git rev-parse --short HEAD)"
else
  TAG="$(date +%Y%m%d%H%M%S)"
fi

IMG="${IMAGE}:${TAG}"
[ -n "$REGISTRY" ] && IMG="${REGISTRY%/}/${IMG}"

echo "Building image: $IMG"
$ENG build -t "$IMG" .

# Stop and remove existing container if running
if $ENG ps -a --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
  echo "Stopping existing container: $CONTAINER_NAME"
  $ENG stop "$CONTAINER_NAME" || true
  echo "Removing existing container: $CONTAINER_NAME"
  $ENG rm "$CONTAINER_NAME" || true
fi

# Run new container
# Docker uses --restart unless-stopped; Podman uses --restart=always
RESTART_FLAG="--restart=always"
if [ "$ENG" = "docker" ]; then
  RESTART_FLAG="--restart=unless-stopped"
fi

echo "Starting container: $CONTAINER_NAME on port ${APP_PORT}"
$ENG run -d \
  --name "$CONTAINER_NAME" \
  -p "${APP_PORT}:8000" \
  -e MODEL_VARIANT="$MODEL_VARIANT" \
  $RESTART_FLAG \
  "$IMG"

# Optional: push to registry (uncomment to use)
# if [ -n "$REGISTRY" ]; then
#   echo "Pushing image to registry: $IMG"
#   $ENG push "$IMG"
# fi

# Health check
echo "Waiting for health..."
for i in $(seq 1 30); do
  if curl -fsS "http://127.0.0.1:${APP_PORT}/healthz" >/dev/null 2>&1; then
    echo "Service is healthy: http://127.0.0.1:${APP_PORT}"
    exit 0
  fi
  sleep 1
done

echo "Service did not become healthy in time." >&2
$ENG logs "$CONTAINER_NAME" || true
exit 1

Use it:

chmod +x deploy.sh
./deploy.sh
curl -s http://127.0.0.1:8000/healthz | jq
curl -s -X POST http://127.0.0.1:8000/predict -H 'content-type: application/json' \
  -d '{"text":"hello world"}' | jq

Real-world tip:

  • Tag images by commit and store them in a registry (REGISTRY=...). Rollbacks are just re-running deploy.sh with an older tag.

3) Run it as a service (systemd)

Keep your model server running across reboots and failures.

ai-app.service

[Unit]
Description=AI App container service
After=network-online.target
Wants=network-online.target

[Service]
Type=oneshot
RemainAfterExit=yes
WorkingDirectory=/opt/ai-app
ExecStart=/opt/ai-app/deploy.sh
ExecStop=/usr/bin/bash -lc '
  if command -v podman >/dev/null 2>&1; then podman stop ai-app && podman rm ai-app || true;
  elif command -v docker >/dev/null 2>&1; then docker stop ai-app && docker rm ai-app || true;
  fi'
TimeoutStartSec=300
TimeoutStopSec=60

[Install]
WantedBy=multi-user.target

Install and enable:

# Place your project in /opt/ai-app (or adjust paths)
sudo mkdir -p /opt/ai-app
sudo cp -r . /opt/ai-app

sudo cp ai-app.service /etc/systemd/system/ai-app.service
sudo systemctl daemon-reload
sudo systemctl enable --now ai-app

# Check status and logs
systemctl status ai-app
journalctl -u ai-app -f

Notes:

  • For rootless Podman services per-user, use systemctl --user and loginctl enable-linger, but the above system unit is simplest for servers.

  • For Docker, ensure the docker daemon is running and your user has the docker group if you’ll manage locally.

4) Automate batch inference (cron + Bash)

You don’t always need a long-running API. For scheduled scoring, run a small script hourly/daily.

batch_infer.sh

#!/usr/bin/env bash
set -euo pipefail

APP_PORT="${APP_PORT:-8000}"
INPUTS=("Hello" "Automate all the things" "Bash + AI")

for text in "${INPUTS[@]}"; do
  RESP="$(curl -fsS -X POST "http://127.0.0.1:${APP_PORT}/predict" \
    -H 'content-type: application/json' \
    -d "$(jq -n --arg t "$text" '{text:$t}')")"
  echo "$(date -Is) | $text | $(echo "$RESP" | jq -c '.result')"
done

Make it executable and schedule it:

chmod +x batch_infer.sh

# Edit crontab (runs hourly)
crontab -e
# Add:
0 * * * * /opt/ai-app/batch_infer.sh >> /var/log/ai-batch.log 2>&1

Real-world examples:

  • Hourly fraud-scoring of recent transactions.

  • Nightly document embeddings for new uploads.

  • Periodic data drift checks and metrics export.

5) Production hardening ideas

  • Observability: add request logging and Prometheus metrics; scrape with node_exporter or use a sidecar.

  • Zero-downtime updates: run deploy.sh with a blue/green or canary container name, then swap ports or a reverse proxy.

  • Secrets: inject via environment variables, files, or your secrets manager (avoid baking secrets into images).

  • GPUs: for NVIDIA, add the container runtime and pass through devices; use framework-specific CUDA images.

Conclusion and next steps

With a few Bash scripts and containers, you can turn models into reliable services or batch jobs—no platform lock-in, no waiting on ops. From here:

  • Extend the API with your real model code and dependencies.

  • Push images to a registry and wire this into CI (GitHub Actions, GitLab CI, Jenkins).

  • Put a TLS reverse proxy (Caddy/NGINX/Traefik) in front of the service.

Want a challenge? Fork this pattern, swap in your model, and ship your first automated deployment today. Your future self (and your users) will thank you.

Related posts