Posted on
• Artificial Intelligence

Artificial Intelligence GitOps Guide

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence GitOps Guide: Ship Models Like Code from Your Terminal

Ever shipped a model that worked flawlessly in your notebook but drifted in production within days? Or struggled to reproduce “the exact same” environment that scored 92% offline but 78% in prod? GitOps solves this with a simple promise: declare your state in Git, let automation reconcile your cluster to match it, and audit everything through commits.

This guide shows how to run AI the GitOps way—entirely from your Linux shell—so you can version, test, deploy, and roll back models as confidently as application code.

Why GitOps belongs in AI

  • Reproducibility: Your training parameters, model artifacts, and deployment manifests live in Git. Rebuild and redeploy deterministically at any commit.

  • Fast rollback and drift control: If latency spikes after a new model, revert the Git commit. The cluster corrects itself—no manual surgery.

  • Reviewable changes: Models and infra changes go through pull requests. You gain peer review, history, and compliance trails.

  • Clear separation of concern: Data scientists focus on artifacts and interfaces; platform teams provide safe templates and policies (Helm/Kustomize).

  • Continuous delivery by default: New model tags update manifests; GitOps controllers roll out, verify, and keep the cluster converged.

Prerequisites: Tools you’ll use

We’ll use:

  • Git for versioning

  • Podman (or Docker) for building containers

  • kubectl and Helm for Kubernetes work

  • Python for a minimal model service

  • Flux for GitOps reconciliation

  • jq for JSON parsing (handy in scripts)

You need a Kubernetes cluster (any distribution; managed or local). If you don’t have one yet, you can still follow along until the “Deploy with Flux” step.

Install on Debian/Ubuntu (apt)

sudo apt-get update
sudo apt-get install -y git curl make jq

# Podman (recommended for rootless builds)
sudo apt-get install -y podman

# Python
sudo apt-get install -y python3 python3-venv python3-pip

# kubectl (official repo)
sudo apt-get install -y apt-transport-https ca-certificates gnupg
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key \
  | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /" \
  | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubectl

# Helm (official repo)
curl -fsSL https://baltocdn.com/helm/signing.asc \
  | sudo gpg --dearmor -o /usr/share/keyrings/helm.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/helm.gpg] https://baltocdn.com/helm/stable/debian/ all main" \
  | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list
sudo apt-get update
sudo apt-get install -y helm

Install on Fedora/RHEL (dnf)

sudo dnf install -y git curl make jq podman python3 python3-pip

# kubectl (official repo)
sudo dnf install -y 'dnf-command(config-manager)'
sudo dnf config-manager --add-repo https://pkgs.k8s.io/core:/stable:/v1.30/rpm/
sudo rpm --import https://pkgs.k8s.io/core:/stable:/v1.30/rpm/repodata/repomd.xml.key
sudo dnf install -y kubectl

# Helm (official repo)
sudo rpm --import https://baltocdn.com/helm/signing.asc
sudo tee /etc/yum.repos.d/helm.repo <<'EOF'
[helm]
name=Helm
baseurl=https://baltocdn.com/helm/stable/rpm
enabled=1
gpgcheck=1
gpgkey=https://baltocdn.com/helm/signing.asc
EOF
sudo dnf install -y helm

Install on openSUSE (zypper)

sudo zypper refresh
sudo zypper install -y git curl make jq podman python3 python3-pip

# kubectl (official repo)
sudo rpm --import https://pkgs.k8s.io/core:/stable:/v1.30/rpm/repodata/repomd.xml.key
sudo zypper addrepo https://pkgs.k8s.io/core:/stable:/v1.30/rpm/ kubernetes
sudo zypper refresh
sudo zypper install -y kubectl

# Helm (official repo)
sudo rpm --import https://baltocdn.com/helm/signing.asc
sudo zypper addrepo https://baltocdn.com/helm/stable/rpm helm
sudo zypper refresh
sudo zypper install -y helm

Flux CLI (all distros)

curl -s https://fluxcd.io/install.sh | sudo bash
# or without sudo to install in ~/.local/bin:
# curl -s https://fluxcd.io/install.sh | bash

A minimal, practical AI GitOps flow (5 steps)

We’ll containerize a tiny FastAPI service that loads a toy scikit-learn model, push its image to a registry, commit K8s manifests to Git, and let Flux reconcile the cluster.

1) Create a repo and scaffold files

mkdir -p ai-gitops/apps/ai-svc/manifests
cd ai-gitops
git init

Project layout:

ai-gitops/
  apps/ai-svc/
    app/          # Python service (model server)
    manifests/    # Kubernetes YAMLs deployed by Flux

2) Build a simple model service

Create apps/ai-svc/app/app.py:

from fastapi import FastAPI
from pydantic import BaseModel
import joblib
import os

MODEL_PATH = os.environ.get("MODEL_PATH", "/model/model.joblib")
app = FastAPI()
model = joblib.load(MODEL_PATH)

class Item(BaseModel):
    x1: float
    x2: float

@app.get("/healthz")
def healthz():
    return {"status": "ok"}

@app.post("/predict")
def predict(item: Item):
    import numpy as np
    X = np.array([[item.x1, item.x2]])
    y = model.predict_proba(X)[0].tolist()
    return {"proba": y}

Create apps/ai-svc/app/requirements.txt:

fastapi==0.111.0
uvicorn[standard]==0.30.0
scikit-learn==1.5.0
joblib==1.4.2
numpy==1.26.4

Create apps/ai-svc/app/Dockerfile:

FROM python:3.11-slim

ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Train a tiny model at build-time (toy example for reproducibility)
RUN python - <<'PY'
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
import joblib, os
X, y = make_classification(n_samples=200, n_features=2, n_informative=2, n_redundant=0, random_state=42)
clf = LogisticRegression().fit(X, y)
os.makedirs("/model", exist_ok=True)
joblib.dump(clf, "/model/model.joblib")
PY

COPY app.py .
EXPOSE 8080
CMD ["uvicorn", "app:app", "--host=0.0.0.0", "--port=8080"]

Build and test locally:

cd apps/ai-svc/app
podman build -t registry.example.com/yourname/ai-svc:0.1.0 .
podman run --rm -p 8080:8080 registry.example.com/yourname/ai-svc:0.1.0
# In another terminal:
curl -s http://localhost:8080/healthz
curl -s -X POST http://localhost:8080/predict -H 'content-type: application/json' \
  -d '{"x1":0.1,"x2":-0.2}'

Push to your registry (replace URL with your own; e.g., GHCR, Docker Hub, Harbor):

podman login registry.example.com
podman push registry.example.com/yourname/ai-svc:0.1.0

Tip: Tag models with semver matching your model version (e.g., 0.1.0 → 0.1.1).

3) Declare Kubernetes manifests in Git

Create apps/ai-svc/manifests/namespace.yaml:

apiVersion: v1
kind: Namespace
metadata:
  name: ai

Create apps/ai-svc/manifests/deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ai-svc
  namespace: ai
spec:
  replicas: 2
  selector:
    matchLabels:
      app: ai-svc
  template:
    metadata:
      labels:
        app: ai-svc
    spec:
      containers:
      - name: api
        image: registry.example.com/yourname/ai-svc:0.1.0
        imagePullPolicy: IfNotPresent
        env:
        - name: MODEL_PATH
          value: /model/model.joblib
        ports:
        - containerPort: 8080
        readinessProbe:
          httpGet:
            path: /healthz
            port: 8080
          initialDelaySeconds: 3
          periodSeconds: 5
        resources:
          requests:
            cpu: "100m"
            memory: "128Mi"
          limits:
            cpu: "500m"
            memory: "512Mi"

Create apps/ai-svc/manifests/service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: ai-svc
  namespace: ai
spec:
  selector:
    app: ai-svc
  ports:
  - port: 80
    targetPort: 8080

Commit and push:

cd ../../../
git add .
git commit -m "AI svc: model service, K8s manifests v0.1.0"
git branch -M main
git remote add origin git@github.com:YOURUSER/ai-gitops.git
git push -u origin main

4) Connect your cluster with Flux (GitOps)

Install Flux controllers in your cluster:

kubectl create namespace flux-system || true
flux install
kubectl -n flux-system get deployments

Tell Flux where your Git repo and manifests live. Create clusters/production/flux.yaml:

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: ai-gitops
  namespace: flux-system
spec:
  interval: 1m
  url: https://github.com/YOURUSER/ai-gitops
  ref:
    branch: main
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: ai-svc
  namespace: flux-system
spec:
  interval: 1m
  targetNamespace: ai
  prune: true
  sourceRef:
    kind: GitRepository
    name: ai-gitops
  path: ./apps/ai-svc/manifests
  wait: true

Apply it:

kubectl apply -f clusters/production/flux.yaml

Within a minute, Flux will:

  • Create the ai namespace

  • Deploy the ai-svc Deployment and Service

Verify:

kubectl -n ai get pods,svc
kubectl -n ai port-forward svc/ai-svc 8080:80 &
curl -s http://localhost:8080/healthz

Real-world change flow:

  • Edit apps/ai-svc/manifests/deployment.yaml to image: ...:0.1.1

  • Commit and push to main

  • Flux detects the commit and rolls out the update automatically

If the new model regresses, revert the commit. Flux reconciles the cluster back to the good version.

Optional: Flux “bootstrap” with GitHub can manage the entire cluster config from Git. If you prefer that model, export GITHUB_TOKEN and run:

export GITHUB_TOKEN=ghp_your_token_here
flux bootstrap github --owner=YOURUSER --repository=ai-gitops --branch=main --path=clusters/production

5) Automate image builds from tags

Simple Makefile to standardize builds:

Create Makefile at repo root:

REGISTRY ?= registry.example.com/yourname
IMAGE ?= $(REGISTRY)/ai-svc
TAG ?= 0.1.0

build:
    cd apps/ai-svc/app && podman build -t $(IMAGE):$(TAG) .

push:
    podman push $(IMAGE):$(TAG)

release: build push
    @echo "Released $(IMAGE):$(TAG)"

Usage:

make TAG=0.1.1 release
# then bump the image tag in apps/ai-svc/manifests/deployment.yaml and git commit/push

You can later add CI (e.g., GitHub Actions) to run make release on tags, sign images, scan, and open a PR to bump the manifest tag.

Production-minded tips

  • Separate config/data from code:

    • Store large model artifacts in a registry or object store. Reference immutable tags or digests in manifests.

  • Use image digests, not tags, for immutability:

    • image: registry/...@sha256:...

  • Add canaries and progressive delivery:

    • Use tools like Flagger with Flux to route a percentage of traffic to the new model and roll back on metrics.

  • Manage secrets safely:

    • Pair GitOps with sealed-secrets or external-secrets; never commit raw credentials.

  • Observe everything:

    • Emit metrics (latency, error rate, drift signals) and link rollouts to SLO dashboards.

Conclusion and next steps

GitOps gives AI teams a reliable path from model to production: every change is declarative, audited, and reversible. You just: 1) Build and tag a model image 2) Update a manifest in Git 3) Let Flux handle the rest

Your next step:

  • Stand up a branch called “staging,” replicate the Flux Kustomization to clusters/staging, and practice promoting a model from staging → production via pull requests. When you’re ready, wire in CI to build images on tags and open automated PRs that bump image digests.

If you want a follow-up post with canary deployments, model metrics, and automated rollbacks, tell me which stack you’re using (cloud, registry, cluster) and I’ll tailor the scripts.

Related posts