Posted on
• Artificial Intelligence

Artificial Intelligence Platform Engineering

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence Platform Engineering on Linux: From Notebook to Production, the Bash Way

You have a promising model in a notebook. But as soon as you try to turn it into a product, the build breaks, environments drift, training isn’t reproducible, and costs spiral. Sound familiar? That’s the gap AI Platform Engineering closes: a disciplined way to build, ship, run, and observe AI systems across your team and infrastructure.

This article shows you, from a Linux Bash perspective, how to stand up a practical, minimal AI platform pattern you can use today: containerized runtimes, experiment tracking, CI/CD, and observability—using tools you can install with your distro’s package manager and run from your terminal.

Why AI Platform Engineering matters

  • Reproducibility: The same code, data, and runtime must produce the same results everywhere.

  • Speed with safety: Fast iteration through automation, but with guardrails (tests, reviews, policies).

  • Cost control: Right-size compute and release engineering so you’re paying for outcomes, not waste.

  • Compliance and security: Standardized images, dependency scanning, audit trails.

  • Operability: Clear health signals and metrics so you can trust model behavior in production.

Prerequisites: install core tools

We’ll use Podman for rootless containers (works similarly to Docker), Git for versioning, Python/pip for local ML tasks, and curl for quick tests.

Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y podman git python3 python3-pip curl

Fedora/RHEL/CentOS Stream (dnf):

sudo dnf install -y podman git python3 python3-pip curl

openSUSE Leap/Tumbleweed (zypper):

sudo zypper refresh
sudo zypper install -y podman git python3 python3-pip curl

Verify:

podman --version
git --version
python3 --version
pip3 --version

Tip: If you prefer Docker, you can substitute docker for podman in the commands below.

1) Standardize runtimes with containers

We’ll create a tiny model service with FastAPI, trained on the Iris dataset. First, train a minimal model and save it as an artifact:

Create train.py:

cat > train.py << 'EOF'
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
import joblib

X, y = load_iris(return_X_y=True)
clf = LogisticRegression(max_iter=200).fit(X, y)
joblib.dump(clf, "model.joblib")
print("Saved model.joblib")
EOF

Train the model in an isolated container:

podman run --rm -v "$PWD":/work -w /work python:3.11-slim \
  bash -lc "pip install --no-cache-dir scikit-learn joblib && python train.py"

Create a FastAPI app that loads the saved model, exposes health, prediction, and Prometheus metrics:

Create app.py:

cat > app.py << 'EOF'
from fastapi import FastAPI, Response
from pydantic import BaseModel
import joblib, numpy as np, time
from prometheus_client import Counter, Histogram, generate_latest, CONTENT_TYPE_LATEST

api = FastAPI()
REQUESTS = Counter('inference_requests_total', 'Total inference requests')
LATENCY = Histogram('inference_latency_seconds', 'Latency per inference seconds')

class Item(BaseModel):
    features: list[float]

model = joblib.load("model.joblib")

@api.get("/healthz")
def healthz():
    return {"status": "ok"}

@api.post("/predict")
def predict(item: Item):
    REQUESTS.inc()
    start = time.time()
    x = np.array(item.features).reshape(1, -1)
    y = model.predict(x).tolist()
    LATENCY.observe(time.time() - start)
    return {"prediction": y}

@api.get("/metrics")
def metrics():
    return Response(content=generate_latest(), media_type=CONTENT_TYPE_LATEST)
EOF

Define a container image:

Create Dockerfile:

cat > Dockerfile << 'EOF'
FROM python:3.11-slim

# Minimal system deps for science stacks can improve reliability
RUN pip install --no-cache-dir fastapi uvicorn[standard] scikit-learn joblib prometheus-client pydantic

WORKDIR /app
COPY model.joblib /app/model.joblib
COPY app.py /app/app.py

EXPOSE 8080
CMD ["uvicorn", "app:api", "--host", "0.0.0.0", "--port", "8080"]
EOF

Build and run:

podman build -t ai-platform/api:0.1 .
podman run --rm -p 8080:8080 ai-platform/api:0.1

In a second terminal, test the service:

curl -s http://localhost:8080/healthz
curl -s -X POST http://localhost:8080/predict \
  -H 'content-type: application/json' \
  -d '{"features":[5.1,3.5,1.4,0.2]}'
curl -s http://localhost:8080/metrics | head

You now have a reproducible, portable inference service with built-in metrics.

2) Track experiments and artifacts with MLflow

Centralize metrics, params, and models so you can compare runs and reproduce results.

Install MLflow locally (user site to avoid system Python issues):

python3 -m pip install --user mlflow==2.14.1 scikit-learn pandas

Start an MLflow tracking server using SQLite for metadata and the local filesystem for artifacts:

mkdir -p mlruns
mlflow server \
  --backend-store-uri sqlite:///mlruns/mlflow.db \
  --default-artifact-root file://$PWD/mlruns \
  --host 0.0.0.0 --port 5000

In a new terminal, run a training script that logs to MLflow:

Create train_mlflow.py:

cat > train_mlflow.py << 'EOF'
import mlflow, mlflow.sklearn
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
import pandas as pd

mlflow.set_experiment("iris-demo")
mlflow.sklearn.autolog(log_model=True)

X, y = load_iris(return_X_y=True, as_frame=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

with mlflow.start_run():
    model = LogisticRegression(max_iter=300)
    model.fit(X_train, y_train)
    preds = model.predict(X_test)
    acc = accuracy_score(y_test, preds)
    mlflow.log_metric("accuracy", acc)
    print("Logged run with accuracy:", acc)
EOF

Point the client to the tracking server and run:

export MLFLOW_TRACKING_URI=http://127.0.0.1:5000
python3 train_mlflow.py

Open your browser to:

http://127.0.0.1:5000

You’ll see runs, metrics, parameters, and models logged for future reuse and comparison.

3) Ship images with CI/CD

Turn your container build into a repeatable pipeline that runs on every commit. Here’s a minimal GitHub Actions workflow that builds and pushes your image to a registry.

Create .github/workflows/build.yaml:

cat > .github/workflows/build.yaml << 'EOF'
name: build-and-push
on:
  push:
    branches: [ "main" ]
jobs:
  build:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
    steps:
      - uses: actions/checkout@v4
      - name: Set up QEMU
        uses: docker/setup-qemu-action@v3
      - name: Set up Buildx
        uses: docker/setup-buildx-action@v3
      - name: Login to GHCR
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - name: Build and push
        uses: docker/build-push-action@v6
        with:
          context: .
          push: true
          tags: ghcr.io/${{ github.repository }}/ai-platform-api:0.1
EOF

If you’re using a different registry (e.g., Docker Hub, Quay), adjust the login and tags accordingly.

Manual push from Linux with Podman:

podman login ghcr.io
podman tag ai-platform/api:0.1 ghcr.io/youruser/ai-platform-api:0.1
podman push ghcr.io/youruser/ai-platform-api:0.1

Now every change to main produces a fresh, immutable image you can deploy to Kubernetes, Nomad, or a VM.

4) Observe what your models do

You’ve already exposed Prometheus metrics at /metrics. You can scrape them with Prometheus and visualize in Grafana.

Example Prometheus scrape job (add to prometheus.yml):

scrape_configs:
  - job_name: 'ai-api'
    static_configs:
      - targets: ['host.docker.internal:8080']  # on macOS/Windows with Docker
      - targets: ['localhost:8080']            # on Linux host if port-forwarded

To quickly verify metrics without Prometheus:

curl -s http://localhost:8080/metrics | grep inference_requests_total
curl -s http://localhost:8080/metrics | grep inference_latency_seconds_bucket | head

Structured logging tip: write JSON logs for easier parsing downstream. For FastAPI/Uvicorn, you can enable JSON logging by passing --log-config or configuring Python’s logging module; centralize logs via your platform’s log collector.

Putting it together: a minimal, repeatable AI platform loop

  • Develop: Code + notebooks → extract training/eval into scripts.

  • Repro: Build container images to standardize runtimes.

  • Track: Use MLflow for experiments, metrics, and models.

  • Ship: CI/CD builds and pushes images on every commit.

  • Run and Observe: Deploy containers, scrape metrics, inspect logs, iterate.

This approach scales from your laptop to a cluster without changing commands or code structure.

Conclusion and next steps

You just built the core of an AI platform from your Linux terminal: a reproducible model service, experiment tracking, CI/CD, and observability. The value is fewer surprises, faster iteration, and lower risk when moving from prototype to production.

Your next steps:

  • Wrap your team’s existing notebooks into train/eval scripts and containerize them.

  • Stand up a shared MLflow server and agree on experiment naming and model lifecycle.

  • Add tests and automated checks to your CI (data schema checks, linting, unit tests).

  • Deploy to your target runtime (Kubernetes, VMs) and hook up Prometheus/Grafana.

If you want a follow-up, ask for a Kubernetes deployment walkthrough (kustomize/helm), secrets management patterns, or GPU scheduling guidance—all runnable from Bash on your favorite Linux distro.

Related posts