Posted on
• Artificial Intelligence

Artificial Intelligence Podman Workflows

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence Podman Workflows: Reproducible, Rootless, GPU‑Ready on Linux

What if you could go from zero to a GPU‑accelerated Jupyter lab or batch inference job in minutes—without running a Docker daemon, without sudo, and with production‑grade reproducibility baked in?

That’s the promise of Podman for AI/ML: rootless containers, first‑class systemd integration, easy image builds with Buildah, image movement with Skopeo, and strong security defaults. In this guide, you’ll set up a clean Podman toolchain, enable GPU acceleration, and walk through practical workflows to build, run, and ship AI workloads on Linux.

Why Podman for AI?

  • Security by default: Run rootless for everyday experiments; escalate only when you must.

  • No daemon: Fewer moving parts, easier debugging, less “works on my machine”.

  • Modular tools: Buildah, Skopeo, and Podman compose a powerful, script‑friendly toolchain.

  • Production‑ready: Pods, healthchecks, systemd unit generation, and registries all work cleanly.

  • GPU support: NVIDIA and AMD GPUs can be passed into containers for accelerated training/inference.

1) Install the Toolchain

Install Podman plus the core container tools. Pick your package manager.

  • APT (Debian/Ubuntu):
sudo apt update
sudo apt install -y podman buildah skopeo uidmap slirp4netns fuse-overlayfs podman-compose
  • DNF (Fedora/RHEL/CentOS Stream):
sudo dnf -y install podman buildah skopeo slirp4netns fuse-overlayfs podman-compose
  • Zypper (openSUSE Leap/Tumbleweed, SLE):
sudo zypper refresh
sudo zypper install -y podman buildah skopeo slirp4netns fuse-overlayfs podman-compose

Optional: Ensure your user has subuid/subgid ranges for rootless containers (usually done by default). If not:

grep -q "$(whoami)" /etc/subuid || echo "$(whoami):100000:65536" | sudo tee -a /etc/subuid
grep -q "$(whoami)" /etc/subgid || echo "$(whoami):100000:65536" | sudo tee -a /etc/subgid

Validate your setup:

podman --version
podman info --format '{{.Host.OCIRuntime.Name}}'     # crun or runc
podman run --rm quay.io/podman/hello

2) Enable GPU Acceleration (Optional but common for AI)

NVIDIA GPUs with the NVIDIA Container Toolkit

1) Add the NVIDIA Container Toolkit repository and install the package.

  • APT (Debian/Ubuntu):
distribution=$(. /etc/os-release; echo ${ID}${VERSION_ID})
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/${distribution}/libnvidia-container.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update
sudo apt install -y nvidia-container-toolkit
  • DNF (Fedora/RHEL/CentOS Stream):
distribution=$(. /etc/os-release; echo ${ID}${VERSION_ID})
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/${distribution}/libnvidia-container.repo | \
  sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
sudo dnf -y install nvidia-container-toolkit
  • Zypper (openSUSE/SLE):
distribution=$(. /etc/os-release; echo ${ID}${VERSION_ID})
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/${distribution}/libnvidia-container.repo | \
  sudo tee /etc/zypp/repos.d/nvidia-container-toolkit.repo
sudo zypper refresh
sudo zypper install -y nvidia-container-toolkit

2) Configure the toolkit for Podman’s OCI runtime.

runtime=$(podman info --format '{{.Host.OCIRuntime.Name}}')   # crun or runc
sudo nvidia-ctk runtime configure --runtime="${runtime}"

Note: On some rootless setups you may also need:

sudo nvidia-ctk config --set nvidia-container-cli.no-cgroups=true --in-place

3) Test inside a container (SELinux hosts often require disabling labels for GPU devices):

podman run --rm \
  --env NVIDIA_VISIBLE_DEVICES=all \
  --env NVIDIA_DRIVER_CAPABILITIES=compute,utility \
  --security-opt=label=disable \
  docker.io/nvidia/cuda:12.4.1-runtime-ubuntu22.04 nvidia-smi

If you see your GPU(s) listed, you’re good.

AMD GPUs (ROCm)

For ROCm‑enabled containers, pass the GPU device nodes and keep supplementary groups:

podman run --rm \
  --device=/dev/kfd --device=/dev/dri \
  --group-add keep-groups \
  --security-opt=label=disable \
  <rocm-enabled-image> rocminfo    # or clinfo

Notes:

  • Install and configure AMDGPU/ROCm on the host per AMD’s docs.

  • Your user typically needs to be in the video/render groups on the host for access:

groups | grep -E 'video|render' || echo "Add your user to video/render groups and re-login"

3) Core Workflows You Can Use Today

Below are practical, copy‑pasteable building blocks for daily AI work.

A) Build a Reproducible AI Notebook Image (CPU‑only)

1) Create a Containerfile:

# Containerfile (CPU)
FROM docker.io/python:3.11-slim
WORKDIR /app

# Minimal OS deps and fast Python builds
RUN apt-get update && apt-get install -y --no-install-recommends git && \
    rm -rf /var/lib/apt/lists/*

RUN python -m pip install --upgrade pip wheel
# CPU PyTorch + Transformers + Jupyter
RUN pip install --no-cache-dir torch --index-url https://download.pytorch.org/whl/cpu && \
    pip install --no-cache-dir transformers jupyter

# Optional demo
COPY demo.py .

EXPOSE 8888
CMD ["bash","-lc","jupyter lab --ip=0.0.0.0 --no-browser --allow-root"]

2) Add a small demo script:

# demo.py
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch, sys

model_id = "distilbert-base-uncased-finetuned-sst-2-english"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id).to(
    "cuda" if torch.cuda.is_available() else "cpu"
)

text = " ".join(sys.argv[1:]) or "Podman makes AI reproducible!"
inputs = tok(text, return_tensors="pt").to(model.device)
with torch.inference_mode():
    out = model(**inputs)
pred = out.logits.softmax(-1).argmax(-1).item()
print({0: "negative", 1: "positive"}[pred])

3) Build with Podman or Buildah:

podman build -t localhost/ai-notebook:cpu -f Containerfile .
# or:
buildah bud -t localhost/ai-notebook:cpu -f Containerfile .

4) Run a quick test:

podman run --rm localhost/ai-notebook:cpu python demo.py "This workflow is awesome!"

5) Launch Jupyter Lab with persistent caches:

mkdir -p ~/ai-cache/hf ~/ai-cache/torch ~/ai-data
podman run --rm -p 8888:8888 \
  -v $HOME/ai-cache/hf:/root/.cache/huggingface \
  -v $HOME/ai-cache/torch:/root/.cache/torch \
  -v $HOME/ai-data:/workspace/data \
  localhost/ai-notebook:cpu

Open the printed URL with the token in your browser.

B) Build a CUDA‑Enabled Notebook Image (NVIDIA)

1) Create a CUDA‑based Containerfile:

# Containerfile (NVIDIA GPU)
FROM docker.io/nvidia/cuda:12.4.1-runtime-ubuntu22.04
RUN apt-get update && apt-get install -y python3-pip git && \
    rm -rf /var/lib/apt/lists/*
RUN python3 -m pip install --upgrade pip wheel
# PyTorch matching CUDA 12.4 wheels (cu124)
RUN pip3 install --no-cache-dir --index-url https://download.pytorch.org/whl/cu124 \
    torch torchvision
RUN pip3 install --no-cache-dir transformers jupyter
WORKDIR /app
COPY demo.py .
EXPOSE 8888
CMD ["bash","-lc","jupyter lab --ip=0.0.0.0 --no-browser --allow-root"]

2) Build it:

podman build -t localhost/ai-notebook:cuda -f Containerfile .

3) Run with GPU access (env variables trigger the NVIDIA OCI hook):

podman run --rm -p 8888:8888 \
  --env NVIDIA_VISIBLE_DEVICES=all \
  --env NVIDIA_DRIVER_CAPABILITIES=compute,utility \
  --security-opt=label=disable \
  -v $HOME/ai-cache/hf:/root/.cache/huggingface \
  -v $HOME/ai-cache/torch:/root/.cache/torch \
  -v $HOME/ai-data:/workspace/data \
  localhost/ai-notebook:cuda

4) Sanity check GPU:

podman run --rm \
  --env NVIDIA_VISIBLE_DEVICES=all \
  --env NVIDIA_DRIVER_CAPABILITIES=compute,utility \
  --security-opt=label=disable \
  localhost/ai-notebook:cuda python -c "import torch; print(torch.cuda.is_available())"

C) Batch Inference Over a File

Mount your workspace and run jobs non‑interactively:

podman run --rm -v $PWD:/work -w /work localhost/ai-notebook:cpu \
  python /app/demo.py "Batch inference from a mounted folder."

For NVIDIA:

podman run --rm -v $PWD:/work -w /work \
  --env NVIDIA_VISIBLE_DEVICES=all --env NVIDIA_DRIVER_CAPABILITIES=compute,utility \
  --security-opt=label=disable \
  localhost/ai-notebook:cuda \
  python /app/demo.py "GPU‑accelerated batch inference."

D) Compose and Systemd: From Notebook to Service

Podman‑Compose helps you orchestrate multi‑container workflows on your laptop or workstation.

compose.yaml:

services:
  notebook:
    image: localhost/ai-notebook:cpu
    ports:
      - "8888:8888"
    volumes:
      - ${HOME}/ai-cache/hf:/root/.cache/huggingface
      - ${HOME}/ai-cache/torch:/root/.cache/torch
      - ${HOME}/ai-data:/workspace/data
    # For NVIDIA, also set:
    # environment:
    #   - NVIDIA_VISIBLE_DEVICES=all
    #   - NVIDIA_DRIVER_CAPABILITIES=compute,utility
    # security_opt:
    #   - label=disable

Run it:

podman-compose up

Make a long‑running service with systemd:

podman run -d --name ai-notebook -p 8888:8888 localhost/ai-notebook:cpu
podman generate systemd --name ai-notebook --new --files
mkdir -p ~/.config/systemd/user
mv container-ai-notebook.service ~/.config/systemd/user/
systemctl --user enable --now container-ai-notebook.service
# Allow lingering so it stays running after logout:
loginctl enable-linger "$USER"

E) Share and Move Images (Skopeo/Podman)

Tag and push with Podman:

podman tag localhost/ai-notebook:cpu quay.io/<yourname>/ai-notebook:cpu
podman login quay.io
podman push quay.io/<yourname>/ai-notebook:cpu

Mirror between registries with Skopeo (no local pull needed):

skopeo copy \
  docker://docker.io/library/python:3.11 \
  docker://quay.io/<yourname>/python:3.11-mirrored

Real‑World Tips

  • Fast, repeatable installs: Prefer building images over installing in “pet” containers. That’s what Containerfiles are for.

  • Cache model artifacts: Mount $HOME/ai-cache into containers to avoid repeated downloads and accelerate CI.

  • SELinux gotchas: On Fedora/RHEL, add --security-opt=label=disable when passing GPUs or host devices.

  • Check your runtime: podman info --format '{{.Host.OCIRuntime.Name}}' and configure NVIDIA to match (crun vs runc).

  • Rootless GPUs: If you hit cgroup errors with NVIDIA, set no-cgroups=true via nvidia-ctk as shown above.

Conclusion and Next Steps

You now have a secure, reproducible, GPU‑ready Podman workflow for AI on Linux:

  • Rootless Podman + Buildah + Skopeo installed

  • Optional NVIDIA/ROCm acceleration configured

  • Reusable Containerfiles for notebooks and batch jobs

  • Compose/systemd for orchestration and services

  • Registry flows for sharing and CI/CD

Your next step:

  • Fork the CPU or CUDA Containerfile above and tailor it to your stack (TensorFlow, JAX, OpenVINO, ONNX Runtime).

  • Wrap your experiments in Podman‑Compose, then promote stable ones to systemd services.

  • Push images to your team’s registry and wire into CI for fully reproducible research.

If you found this useful, try turning one of your current AI projects into a Podmanized workflow today—then time how long it takes to move it from your laptop to a teammate’s machine without breaking anything. That’s real value.

Related posts