Posted on
• Artificial Intelligence

Artificial Intelligence Container Security

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence Container Security: 4 Practical Steps for Linux Bash Users

If you ship AI in containers, you’re shipping your models, data preprocessors, and secret keys along with it. One leaked API token or a tampered model file can undo months of work. The good news: with a few Bash-friendly habits, you can harden AI containers without slowing your team down.

This post explains why AI containers need special care, then walks you through 4 pragmatic steps with copy-pasteable commands for apt, dnf, and zypper users.

Why AI container security is different (and urgent)

  • Models are the crown jewels: A single .pt/.onnx/.pkl file can be your competitive edge. Theft, tampering, or rollback to a poisoned model is game over.

  • The supply chain is broader: AI stacks blend CUDA drivers, Python wheels, system libs, and framework extensions. That’s many sources, many CVEs.

  • Images are huge: Bigger images mean a bigger attack surface. Common AI bases ship thousands of packages by default.

  • Secrets hide in plain sight: Notebooks, config files, or checkpoints can accidentally carry tokens, data samples, or PII.

  • GPUs add risk: Extra device access and drivers require careful isolation to avoid privilege escalation.

Security isn’t just compliance—it protects IP, safeguards users, and preserves reliability.

Step 1: Build minimal, reproducible AI containers (and produce SBOMs)

Slimmer images = fewer vulns. Reproducibility = easier audits. An SBOM (Software Bill of Materials) gives you evidence of what’s inside.

Key practices:

  • Pin versions for base images, CUDA, Python, and pip packages.

  • Use multi-stage builds and avoid tools you don’t need at runtime.

  • Generate an SBOM and store it with your artifacts.

Example Dockerfile snippet (multi-stage, pinned wheels):

# Build stage: create wheelhouse
FROM python:3.11-slim AS build
ENV PIP_NO_CACHE_DIR=1 PIP_DISABLE_PIP_VERSION_CHECK=1
RUN apt-get update && apt-get install -y --no-install-recommends gcc g++ && rm -rf /var/lib/apt/lists/*
WORKDIR /wheels
COPY requirements.txt .
RUN pip wheel --no-deps -r requirements.txt -w /wheels

# Runtime stage: minimal
FROM gcr.io/distroless/python3-debian12
WORKDIR /app
COPY --from=build /wheels /wheels
COPY . /app
ENV PYTHONPATH=/app
# Use exact hashes in requirements.txt for stronger pinning
CMD ["-m", "inference.server"]

Generate an SBOM with Syft:

# Install (all distros): official script (puts binary in /usr/local/bin)
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sudo sh -s -- -b /usr/local/bin
# Create SBOM for a container image or a folder
syft myregistry/myai:1.0 -o json > sbom-image.json
syft /path/to/repo -o spdx-json > sbom-source.json

Note: Syft is commonly installed via the script above; distro packages may lag or be unavailable.

Step 2: Scan AI images, filesystems, and secrets with Trivy

Trivy scans container images, directories, SBOMs, and IaC for vulnerabilities and secrets, making it ideal for AI repositories, notebooks, and model folders.

Install Trivy:

  • apt (Ubuntu/Debian):
sudo apt update
sudo apt install -y curl gnupg lsb-release
curl -fsSL https://aquasecurity.github.io/trivy-repo/deb/public.key | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/trivy.gpg
echo "deb https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/trivy.list
sudo apt update
sudo apt install -y trivy
  • dnf (Fedora/RHEL/CentOS):
sudo dnf install -y dnf-plugins-core
sudo dnf config-manager --add-repo https://aquasecurity.github.io/trivy-repo/rpm/releases.repo
sudo dnf install -y trivy
  • zypper (openSUSE/SLE):
sudo zypper addrepo https://aquasecurity.github.io/trivy-repo/rpm/releases.repo
sudo zypper refresh
sudo zypper install -y trivy

Common scans:

# 1) Scan a built image (Podman or Docker)
trivy image myregistry/myai:1.0

# 2) Scan a working directory (models, notebooks, configs) for vulns and secrets
trivy fs --security-checks vuln,secret ./models ./notebooks

# 3) Scan from SBOM you generated earlier
trivy sbom --output trivy-sbom-report.txt sbom-image.json

# 4) Fail CI on critical vulns
trivy image --severity CRITICAL --exit-code 1 myregistry/myai:1.0

Real-world tip:

  • Point Trivy at model directories to catch secrets committed alongside checkpoints or configs.

  • Include --ignorefile .trivyignore to silence known, justified findings.

Step 3: Run rootless with Podman, lock down privileges, and handle GPUs safely

Default to rootless containers, use MAC (SELinux/AppArmor), and drop capabilities you don’t need. For GPU workloads, add only the devices and permissions required.

Install Podman, Buildah, and Skopeo:

  • apt:
sudo apt update
sudo apt install -y podman buildah skopeo
  • dnf:
sudo dnf install -y podman buildah skopeo
  • zypper:
sudo zypper install -y podman buildah skopeo

Enable SELinux or AppArmor:

  • dnf (SELinux for containers):
sudo dnf install -y container-selinux policycoreutils
  • apt (AppArmor):
sudo apt install -y apparmor apparmor-utils
sudo aa-status  # verify it's enabled
  • zypper (AppArmor on openSUSE):
sudo zypper install -y apparmor-parser apparmor-utils
sudo aa-status

Hardened run (rootless Podman example):

# Ensure you're a non-root user; Podman is rootless by default
podman run --rm \
  --user 10001:10001 \
  --read-only \
  --tmpfs /tmp:rw,noexec,nosuid,size=64m \
  --cap-drop ALL \
  --security-opt no-new-privileges \
  --pids-limit 512 \
  --memory 2g --cpus 2 \
  --device /dev/null --device /dev/zero \
  --name ai-infer \
  myregistry/myai:1.0

GPU-aware runs

  • Docker is simplest for GPUs with the NVIDIA Container Toolkit.

  • Install Docker:

    • apt:
    sudo apt update
sudo apt install -y ca-certificates curl gnupg lsb-release
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/$(. /etc/os-release && echo "$ID")/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/$(. /etc/os-release && echo "$ID") $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo systemctl enable --now docker
    • dnf:
    sudo dnf -y install dnf-plugins-core
sudo dnf config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo
sudo dnf -y install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo systemctl enable --now docker
    • zypper:
    sudo zypper install -y docker
sudo systemctl enable --now docker

  • Install NVIDIA Container Toolkit:

    • apt:
    distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -fsSL https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update
sudo apt install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
    • dnf:
    distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /etc/pki/rpm-gpg/NVIDIA-GPG-KEY-lnc.gpg
curl -fsSL https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.repo | sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
sudo dnf -y install nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
    • zypper:
    distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /etc/zypp/trustedkeys/NVIDIA-GPG-KEY-lnc.gpg
curl -fsSL https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.repo | sudo tee /etc/zypp/repos.d/nvidia-container-toolkit.repo
sudo zypper refresh
sudo zypper install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Secure GPU run (Docker):

docker run --rm --gpus all \
  --user 10001:10001 \
  --read-only \
  --cap-drop ALL \
  --security-opt no-new-privileges \
  --pids-limit 512 \
  --memory 8g --cpus 8 \
  -v /secure/models:/models:ro \
  myregistry/myai-gpu:1.0

Notes:

  • Keep the root filesystem read-only; write to explicit tmpfs or bind mounts.

  • Mount models read-only to prevent tampering.

  • If you must use Podman for GPUs, map only the specific /dev/nvidia* devices required and the minimum libraries; avoid blanket privileged flags.

Step 4: Sign your AI images and verify before deploy (cosign)

Signing gives you provenance: only run images your pipeline produced. Use cosign to sign container images (and attach SBOMs/attestations).

Install cosign:

  • apt (if available on your distro):
sudo apt update
sudo apt install -y cosign
  • dnf:
sudo dnf install -y cosign
  • zypper:
sudo zypper install -y cosign
  • Fallback (all distros, static binary):
COSIGN_VERSION=v2.2.3
curl -sSfL https://github.com/sigstore/cosign/releases/download/${COSIGN_VERSION}/cosign-linux-amd64 -o cosign
sudo install -m 0755 cosign /usr/local/bin/cosign

Sign and verify:

# Generate a key-pair (or use keyless/OIDC if configured)
cosign generate-key-pair
# Sign the image
cosign sign --key cosign.key myregistry/myai:1.0
# Verify before deploy
cosign verify --key cosign.pub myregistry/myai:1.0

Attach SBOM as an attestation:

cosign attest --key cosign.key --predicate sbom-image.json --type spdx myregistry/myai:1.0
cosign verify-attestation --type spdx --key cosign.pub myregistry/myai:1.0

Policy tip:

  • Gate production by verifying signatures in CI/CD or admission controls. Only run verified images.

Bonus: A tiny Bash CI gate you can adapt

Drop this in your pipeline to build, SBOM, scan, sign, and push. Fail fast on criticals.

#!/usr/bin/env bash
set -euo pipefail

IMAGE="registry.example.com/ai/my-model:$(git rev-parse --short HEAD)"

# Build (Podman or Docker)
podman build -t "$IMAGE" .

# SBOM with Syft
syft "$IMAGE" -o spdx-json > sbom.json

# Vulnerability + secret scan with Trivy
trivy image --severity HIGH,CRITICAL --exit-code 1 "$IMAGE"
trivy fs --security-checks secret --exit-code 1 .

# Sign and attach SBOM attestation
cosign sign --key cosign.key "$IMAGE"
cosign attest --key cosign.key --predicate sbom.json --type spdx "$IMAGE"

# Push only after all checks pass
podman push "$IMAGE"

Conclusion and CTA

AI apps carry unique risks—valuable models, sprawling dependencies, and GPU permissions. You don’t need a massive platform to be safe. Start with four habits:

  • Build minimal, pinned images and keep an SBOM.

  • Scan everything (images, folders, SBOMs, secrets) with every commit.

  • Run rootless with strong isolation; keep filesystems read-only and drop caps.

  • Sign and verify every image and artifact you ship.

Your next step: 1) Install Trivy and Podman using the commands above. 2) Add the Bash CI gate to your repo. 3) Turn on signature verification before any production deploy.

Small steps, big risk reduction—today.

Related posts