Artificial Intelligence on Linux: A Practical Compliance Playbook

AI on Linux is exploding—from GPU-powered training rigs to edge inference nodes—but so are the questions from security, legal, and audit teams. Can you prove what code and models ran? Where data came from? Who accessed what? If your answer is “not easily,” this post is for you.

This guide shows how to make AI workloads on Linux verifiable, auditable, and compliant without killing developer velocity. You’ll get concrete Bash-friendly steps, real-world examples, and install commands for apt, dnf, and zypper.

Why “AI Linux Compliance” is worth your time

• Regulators and customers increasingly demand evidence: provenance of models/datasets, reproducibility, and access controls (think: healthcare, finance, public sector).

• AI stacks multiply your attack surface: fast-moving Python deps, GPU drivers, container images, model artifacts pulled from the internet, and data that’s often sensitive.

• Compliance improves reliability: deterministic builds, signed artifacts, and auditable logs make outages and “it worked on my machine” far less painful.

Below is a 5-step playbook you can roll out incrementally.

---

1) Establish a hardened, measurable baseline

Use established security baselines (CIS, STIG) and make the results reproducible with OpenSCAP.

Install OpenSCAP tooling and content:

# Debian/Ubuntu
sudo apt update
sudo apt -y install openscap-scanner scap-security-guide || \
  sudo apt -y install openscap-utils scap-security-guide

# Fedora/RHEL/CentOS Stream
sudo dnf -y install openscap-scanner scap-security-guide

# openSUSE/SLES
sudo zypper refresh
sudo zypper -y install openscap-scanner scap-security-guide

If a package isn’t found, search:

# apt
apt-cache search openscap
# dnf
dnf search scap
# zypper
zypper search scap

Discover available content for your distro, then run an evaluation and produce a report:

# Find your content file (examples may be ssg-ubuntu2204-ds.xml, ssg-fedora-ds.xml, ssg-rhel8-ds.xml, etc.)
ls /usr/share/xml/scap/ssg/content/

# List profiles
oscap info /usr/share/xml/scap/ssg/content/ssg-<yourdistro>-ds.xml | sed -n '/Profiles/,/Referenced/p'

# Example: evaluate a CIS or STIG-like profile (adjust profile ID for your distro)
sudo oscap xccdf eval \
  --profile xccdf_org.ssgproject.content_profile_cis \
  --report /var/tmp/openscap-report.html \
  /usr/share/xml/scap/ssg/content/ssg-<yourdistro>-ds.xml

Action:

• Save HTML and ARF/XML outputs in your CI or artifact store for audit trails.

• Automate monthly runs; fail builds if you regress on critical findings.

Real-world example:

• A fintech team bakes an OpenSCAP run into their image pipeline; if a base image slips below their baseline, publishing is blocked.

---

2) Protect sensitive data and models (at rest and in transit)

Encrypt model caches and datasets, and verify integrity with strong hashes.

Install cryptsetup:

# Debian/Ubuntu
sudo apt update && sudo apt -y install cryptsetup

# Fedora/RHEL/CentOS Stream
sudo dnf -y install cryptsetup

# openSUSE/SLES
sudo zypper refresh && sudo zypper -y install cryptsetup

Create a file-backed encrypted vault for models (safer than reformatting a device):

# Create a 50G sparse file vault
sudo dd if=/dev/zero of=/secure/model.vault bs=1 count=0 seek=50G
sudo losetup -fP /secure/model.vault
LOOP=$(losetup -j /secure/model.vault | awk -F: '{print $1}')

# Initialize LUKS (warning: choose a strong passphrase)
sudo cryptsetup luksFormat "$LOOP"
sudo cryptsetup open "$LOOP" model_vault

# Make a filesystem and mount it
sudo mkfs.ext4 /dev/mapper/model_vault
sudo mkdir -p /mnt/model_vault
sudo mount /dev/mapper/model_vault /mnt/model_vault

Hash and verify datasets/models:

cd /mnt/model_vault
sha256sum *.pt *.onnx *.safetensors > SHA256SUMS
sha256sum -c SHA256SUMS   # verify later

Action:

• Keep encryption keys in a KMS or hardware-backed store where possible.

• Rotate and log access to decryption operations.

• Use TLS for model downloads (curl/wget over HTTPS) and pin expected checksums.

Real-world example:

• A hospital research team keeps PHI-derived embeddings in an encrypted vault that is opened for batch inference windows and auto-closed afterward.

---

3) Reproducible, signed containers for AI workloads

Containerize your training/inference with pinned digests and verified signatures. Prefer rootless Podman for least privilege.

Install containers, skopeo, and cosign:

# Debian/Ubuntu
sudo apt update
sudo apt -y install podman skopeo cosign

# Fedora/RHEL/CentOS Stream
sudo dnf -y install podman skopeo cosign

# openSUSE/SLES
sudo zypper refresh
sudo zypper -y install podman skopeo cosign

Pin image by digest and verify trust:

# Inspect and pin a specific digest
skopeo inspect docker://docker.io/library/python:3.11-slim | jq -r .Digest
# Example digest:
DIGEST=sha256:abcdef...   # replace with actual

# Verify signature (if publisher signs with sigstore)
cosign verify docker.io/library/python@${DIGEST}

# Run rootless with Podman, pinned digest
podman run --rm --userns=keep-id \
  docker.io/library/python@${DIGEST} \
  python -c "print('hello, compliant AI')"

Enforce trust policies (allow only signed images from known registries). Create or edit:

sudo mkdir -p /etc/containers
sudo tee /etc/containers/policy.json >/dev/null <<'EOF'
{
  "default": [{"type": "reject"}],
  "transports": {
    "docker": {
      "docker.io/yourorg/*": [{"type": "sigstoreSigned", "keyPath": "/etc/containers/keys/yourorg.pub"}]
    }
  }
}
EOF

Action:

• Use multi-stage builds with locked versions for CUDA/ROCm runtimes.

• Store SBOMs for images (e.g., syft) and attach provenance with cosign attest.

Real-world example:

• A media startup runs inference via Podman with image digests baked into IaC; prod rollouts are reversible and provable.

---

4) Enforce least privilege with SELinux or AppArmor

Mandatory Access Control (MAC) prevents an escaped process from reading your datasets or SSH keys.

Check and install helpers:

# Debian/Ubuntu (AppArmor)
sudo apt update && sudo apt -y install apparmor-utils

# Fedora/RHEL/CentOS Stream (SELinux)
sudo dnf -y install policycoreutils setools-console

# openSUSE/SLES (AppArmor by default)
sudo zypper refresh && sudo zypper -y install apparmor-utils

Quick status checks:

# AppArmor
aa-status

# SELinux
getenforce  # Expect: Enforcing

On SELinux systems, container volume labels matter. Use :z or :Z to relabel volumes for confined containers:

# Share a model directory with proper SELinux label for containers
podman run --rm -v /mnt/model_vault:/models:ro,z \
  --security-opt label=type:container_t \
  your.registry/inference@sha256:<digest> \
  ./serve --model /models/model.onnx

Action:

• Keep SELinux in Enforcing (don’t disable); troubleshoot with audit logs instead.

• For Ubuntu/AppArmor, use aa-status to ensure profiles are loaded; consider custom profiles for your inference servers.

Real-world example:

• A government analytics team stopped a container breakout from touching a mounted SSH key directory because SELinux labeling blocked access.

---

5) Audit, logs, and attestations you can show an auditor

Turn runtime behavior into evidence: who accessed models, when code ran, and which artifacts were used.

Install and enable auditd:

# Debian/Ubuntu
sudo apt update && sudo apt -y install auditd
sudo systemctl enable --now auditd

# Fedora/RHEL/CentOS Stream
sudo dnf -y install audit
sudo systemctl enable --now auditd

# openSUSE/SLES
sudo zypper refresh && sudo zypper -y install audit
sudo systemctl enable --now auditd

Add targeted audit rules (model reads, container/runtime execs):

# Watch model directory for reads and attribute changes
sudo auditctl -w /mnt/model_vault -p ra -k model_access

# Track Python and Podman executions
sudo auditctl -w /usr/bin/python3 -p x -k ai_exec
sudo auditctl -w /usr/bin/podman  -p x -k container_exec

# Persist rules across reboots:
sudo sh -c 'auditctl -l > /etc/audit/rules.d/ai.rules'
sudo augenrules --load  # or systemctl restart auditd

Query evidence:

# Who read models?
sudo ausearch -k model_access --format text

# When did inference job run?
sudo ausearch -k ai_exec -ts today

Attach provenance to containers with cosign attest:

# Create a simple attestation (adjust fields)
cat > predicate.json <<'JSON'
{
  "model_sha256": "e3b0c44298fc1c149afbf4c8996fb924...",
  "dataset_tag": "imagenet-2021-09",
  "git_commit": "f1a2b3c",
  "training_env": "cuda12.2-ubuntu22.04",
  "runner": "podman-rootless"
}
JSON

# Attach to your image (keyless or with your key material)
cosign attest --predicate predicate.json --type slsaprovenance \
  your.registry/inference@sha256:<digest>

Action:

• Ship logs to a central SIEM; keep clock sync (chrony/ntp) for credible timelines.

• Treat audit rules as code in your repo; review like any other change.

Real-world example:

• During a vendor due-diligence review, a retail AI team exported audit logs plus cosign attestations showing exactly which model/dataset powered last quarter’s personalization.

---

Quick compliance starter checklist

• Baseline: Run OpenSCAP monthly; archive reports; remediate fails.

• Supply chain: Pin digests; verify signatures; maintain SBOMs; attach provenance.

• Data: Encrypt at rest; hash and verify models/datasets; restrict who can decrypt.

• Runtime: Use rootless containers; enforce SELinux/AppArmor; least privilege on GPUs and volumes.

• Evidence: Enable auditd; add rules for model paths and runtimes; centralize logs.

---

Conclusion and next steps (CTA)

Compliance shouldn’t slow your AI roadmap—it should make it safer and more repeatable. Start by containerizing with pinned, signed images and enabling auditd. Then add OpenSCAP in CI and encrypt your model/data vaults. Within a sprint, you’ll have a defensible story: hardened hosts, controlled artifacts, and audit-ready evidence.

Next steps:

• Pick one environment (staging) and implement Steps 1–3 this week.

• Add audit rules for your model paths and roll them to prod.

• Socialize a short “AI compliance runbook” in your team wiki with the commands above.

If you want a reference implementation (scripts, policies, and example CI), tell me your distro and AI stack (CUDA/ROCm, frameworks, container registry), and I’ll tailor a drop-in starter repo.