Posted on
• Artificial Intelligence

Artificial Intelligence Governance on Linux

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence Governance on Linux: Practical, Bash-First Controls You Can Use Today

If your AI runs on Linux, your governance does too. Policies written in slide decks don’t stop an unvetted model from exfiltrating data, and “we’ll fix it later” won’t help when auditors ask, “What ran? With what data? Under which controls?” The good news: many of the strongest AI governance controls are just disciplined Linux practices you can automate in Bash.

This post shows how to operationalize AI governance on Linux—covering isolation, integrity and provenance, policy-as-code, and auditing. The steps align with frameworks like NIST AI RMF, the EU AI Act’s technical controls spirit, and ISO/IEC 42001’s management-system mindset—without requiring heavy new platforms.

Why AI governance belongs in your Linux toolbelt

  • Trust and compliance: You need to prove what you ran, that you were allowed to run it, and that it didn’t touch what it shouldn’t.

  • Risk reduction: Least-privilege, isolation, and change control lower the blast radius of model mistakes, prompt injection, or supply-chain issues.

  • Reproducibility: Pinned environments and signed artifacts make runs explainable and rerunnable.

  • Cost: Most controls leverage stock Linux and common FOSS tools you already use.

Prerequisites: Install the basics

We’ll use Podman (rootless containers), Git (change tracking), GnuPG (signing), auditd (auditing), ACLs (fine-grained permissions), and jq (JSON utilities).

Ubuntu/Debian (apt):

sudo apt update
sudo apt install -y podman git gnupg jq auditd acl

Fedora/RHEL/CentOS Stream (dnf):

sudo dnf install -y podman git gnupg2 jq audit acl

openSUSE/SLE (zypper):

sudo zypper refresh
sudo zypper install -y podman git gpg2 jq audit acl

Verify basics:

podman --version
git --version
gpg --version
auditctl -v
jq --version

1) Isolate and constrain AI workloads

Contain the runtime, the users, and the resources.

  • Create a dedicated service account and directories:
sudo useradd -m -r -s /usr/sbin/nologin aiops || true
sudo install -d -o aiops -g aiops -m 0750 /opt/ai/{models,data,logs}
  • Lock down access with ACLs (principle of least privilege):
# Allow only aiops (and a review group if needed) to read models
sudo setfacl -m u:aiops:rX /opt/ai/models
sudo setfacl -dm u:aiops:rX /opt/ai/models

# Remove world access
sudo chmod -R o-rwx /opt/ai
  • Constrain CPU/RAM using systemd:
# Create a governance slice with accounting
sudo systemctl set-property ai.slice CPUAccounting=yes MemoryAccounting=yes

# Run a bounded session (example limits: 8 GiB RAM, 200% of a CPU)
sudo systemd-run --unit ai-run --slice ai.slice \
  -p MemoryMax=8G -p CPUQuota=200% --collect --pty bash
  • Run inference inside a rootless container (Podman):
# As aiops, pull a runtime image and run with no network
sudo -u aiops -H sh -lc '
  podman pull docker.io/library/python:3.11
  podman run --rm --net=none \
    -v /opt/ai/models:/models:ro,Z -v /opt/ai/logs:/logs:Z \
    docker.io/library/python:3.11 -c "print(\"isolated run\")"
'

Tip: Prefer images by content digest for reproducibility. Example:

# Find digest, then pin it
sudo -u aiops podman images --digests | awk "/python.*3.11/ {print \$3}"
sudo -u aiops podman run --rm --net=none docker.io/library/python@sha256:<digest> -c '...'

2) Prove integrity and provenance of data and models

Track where artifacts came from, verify them before use, and sign what you approve.

  • Generate checksums and sign them with GPG:
# Create a signing key if you don't have one
gpg --quick-generate-key "AI Model Signer <ai-signer@example.org>" rsa4096 sign 1y

# Compute and sign model checksums
cd /opt/ai/models
sudo -u aiops find . -type f -maxdepth 1 -exec sha256sum {} \; | sudo -u aiops tee CHECKSUMS
sudo -u aiops gpg --output CHECKSUMS.sig --detach-sign CHECKSUMS
  • Verify before any run:
cd /opt/ai/models
gpg --verify CHECKSUMS.sig CHECKSUMS && sha256sum -c CHECKSUMS
  • Wrap verification in a Bash helper that fails closed:
verify_artifacts() {
  local dir="$1"
  ( cd "$dir" &&
    gpg --verify CHECKSUMS.sig CHECKSUMS &&
    sha256sum -c CHECKSUMS
  )
}
# Usage:
verify_artifacts /opt/ai/models || { echo "Model integrity check failed"; exit 1; }
  • Sign your code/config commits too:
git config --global user.signingkey "$(gpg --list-secret-keys --keyid-format=long | awk '/sec/{print $2}' | head -1 | cut -d/ -f2)"
git config --global commit.gpgsign true

Result: You can prove “exactly these files, unchanged, were approved and used.”

3) Enforce guardrails with policy-as-code (OPA)

Use Open Policy Agent (OPA) to codify what’s allowed: approved images, no network egress, and verified artifacts.

  • Install OPA (binary download; adjust arch if needed):
curl -L -o /tmp/opa https://openpolicyagent.org/downloads/latest/opa_linux_amd64
echo "d2c9f2a1  /tmp/opa" >/dev/null # optionally verify checksum from official site
sudo install -m 0755 /tmp/opa /usr/local/bin/opa
  • Write a simple Rego policy (policy.rego):
package ai

default allow = false

approved_images := {
  "docker.io/library/python@sha256:YOUR_PINNED_DIGEST"
}

allow {
  input.checksum_verified == true
  input.image == approved_images[_]
  not input.net_egress
}
  • Gate your run with OPA from Bash:
#!/usr/bin/env bash
set -euo pipefail

IMAGE="docker.io/library/python@sha256:YOUR_PINNED_DIGEST"
verify_artifacts /opt/ai/models
VERIFIED=$?

INPUT=$(jq -n --arg img "$IMAGE" --argjson ok $([ $VERIFIED -eq 0 ] && echo true || echo false) \
            '{image:$img, checksum_verified:$ok, net_egress:false}')

if [ "$(echo "$INPUT" | opa eval -f raw -d policy.rego 'data.ai.allow' -I)" != "true" ]; then
  echo "Policy denied run"; exit 1
fi

sudo -u aiops -H podman run --rm --net=none \
  -v /opt/ai/models:/models:ro,Z -v /opt/ai/logs:/logs:Z \
  "$IMAGE" -c 'print("policy-allowed, networkless run")'

This pattern makes approvals reviewable: change control is a Git diff on policy.rego and the pinned image digest.

4) Audit what matters (files, processes, and runs)

Auditing creates the forensic trail that governance requires.

  • Enable and start auditd:
sudo systemctl enable --now auditd
sudo auditctl -s
  • Add persistent audit rules for model/data directories and container invocations:
sudo tee /etc/audit/rules.d/ai.rules >/dev/null <<'EOF'
-w /opt/ai/models -p rwxa -k ai-models
-w /opt/ai/data -p rwxa -k ai-data
-w /usr/bin/podman -p x -k ai-runtime
-w /usr/local/bin/opa -p x -k ai-policy
EOF

# Load rules
sudo augenrules --load || sudo systemctl restart auditd
  • Search audit logs by key:
# Who touched models?
sudo ausearch -k ai-models -i | less

# Show runtime executions
sudo ausearch -k ai-runtime -i | aureport -x --summary
  • Keep runtime logs disciplined:
# Example: journal logs for a governed unit
sudo systemd-run --unit llm-infer --slice ai.slice \
  -p MemoryMax=8G -p CPUQuota=200% \
  /usr/bin/true
journalctl -u llm-infer --since "1 hour ago"

Optional hardening:

  • Disable outbound network by default for AI runs (--net=none in Podman).

  • Keep SELinux/AppArmor enforcing (default on Fedora/Ubuntu). Check status quickly:

# SELinux (Fedora/RHEL)
getenforce

# AppArmor (Ubuntu/Debian)
aa-status

5) Make runs reproducible and explainable

Capture enough context to rerun and explain results.

  • Pin container images by digest and freeze inputs via checksums (above).

  • Record environment metadata with each run:

run_manifest() {
  jq -n \
    --arg when "$(date --iso-8601=seconds)" \
    --arg kernel "$(uname -r)" \
    --arg os "$(grep PRETTY_NAME /etc/os-release | cut -d= -f2- | tr -d \")" \
    --arg image "$1" \
    --arg policy_version "$(git -C /path/to/policy rev-parse --short HEAD 2>/dev/null || echo unknown)" \
    '{timestamp:$when, kernel:$kernel, os:$os, image:$image, policy_git:$policy_version}'
}
run_manifest "docker.io/library/python@sha256:YOUR_PINNED_DIGEST" | sudo -u aiops tee -a /opt/ai/logs/run-manifest.json
  • Store manifests, policy files, and CHECKSUMS in Git. That gives you a tamper-evident history.

Real-world example: A research lab serving an on-prem LLM

  • They run inference in a rootless Podman container with --net=none.

  • Models in /opt/ai/models are checksumed and GPG-signed by the lead.

  • A short OPA policy allows only the approved, pinned image digest and requires checksum_verified=true.

  • auditd watches models, data, and runtime invocations; manifests get appended to /opt/ai/logs.

  • When asked “what changed since last month’s release?”, they show Git diffs for policies and CHECKSUMS and re-run the job by digest.

Conclusion and next steps

AI governance on Linux isn’t a new platform—it’s a disciplined way to use the tools you already trust: containers for isolation, GPG and checksums for integrity, OPA for codified approvals, and auditd for evidence.

Your next steps: 1) Install the prerequisites and create the aiops service account. 2) Put your models under /opt/ai/models, create CHECKSUMS, and sign them. 3) Pin a container image by digest and run with --net=none. 4) Add the OPA policy and gate every run through it. 5) Turn on auditd rules and start capturing run manifests.

Start small: implement Steps 1–2 this week, 3–4 next, and add auditing once the basics are stable. Governance is a journey; these Bash-first controls make it repeatable, provable, and ready for scrutiny.

Related posts