Open Source Artificial Intelligence Security: A Linux Bash Playbook

Your models are smart. Attackers are smarter. As soon as you connect an open‑source AI stack to real data, shell tools, or the internet, you inherit a new attack surface: prompt injection, poisoned weights and datasets, secret exfiltration, and model‑runtime escapes. The good news: most of what you need to harden open‑source AI on Linux already lives in your package manager and a handful of well‑understood security patterns.

This post explains why AI security matters even for hobby and internal projects, then walks through a practical, Bash‑first hardening checklist you can run today. Every tool mentioned includes apt, dnf, and zypper install commands.

Why secure open-source AI?

• Models amplify small mistakes. A single over‑permissive volume mount or outbound egress route can turn a prompt injection into a data breach.

• Supply chain isn’t just code anymore. Unverified model weights and datasets are untrusted inputs that can carry malicious artifacts or bias your downstream results.

• AI runtimes touch everything. They read files, call tools, browse URLs, and run interpreters—exactly the places attackers love.

Treat models, datasets, and prompts as untrusted inputs; treat the runtime like a high‑risk service. Here’s how.

1) Contain the model runtime (rootless, no network, least privilege)

Use rootless containers (Podman) to sandbox model execution. Deny network by default, drop capabilities, set resource limits, and mount only what you need as read‑only.

Install Podman:

• Debian/Ubuntu (apt):

  sudo apt update && sudo apt install -y podman
  

• Fedora/RHEL/CentOS (dnf):

  sudo dnf install -y podman
  

• openSUSE (zypper):

  sudo zypper install -y podman
  

Example: run your model inference with zero network, read‑only code, and strict limits.

# From your project directory with code in ./app
podman run --rm \
  --userns keep-id \
  --security-opt=no-new-privileges \
  --cap-drop ALL \
  --pids-limit 512 \
  --memory 4g \
  --read-only \
  --network none \
  -v "$PWD/app":/app:ro \
  -w /app \
  docker.io/library/python:3.11-slim \
  python run_inference.py --model-path /app/models/model.gguf --input /app/samples/prompt.txt

Tips:

• Avoid mounting $HOME. Bind only a purpose‑built working dir.

• Prefer --network none. If you must allow egress, see step 2 to constrain it.

• Use --device sparingly (e.g., GPUs). Map only what’s required.

Real‑world win: Prompt injection tries to fetch exfil domains? With --network none, it fails silently.

2) Egress control: block by default, allow by exception

Even if your container has no network, the host often does. Lock down egress for container interfaces with firewalld and assign Podman’s interface (often podman0) to a DROP zone.

Install firewalld:

• Debian/Ubuntu (apt):

  sudo apt update && sudo apt install -y firewalld
  sudo systemctl enable --now firewalld
  

• Fedora/RHEL/CentOS (dnf):

  sudo dnf install -y firewalld
  sudo systemctl enable --now firewalld
  

• openSUSE (zypper):

  sudo zypper install -y firewalld
  sudo systemctl enable --now firewalld
  

Create an “airgap” zone and bind Podman’s bridge to it:

# Discover Podman bridge (commonly podman0)
ip link show | awk -F: '/podman0/ {print $2}'

# Create DROP-by-default zone and attach podman0
sudo firewall-cmd --permanent --new-zone=airgap
sudo firewall-cmd --permanent --zone=airgap --set-target=DROP
sudo firewall-cmd --permanent --zone=airgap --add-interface=podman0
sudo firewall-cmd --reload

Need limited egress (e.g., to fetch a model from an internal cache)?

# Allow only HTTPS to 10.10.10.50 from airgapped zone
sudo firewall-cmd --permanent --zone=airgap \
  --add-rich-rule='rule family=ipv4 destination address=10.10.10.50/32 port port=443 protocol=tcp accept'
sudo firewall-cmd --reload

Caution: Changing default egress can disrupt services—test in a VM first.

3) Verify model and code provenance (checksums, signatures, Git LFS)

Treat weights and datasets like binaries: verify before use. Keep large artifacts out of your regular Git history with Git LFS.

Install Git, GnuPG, and Git LFS:

• Debian/Ubuntu (apt):

  sudo apt update && sudo apt install -y git gnupg git-lfs
  

• Fedora/RHEL/CentOS (dnf):

  sudo dnf install -y git gnupg2 git-lfs
  

• openSUSE (zypper):

  sudo zypper install -y git gpg2 git-lfs
  

Verify downloads:

# Example with model.gguf, its checksum, and a detached signature
sha256sum -c model.gguf.sha256sum

# Import the publisher’s key (replace with the actual fingerprint)
gpg --keyserver hkps://keys.openpgp.org --recv-keys 0xDEADBEEFCAFEBABE
gpg --verify model.gguf.asc model.gguf

Use Git LFS for weights/datasets:

git lfs install
git lfs track "*.gguf"
git add .gitattributes
git add models/model.gguf
git commit -m "Track and add model weights via LFS"

Bonus: Enable signed Git commits and tags so your CI can reject untrusted code:

git config --global user.signingkey <YOUR_KEYID>
git config --global commit.gpgsign true
git tag -s v1.0.0 -m "Signed release"

Real‑world win: Poisoned weights from a mirror fail checksum/signature verification and never reach prod.

4) Observe and alert: auditd and eBPF tooling for AI workloads

You can’t defend what you can’t see. Use auditd to watch sensitive paths and bcc/eBPF to observe outbound connections.

Install auditd and eBPF tools:

• Debian/Ubuntu (apt):

  sudo apt update && sudo apt install -y auditd bpftrace bpfcc-tools
  sudo systemctl enable --now auditd
  

• Fedora/RHEL/CentOS (dnf):

  sudo dnf install -y audit bpftrace bcc-tools
  sudo systemctl enable --now auditd
  

• openSUSE (zypper):

  sudo zypper install -y audit bpftrace bcc-tools
  sudo systemctl enable --now auditd
  

Watch for reads/writes to SSH keys and unexpected network connects:

# /etc/audit/rules.d/ai.rules
-w /home/*/.ssh -p rwa -k ai-ssh-guard
-w /etc -p wa -k ai-etc-guard
-a always,exit -F arch=b64 -S connect -k ai-net-conn
-a always,exit -F arch=b32 -S connect -k ai-net-conn

Apply and review:

sudo augenrules --load
sudo systemctl restart auditd

# Later, search alerts
sudo ausearch -k ai-ssh-guard
sudo ausearch -k ai-net-conn | aureport -f

Live‑trace outbound connections from a specific AI process with bcc:

# Find the PID of your model server
PID=$(pgrep -f run_inference.py)

# Use the bcc tcpconnect tool (path may vary by distro)
sudo /usr/share/bcc/tools/tcpconnect -p "$PID"

Real‑world win: A “tools‑enabled” agent suddenly tries to curl pastebins? You’ll see it and can kill it.

5) Handle secrets safely (no plaintext env, use Podman secrets or pass)

Avoid exporting API keys and tokens into global environments or baking them into images.

Install pass (password-store):

• Debian/Ubuntu (apt):

  sudo apt update && sudo apt install -y pass
  

• Fedora/RHEL/CentOS (dnf):

  sudo dnf install -y pass
  

• openSUSE (zypper):

  sudo zypper install -y pass
  

Use Podman secrets to inject runtime credentials with minimal exposure:

# Create a secret from stdin
printf "MY_SUPER_TOKEN\n" | podman secret create ai_api_token -

# Run with secret mapped to an env var in the container
podman run --rm \
  --userns keep-id \
  --network none \
  --secret ai_api_token,type=env,target=AI_API_TOKEN \
  -v "$PWD/app":/app:ro -w /app \
  docker.io/library/python:3.11-slim \
  bash -lc 'python run_inference.py'

If you must use files, store them outside the project tree with strict perms and mount read‑only:

install -m 600 /dev/null ~/.secrets/ai_api_token
printf "MY_SUPER_TOKEN\n" > ~/.secrets/ai_api_token

podman run --rm \
  --read-only \
  -v "$HOME/.secrets/ai_api_token":/run/creds/ai_api_token:ro \
  ...

Real‑world win: Even if a prompt injection lists environment variables, your token isn’t there—and your container has no network anyway.

Putting it together: a minimal secure inference recipe

• Verify inputs:

  sha256sum -c models/model.gguf.sha256sum
  gpg --verify models/model.gguf.asc models/model.gguf
  

• Run sandboxed:

  podman run --rm --userns keep-id --network none --read-only --cap-drop ALL \
  --pids-limit 512 --memory 4g \
  -v "$PWD/app":/app:ro -w /app \
  docker.io/library/python:3.11-slim python run_inference.py
  

• Observe:

  sudo /usr/share/bcc/tools/tcpconnect -p $(pgrep -f run_inference.py)
  sudo ausearch -k ai-ssh-guard
  

Bonus: Mandatory Access Control (AppArmor/SELinux) profiles

Strengthen containment with MAC. Choose the native default per distro family.

• Debian/Ubuntu (AppArmor):

  sudo apt update && sudo apt install -y apparmor apparmor-utils
  sudo aa-status
  # Put custom profiles under /etc/apparmor.d/ and enforce with:
  sudo aa-enforce /etc/apparmor.d/usr.bin.python3
  

• Fedora/RHEL/CentOS (SELinux):

  sudo dnf install -y selinux-policy-targeted policycoreutils policycoreutils-python-utils setools-console
  getenforce
  # Example: confine container directories
  sudo semanage fcontext -a -t container_file_t "/srv/ai(/.*)?"
  sudo restorecon -Rv /srv/ai
  

• openSUSE (AppArmor by default):

  sudo zypper install -y apparmor apparmor-utils
  sudo aa-status
  

Conclusion and next steps

Open‑source AI stacks don’t need proprietary magic to be safe. With containers, egress control, provenance checks, system auditing, and sane secret handling, you can cut entire classes of AI‑specific risk—prompt injection, exfiltration, and supply‑chain tampering—down to size.

Your next step:

• Pick one live AI workload (local RAG, inference microservice, or notebook) and implement steps 1–3 today.

• Schedule steps 4–5 for your next sprint and add basic alerts.

• Document your “AI Hardening Baseline” in your repo so every new model or toolchain starts secure by default.

Have questions or want a deeper hardening checklist? Tell me your distro and AI stack (framework, container engine, GPU), and I’ll generate a tailored Bash plan.