Posted on
• Artificial Intelligence

Artificial Intelligence for Red Hat Enterprise Linux

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence for Red Hat Enterprise Linux: A Practical Bash-First Playbook

If you manage Red Hat Enterprise Linux (RHEL) systems, you’ve probably wondered: can I bring AI directly to my servers—securely, offline, and without sending data to third-party clouds? Good news: yes. Modern open models and container-native workflows make it straightforward to run AI locally on RHEL and wire it into your Bash-centric workflows.

In this post, you’ll set up a local large language model (LLM) on RHEL using Podman, create Bash helpers that turn AI into a daily driver, and try a few real-world ops recipes. Cross-distro instructions (apt, dnf, zypper) are included so you can test on RHEL, Fedora, CentOS Stream, Ubuntu/Debian, or openSUSE.

Why AI on RHEL makes sense now

  • Privacy and compliance: keep prompts, logs, and outputs on your own machines.

  • Low friction: Podman on RHEL runs containers rootless by default; no daemon needed.

  • Cost control: quantized open models run acceptably on CPUs; GPUs accelerate further when available.

  • Reliability: RHEL’s stability, SELinux, and systemd make AI services predictable and production-friendly.

What you’ll build

  • A local model server (Ollama) running in a rootless Podman container.

  • Bash functions that let you “ask AI” from the terminal, explain commands, or draft scripts.

  • Drop-in, real-world examples: summarize logs, propose remediations, and generate config scaffolds.

  • Optional systemd integration for long-running AI services.

Prerequisites: CLI tools you’ll use

Install Podman (for containers), curl and jq (for API calls), and git (optional for examples).

On RHEL/Fedora/CentOS Stream (dnf):

sudo dnf install -y podman curl jq git

On Ubuntu/Debian (apt):

sudo apt update
sudo apt install -y podman curl jq git

On openSUSE Leap/Tumbleweed (zypper):

sudo zypper refresh
sudo zypper install -y podman curl jq git

Tip for RHEL: rootless Podman typically “just works.” Ensure your user is not artificially restricted from creating user namespaces (default is OK on supported RHEL releases).

Step 1: Run a local LLM on RHEL with Podman (Ollama)

Ollama is a compact model server that pulls and runs open models (e.g., Llama 3, Mistral). We’ll run it rootless with Podman and persist models in a named volume.

1) Create a persistent volume:

podman volume create ollama

2) Start the Ollama server:

podman run -d --name ollama \
  -p 11434:11434 \
  -v ollama:/root/.ollama \
  --restart=always \
  docker.io/ollama/ollama:latest

3) Verify the API is listening:

curl http://localhost:11434/api/tags

4) Pull a model (example: Llama 3 8B):

podman exec -it ollama ollama pull llama3:8b

5) Quick sanity check:

podman exec -it ollama ollama run llama3:8b "In one sentence, explain what RHEL is."

Notes:

  • CPU-only works; performance improves with more cores/RAM.

  • If you have a GPU and the appropriate runtime configured, Ollama can use it; consult your GPU vendor’s container toolkit docs for Podman.

Step 2: Wire Bash to your model (AI helpers)

Add a few helpers to your shell so you can ask questions, explain commands, and draft snippets—right from Bash.

Append these functions to your ~/.bashrc (or ~/.bash_profile) and reload your shell (source ~/.bashrc):

# Core AI function (non-streaming)
ai() {
  local prompt="$*"
  curl -s http://localhost:11434/api/generate \
    -H "Content-Type: application/json" \
    -d "{\"model\":\"llama3:8b\",\"prompt\":\"${prompt//\"/\\\"}\",\"stream\":false}" \
  | jq -r '.response'
}

# Explain a Linux command
explain() {
  ai "Explain this Linux command. Provide a concise description, bullet-list flags, and one cautionary note:\n\n$*"
}

# Ask AI to write code-only Bash snippets
writebash() {
  ai "Write a Bash snippet that $*. Only output code; include brief inline comments."
}

# Summarize input from stdin (e.g., logs)
summarize_stdin() {
  local topic="${1:-Summarize the following text:}"
  local data
  data="$(cat)" || return 1
  jq -Rs --arg model "llama3:8b" --arg p "$topic\n\n$data" \
    '{model:$model, prompt:$p, stream:false}' \
  | curl -s http://localhost:11434/api/generate -d @- \
  | jq -r '.response'
}

Try it out:

explain "dnf update -y"
writebash "backs up /etc/ssh/sshd_config with a timestamp and validates syntax"
dmesg | tail -n 200 | summarize_stdin "Summarize error messages and suggest next steps:"

Step 3: Real-world ops recipes

Here are practical patterns you can adapt to your environment.

1) Summarize system logs and propose actions:

journalctl -p 3 -n 400 --no-pager \
| summarize_stdin "Summarize these recent error logs into the top 5 issues with likely causes and actionable remediations:"

2) Turn a rough task into a Bash-ready scaffold:

writebash "rotates logs in /var/log/myapp weekly, keeping 8 archives, and restarts the service if size exceeds 200MB"

3) Generate a Podmanfile (Containerfile) draft for a simple app:

ai "Draft a minimal Containerfile for a Python 3.11 CLI tool on RHEL UBI, using a non-root user, and explain SELinux-friendly volume mounts."

4) Hardening checklist from live config:

awk '/^#|^$/ {next} {print}' /etc/ssh/sshd_config \
| summarize_stdin "Review this sshd_config and list 5 hardening improvements with exact directives:"

5) Explain a risky one-liner before you run it:

explain "find /var -type f -mtime +30 -delete"

Step 4: Make it ops-grade (limits, services, updates)

  • Resource limits (avoid starving other workloads):
podman stop ollama
podman rm ollama
podman run -d --name ollama \
  -p 11434:11434 \
  -v ollama:/root/.ollama \
  --cpus=4 -m 8g \
  --restart=always \
  docker.io/ollama/ollama:latest
  • Keep it running with systemd (user scope):
podman generate systemd --name ollama --files --new
mkdir -p ~/.config/systemd/user
mv container-ollama.service ~/.config/systemd/user/
systemctl --user daemon-reload
systemctl --user enable --now container-ollama.service
loginctl enable-linger "$USER"

  • Updates and offline operation:

    • Models live in the ollama volume. You can back it up:
    podman run --rm -v ollama:/data -v "$PWD":/backup \
  alpine tar czf /backup/ollama-models-backup.tgz -C / data
    • To update the server:
    podman pull docker.io/ollama/ollama:latest
podman stop ollama && podman rm ollama
# re-run with the same options as before

  • Security notes:

    • Prefer rootless Podman.
    • For bind mounts on RHEL with SELinux enforcing, add :Z to relabel the mount (e.g., -v $PWD/models:/root/.ollama:Z). Named volumes typically don’t require relabeling.
    • Audit who can access the API port (11434). Bind to localhost or firewall accordingly.

Troubleshooting quick hits

  • “Connection refused” on 11434: podman logs -f ollama and verify the container is healthy.

  • Slow responses: try a smaller model (e.g., llama3:instruct or a 7–8B variant), increase CPUs/memory caps, or consider GPU acceleration.

  • SELinux denials: check sudo journalctl -t setroubleshoot -g ollama or use sealert for guidance; prefer named volumes or proper :Z relabeling for bind mounts.

Optional: Alternative native tools

Prefer building a tiny local runner without containers? You can compile llama.cpp for CPU inference.

Install build prerequisites:

  • On RHEL/Fedora/CentOS Stream (dnf):
sudo dnf install -y git gcc gcc-c++ cmake make
  • On Ubuntu/Debian (apt):
sudo apt update
sudo apt install -y git build-essential cmake
  • On openSUSE (zypper):
sudo zypper refresh
sudo zypper install -y git gcc gcc-c++ cmake make

Then:

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
make -j
# Follow the repo docs to download/quantize a model and run:
./main -m ./models/YourModel.gguf -p "Hello from RHEL"

This route is great for minimal dependencies and scripted environments.

Where RHEL fits in your AI roadmap

RHEL gives you a stable, secure base for AI services—especially when you need offline inference, predictable updates, and container-native operations. As your use cases grow, you can pair this approach with broader Red Hat tooling (e.g., GitOps, Ansible automation, and enterprise container registries) and graduate to GPU-backed clusters when necessary.

Conclusion and next steps

You just:

  • Stood up a local LLM on RHEL using Podman.

  • Added Bash helpers to explain commands, draft scripts, and summarize logs.

  • Practiced real ops recipes you can adapt to your environment.

  • Wrapped it with resource limits and systemd for reliability.

Your next steps: 1) Add ai/explain/summarize helpers to your team’s shell profiles.
2) Curate 2–3 “golden prompts” for your environment (e.g., log triage, RFC drafting).
3) Pilot on a non-production RHEL host, collect feedback, and iterate on model size and prompts.

If you found this useful, try swapping in a different model (e.g., Mistral) and compare outputs. AI on RHEL doesn’t have to be complicated—keep it containerized, keep it scriptable, and keep it in your control.

Related posts