Posted on
• Artificial Intelligence

Artificial Intelligence for AlmaLinux

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence on AlmaLinux: A Practical, Bash-First Guide

If you’ve ever tried to turn an enterprise-grade Linux box into an AI workstation or inference server, you already know the pain: drivers that don’t match kernels, Python environments that drift, and containers that behave differently under SELinux. The good news is that AlmaLinux, with its RHEL-compatibility and long-term stability, is an excellent foundation for reliable AI development and deployment—on laptops, servers, and clusters.

This article shows you, step by step, how to get AI working smoothly on AlmaLinux, with cross-distro notes and installation commands for apt, dnf, and zypper. You’ll prep the OS, wire up NVIDIA CUDA or AMD ROCm (or go CPU-only), install a Python AI stack, run a real workload, and serve models cleanly with containers.

Why AlmaLinux for AI?

  • Enterprise stability: Binary-compatible with RHEL, long life cycles, predictable updates.

  • Reproducibility: EPEL and CRB (PowerTools) give you modern tooling while keeping the base solid.

  • Containers first: Podman is the default on RHEL-like systems; Docker is equally viable.

  • Secure by default: SELinux, firewalld, and hardened defaults help you ship production-grade inference services.

  • Scales up: Plays nicely on bare metal, VMs, and HPC clusters.

1) Prepare the base OS (AlmaLinux focus, plus cross-distro equivalents)

On AlmaLinux (and other RHEL-like systems), enable EPEL and CRB, update, and install developer tools.

AlmaLinux/RHEL/CentOS Stream (dnf):

sudo dnf -y update
sudo dnf -y install epel-release
sudo dnf config-manager --set-enabled crb
sudo dnf -y groupinstall "Development Tools"
sudo dnf -y install git cmake pkgconf-pkg-config python3 python3-pip python3-setuptools
# Optional: headers for kernel modules (useful for GPU driver builds)
sudo dnf -y install kernel-devel-$(uname -r) kernel-headers

Debian/Ubuntu (apt):

sudo apt update && sudo apt -y upgrade
sudo apt -y install build-essential git cmake pkg-config python3 python3-pip python3-venv python3-setuptools

openSUSE Leap/Tumbleweed (zypper):

sudo zypper refresh
sudo zypper update -y
sudo zypper install -y -t pattern devel_C_C++
sudo zypper install -y git cmake pkg-config python3 python3-pip python3-venv python3-setuptools

Create a clean Python virtual environment for your AI work:

python3 -m venv ~/ai-venv
source ~/ai-venv/bin/activate
python -m pip install --upgrade pip wheel setuptools

Tip: After kernel updates (especially before GPU driver installs), reboot:

sudo reboot

2) Pick your acceleration: NVIDIA CUDA, AMD ROCm, or CPU-only

You can do a lot on CPU today, but GPUs help significantly for training and larger inference workloads. Choose what matches your hardware.

A) NVIDIA CUDA (recommended for NVIDIA GPUs)

  • AlmaLinux/RHEL 9 (dnf):
sudo dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/cuda-rhel9.repo
sudo dnf -y install kernel-devel-$(uname -r) kernel-headers gcc make dkms
sudo dnf -y install cuda-toolkit
# Optional for containers (after repo is added):
sudo dnf -y install nvidia-container-toolkit
  • Ubuntu 22.04+ (apt):
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update
sudo apt -y install cuda-toolkit
# Optional for containers:
sudo apt -y install nvidia-container-toolkit
  • openSUSE Leap 15 (zypper):
sudo zypper addrepo https://developer.download.nvidia.com/compute/cuda/repos/opensuse15/x86_64/cuda-opensuse15.repo
sudo zypper refresh
sudo zypper install -y cuda-toolkit
# Optional for containers (if available in the NVIDIA repo):
sudo zypper install -y nvidia-container-toolkit

Reboot after driver/toolkit installs:

sudo reboot

B) AMD ROCm (recommended for supported AMD GPUs)

  • AlmaLinux/RHEL 9 (dnf):
sudo curl -o /etc/yum.repos.d/rocm.repo https://repo.radeon.com/rocm/yum/6.1/rocm.repo
sudo dnf -y install rocm-dev
  • Ubuntu 22.04 (apt):
wget -qO - https://repo.radeon.com/rocm/rocm.gpg.key | gpg --dearmor | sudo tee /usr/share/keyrings/rocm.gpg >/dev/null
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/6.1 jammy main" | sudo tee /etc/apt/sources.list.d/rocm.list
sudo apt update
sudo apt -y install rocm-dev
  • openSUSE (zypper): At time of writing, an official ROCm zypper repository is not provided. Consider containers or source builds. Check AMD’s documentation for the latest support matrix.

C) CPU-only (portable and simple) You can skip GPU drivers entirely. Many frameworks provide excellent CPU backends, and OpenVINO can accelerate CPU inference on Intel hardware.

3) Build your Python AI stack (Torch, TensorFlow, ONNX Runtime, OpenVINO)

Activate your venv first:

source ~/ai-venv/bin/activate

Baseline scientific stack:

pip install numpy scipy pandas matplotlib jupyter

PyTorch options:

  • CPU-only:
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
  • NVIDIA CUDA 12.1 (example):
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
  • AMD ROCm 6.0+ (example):
pip install --index-url https://download.pytorch.org/whl/rocm6.0 torch torchvision torchaudio

TensorFlow (CPU by default; GPU requires specific CUDA/cuDNN versions):

pip install tensorflow

ONNX Runtime:

  • CPU:
pip install onnxruntime
  • NVIDIA GPU:
pip install onnxruntime-gpu

OpenVINO (great for CPU acceleration):

pip install openvino-dev

Install ffmpeg (handy for audio and video tasks):

  • dnf:
sudo dnf -y install ffmpeg
  • apt:
sudo apt -y install ffmpeg
  • zypper:
sudo zypper install -y ffmpeg

4) Try a real workload (two quick, meaningful tests)

A) Verify GPU with PyTorch:

python - << 'PY'
import torch
print("Torch version:", torch.__version__)
print("CUDA available:", torch.cuda.is_available())
if torch.cuda.is_available():
    print("CUDA device:", torch.cuda.get_device_name(0))
    x = torch.randn(10000, 10000, device='cuda')
    print("Compute OK:", (x @ x.t()).mean().item())
PY

B) Transcribe audio locally (faster-whisper + ffmpeg)

pip install faster-whisper
# Download a sample file
curl -L -o sample.wav https://github.com/lingthio/whisper-small-files/raw/main/jfk.wav
python - << 'PY'
from faster_whisper import WhisperModel
model = WhisperModel("small", device="cuda" if __import__('torch').cuda.is_available() else "cpu")
segments, info = model.transcribe("sample.wav")
print("Detected language:", info.language)
for s in segments:
    print(f"[{s.start:.2f} -> {s.end:.2f}] {s.text}")
PY

This gives you an immediate, practical win: speech-to-text on your AlmaLinux box, GPU-accelerated if available.

5) Serve models with containers (Podman/Docker) and keep ops clean

Install a container engine:

  • AlmaLinux/RHEL (dnf):
sudo dnf -y install podman
  • Ubuntu/Debian (apt):
sudo apt -y install podman
  • openSUSE (zypper):
sudo zypper install -y podman

Optional: NVIDIA GPUs inside containers (after installing the NVIDIA repository/toolkit in Step 2)

  • dnf:
sudo dnf -y install nvidia-container-toolkit
  • apt:
sudo apt -y install nvidia-container-toolkit
  • zypper:
sudo zypper install -y nvidia-container-toolkit

Example: Serve an LLM quickly with Ollama (simple, local, supports GPU where available).

Install Ollama (vendor script, works across distros):

curl -fsSL https://ollama.com/install.sh | sh
# Start service (systemd)
sudo systemctl enable --now ollama

Pull and run a model:

ollama pull llama3
ollama run llama3

Prefer containers? You can run Ollama in a container too:

  • Podman (CPU; for NVIDIA GPU support, configure nvidia-container-toolkit and consult its Podman integration docs):
podman run --rm -d --name ollama -p 11434:11434 docker.io/ollama/ollama
podman logs -f ollama

Open a firewall port for remote access (AlmaLinux/RHEL with firewalld):

sudo dnf -y install firewalld
sudo systemctl enable --now firewalld
sudo firewall-cmd --add-port=11434/tcp --permanent
sudo firewall-cmd --reload

Note on SELinux: Keep SELinux enforcing if you can. If your container or service binds to a nonstandard port, label it instead of disabling SELinux. Example for port 11434:

sudo dnf -y install policycoreutils-python-utils
sudo semanage port -a -t http_port_t -p tcp 11434

Real-world tips for smooth AI on AlmaLinux

  • Pin kernels on production nodes: Upgrade deliberately, especially with out-of-tree GPU drivers.

  • Use dedicated venvs per project: Avoid dependency drift and make rollbacks easy.

  • Prefer containers for serving: Immutable images reduce “works on my machine” issues.

  • Monitor thermals and power: For sustained inference, watch nvidia-smi, rocm-smi, and sensors.

  • Document exact versions: Driver, CUDA/ROCm, framework wheels—this is your golden recipe.

Conclusion and Call to Action

AlmaLinux gives you a durable foundation for AI: stable kernels, strong security defaults, and a container-first culture. With the steps above, you can go from clean install to GPU-accelerated inference or LLM serving in under an hour—and keep it maintainable.

Next steps:

  • Turn one of today’s tests into a systemd service or container you can auto-start on boot.

  • Add monitoring (Prometheus + Grafana) to track GPU/CPU utilization and latency.

  • Explore model optimization with ONNX Runtime or OpenVINO for lower-latency CPU serving.

Have a specific workload or hardware stack? Tell me what you’re building (GPU model, target models, latency/SLA) and I’ll help you tailor an AlmaLinux-native setup script.

Related posts