Posted on
• Artificial Intelligence

ROCm for Artificial Intelligence on Linux

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

ROCm for Artificial Intelligence on Linux: Turn Your AMD GPU into an AI Workhorse

If you think “AI on GPU” automatically means “NVIDIA + CUDA,” you’re missing out. AMD’s ROCm stack has matured into a powerful, open ecosystem for machine learning and HPC — and it runs natively on Linux. Whether you’re fine‑tuning LLMs, accelerating Stable Diffusion, or scaling training across multiple GPUs, ROCm can deliver excellent performance on supported AMD hardware at a compelling cost.

In this guide, you’ll learn why ROCm is worth your time, how to install it cleanly on major Linux distros (apt, dnf, zypper), and how to validate your setup and run real AI workloads.

Why ROCm matters for AI on Linux

  • Open and modular: ROCm’s user-space is open source, with battle‑tested math libraries (rocBLAS, hipBLASLt, MIOpen), communication (RCCL), and HIP (a portable CUDA‑like API).

  • Framework support: PyTorch provides official ROCm wheels; popular inference stacks (e.g., llama.cpp, vLLM, TGI variants) increasingly offer ROCm/HIP support.

  • Modern precision: BF16/FP16 mixed precision, graph optimizations, and kernel fusion paths on supported GPUs.

  • Linux‑first: ROCm development targets Linux, with enterprise‑grade support on Ubuntu LTS, RHEL‑family, and SUSE.

Value in one line: ROCm lets you do serious AI on AMD GPUs without vendor lock‑in — right on Linux.

What you’ll do in this guide

  • Verify your GPU/OS prerequisites for ROCm

  • Install ROCm on Ubuntu/Debian (apt), RHEL/CentOS/Fedora (dnf), and SLES/openSUSE (zypper)

  • Set up environment, permissions, and validate with CLI tools

  • Install a ROCm‑enabled AI framework (PyTorch), and run a quick test

  • See real‑world examples and troubleshooting tips

Note: Package names and repository paths can vary by ROCm release and distro version. Always cross‑check with the official ROCm docs for the exact versions you target.

1) Prerequisites: hardware and OS sanity check

1) Confirm your GPU and driver

  • ROCm officially supports a defined set of AMD GPUs (MI series, select RDNA3/RDNA2). Check the current support matrix in AMD’s ROCm docs.

  • Verify the kernel driver is loaded:

lspci -k | grep -EA3 'VGA|Display'
dmesg | grep -i amdgpu | tail -n 50

2) Check Linux basics

  • You’ll want recent kernel/mesa stacks (your distro’s defaults on supported versions are typically fine).

  • Ensure you have build tools if you plan to compile samples:

sudo apt-get update && sudo apt-get install -y build-essential cmake git
# RHEL/CentOS/Fedora:
sudo dnf groupinstall -y "Development Tools" && sudo dnf install -y cmake git
# SLES/openSUSE:
sudo zypper install -y -t pattern devel_C_C++ && sudo zypper install -y cmake git

3) Optional but recommended: container support

  • If you plan to use ROCm containers:
# Docker
sudo groupadd -f docker
sudo usermod -aG docker $USER
# Podman (RHEL/SUSE-centric alternative)
sudo dnf install -y podman || sudo zypper install -y podman

2) Install ROCm

Below are straightforward, distro‑native ways to add AMD’s ROCm repo and install common packages. Adjust “jammy/noble, rhel9, sles15” to your OS version, and confirm the repo path for your target ROCm release in the official docs.

Ubuntu/Debian (apt)

# 1) Add AMD ROCm GPG key
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://repo.radeon.com/rocm/rocm.gpg.key | sudo gpg --dearmor -o /etc/apt/keyrings/rocm.gpg

# 2) Add ROCm repo (use your Ubuntu codename: focal/jammy/noble)
. /etc/os-release
echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/latest ${UBUNTU_CODENAME} main" | \
  sudo tee /etc/apt/sources.list.d/rocm.list

# 3) Install ROCm user-space (typical AI stack)
sudo apt-get update
sudo apt-get install -y \
  rocm-hip-sdk rocm-dev rocm-smi \
  rocblas hipblaslt rccl miopen-hip

# 4) (Optional) ensure environment is set on login
echo 'export PATH=/opt/rocm/bin:$PATH' | sudo tee /etc/profile.d/rocm.sh
echo 'export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH' | sudo tee -a /etc/profile.d/rocm.sh

RHEL/CentOS/AlmaLinux/Rocky/Fedora (dnf)

# 1) Add ROCm repo (use the matching repo file for your OS; rhel8/rhel9/fedora)
sudo dnf install -y dnf-plugins-core
sudo dnf config-manager --add-repo https://repo.radeon.com/rocm/rhel9/rocm.repo

# 2) Import GPG key (if not embedded in the .repo)
sudo rpm --import https://repo.radeon.com/rocm/rocm.gpg.key

# 3) Install ROCm user-space
sudo dnf makecache
sudo dnf install -y \
  rocm-hip-sdk rocm-dev rocm-smi \
  rocblas hipblaslt rccl miopen-hip

# 4) (Optional) environment for all users
echo 'export PATH=/opt/rocm/bin:$PATH' | sudo tee /etc/profile.d/rocm.sh
echo 'export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH' | sudo tee -a /etc/profile.d/rocm.sh

Tip: If your distro release differs, replace rhel9 with rhel8 or an appropriate Fedora path per ROCm’s repo layout.

SLES/openSUSE (zypper)

# 1) Add ROCm repo (SLES 15 shown; adjust for your variant)
sudo zypper addrepo --refresh https://repo.radeon.com/rocm/sles/15/rocm.repo ROCm

# 2) Import GPG key (if required)
sudo rpm --import https://repo.radeon.com/rocm/rocm.gpg.key

# 3) Install ROCm user-space
sudo zypper refresh
sudo zypper install -y \
  rocm-hip-sdk rocm-dev rocm-smi \
  rocblas hipblaslt rccl miopen-hip

# 4) (Optional) environment
echo 'export PATH=/opt/rocm/bin:$PATH' | sudo tee /etc/profile.d/rocm.sh
echo 'export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH' | sudo tee -a /etc/profile.d/rocm.sh

Notes:

  • Some deployments use versioned directories like /opt/rocm-6.x.y. The /opt/rocm path is commonly provided via symlink; if not, point PATH/LD_LIBRARY_PATH to the versioned directory you installed.

  • If you need AMD’s packaged kernel driver stack, consult the “amdgpu-install” tool from AMD; it can set up both the driver and ROCm and still uses apt/dnf/zypper under the hood.

3) Post‑install setup and validation

1) Permissions for GPU access:

# Add your user to common GPU access groups
sudo usermod -aG video,render $USER
# Log out and back in (or reboot) to pick up group changes

2) Validate ROCm tools:

# Check HIP compiler and device query
/opt/rocm/bin/hipcc --version
/opt/rocm/bin/rocminfo | head -n 50
/opt/rocm/bin/rocm-smi

3) Build and run a HIP sample:

git clone https://github.com/ROCm-Developer-Tools/HIP-Examples.git
cd HIP-Examples/roc-1.0/VectorAdd
make
./vectorAdd

If vectorAdd completes successfully and rocminfo lists your GPU, your ROCm user-space is functional.

4) Install an AI framework on ROCm (PyTorch example)

PyTorch publishes AMD ROCm wheels. Choose the ROCm wheel channel matching your installed ROCm major (consult PyTorch’s “Get Started” page for the latest mapping).

Example for a ROCm 6.x channel:

# Recommended: a fresh virtual environment
python3 -m venv ~/venvs/rocm-ai
source ~/venvs/rocm-ai/bin/activate
pip install --upgrade pip

# Install ROCm-enabled PyTorch (adjust rocm6.x to the channel PyTorch lists)
pip install --index-url https://download.pytorch.org/whl/rocm6.1 torch torchvision torchaudio

Quick GPU sanity test in Python:

python - <<'PY'
import torch
print("Torch version:", torch.__version__)
print("CUDA visible? (should be False on ROCm):", torch.cuda.is_available())
print("HIP visible?", torch.backends.mps.is_built() or hasattr(torch.version, "hip"))
print("AMD GPU count:", torch.cuda.device_count())
for i in range(torch.cuda.device_count()):
    print(i, torch.cuda.get_device_name(i))
x = torch.randn(8192, 8192, device="cuda")
y = torch.randn(8192, 8192, device="cuda")
z = x @ y
print("OK:", z.shape)
PY

Other common stacks:

  • llama.cpp with HIP BLAS:
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build -DGGML_HIPBLAS=ON
cmake --build build -j
# Run with: ./build/bin/llama-cli -m <your-model.gguf> -p "Hello"
  • Containers (great for isolating dependencies):
# Example: run a ROCm-enabled PyTorch image (adjust image/tag to your needs)
docker run --rm -it --device=/dev/kfd --device=/dev/dri \
  --group-add video --ipc=host --shm-size 8G \
  rocm/pytorch:latest bash

Tip: Ensure your container runtime has access to /dev/kfd and /dev/dri devices and your user is in video/render groups.

5) Real‑world tips and troubleshooting

  • Confirm support early: Not every AMD GPU is supported by every ROCm version. Check the GPU support matrix in the ROCm docs that match your distro and kernel.

  • Keep it simple first: Validate with rocminfo, rocm-smi, and a tiny HIP sample before layering frameworks.

  • Environment gotchas:

    • Ensure PATH and LD_LIBRARY_PATH include ROCm’s bin/lib if your distro doesn’t ship profile.d entries.
    • For multi‑GPU rigs, control visibility:
    export HIP_VISIBLE_DEVICES=0,1

  • Performance sanity checks:

    • Use rocblas-bench and rocm-smi to check clocks/temps and observe kernel throughput under load.

  • When in doubt, containerize:

    • AMD publishes container images for ROCm; starting with containers can de‑risk host configuration differences.

Conclusion and next steps (CTA)

ROCm has evolved into a capable, Linux‑first platform for AI on AMD GPUs. With the right repo setup and a few validation steps, you can run PyTorch, LLM inference engines, and custom HIP kernels at full speed — all without vendor lock‑in.

Your next steps:

  • Install ROCm with the apt/dnf/zypper instructions above and validate with rocminfo and a HIP sample.

  • Install a ROCm‑enabled PyTorch wheel and run the GPU sanity test.

  • Try a real workload: clone llama.cpp and run inference, or fine‑tune a model using your favorite trainer on ROCm.

Bookmark the official ROCm docs for your exact versions and keep your stack updated. Have questions or want a deeper dive into performance tuning, multi‑GPU RCCL, or mixed precision? Tell me your distro, GPU model, and ROCm version — I’ll help you optimize your setup.

Related posts