Posted on
• Artificial Intelligence

Installing CUDA for Artificial Intelligence Workloads

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Installing CUDA for Artificial Intelligence Workloads on Linux (apt, dnf, zypper)

Your GPU is the fastest employee on your machine—if you set it up right. A proper CUDA install can turn overnight AI training jobs into coffee-break sprints. But driver mismatches, missing headers, or the wrong repo can waste hours. This guide gets you from zero to CUDA with clean, reproducible steps for apt, dnf, and zypper, plus verification and real-world usage tips.

Why this matters

  • AI frameworks like PyTorch and TensorFlow accelerate most tensor ops with NVIDIA GPUs via CUDA. Without CUDA, you’re leaving massive performance on the table.

  • A correct install means fewer runtime surprises: fewer segfaults, version clashes, or “CUDA driver version is insufficient” errors.

  • Package-manager installs are repeatable and scriptable, making them ideal for dev machines, CI, and servers.

What you’ll get here:

  • A clear path to install NVIDIA drivers and the CUDA Toolkit from the official NVIDIA repositories (recommended) using apt, dnf, or zypper.

  • A minimal, distro-native “quick install” alternative where it makes sense.

  • Post-install sanity checks and a small real-world example so you know it actually works.

Note: Replace version placeholders (like 12-4) with the latest CUDA minor version your framework supports. Always check your framework’s compatibility matrix.

1) Prep: Verify hardware and build prerequisites

  • Confirm you have an NVIDIA GPU:
    lspci | grep -i nvidia

  • Update your system:

    • apt: sudo apt update && sudo apt -y upgrade
    • dnf: sudo dnf -y upgrade
    • zypper: sudo zypper refresh && sudo zypper update

  • Install compiler, make, DKMS, and kernel headers (needed for the driver module):

    • apt (Ubuntu/Debian):
      sudo apt install -y build-essential dkms linux-headers-$(uname -r)
    • dnf (Fedora/RHEL/Alma/Rocky):
      sudo dnf groupinstall -y "Development Tools"
      sudo dnf install -y kernel-devel-$(uname -r) kernel-headers dkms
    • zypper (openSUSE Leap/Tumbleweed):
      sudo zypper install -y -t pattern devel_basis
      sudo zypper install -y kernel-default-devel dkms

  • If your system uses Secure Boot, be ready to enroll a Machine Owner Key (MOK) for the NVIDIA kernel module, or temporarily disable Secure Boot in firmware.

2) Choose your install path

  • Recommended: Install from NVIDIA’s official CUDA repositories (newer, consistent).

  • Alternative (Ubuntu-only quick path): Use Ubuntu’s native packages for a fast setup.

You need two things:

  • NVIDIA driver (kernel module that lets Linux talk to your GPU).

  • CUDA Toolkit (compiler, libraries, tools). NVIDIA’s cuda meta-package installs both; cuda-toolkit-<version> installs just the toolkit.

Tip: Decide if you want the driver managed by your distro or by NVIDIA. Using the same source for both driver and CUDA often avoids version drift.

3) Install from NVIDIA’s repository (apt, dnf, zypper)

Set a helper variable for the CUDA minor version you target (adjust as needed):

  • CUDA_VER=12-4

A) apt (Ubuntu/Debian)

  • Add NVIDIA’s keyring and repo (pick the correct repo path for your distro; example uses Ubuntu 22.04):
    wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
    sudo dpkg -i cuda-keyring_1.1-1_all.deb
    sudo apt update

  • Install CUDA Toolkit only:
    sudo apt -y install cuda-toolkit-$CUDA_VER

  • Or install driver + toolkit:
    sudo apt -y install cuda

  • Reboot to load the driver:
    sudo reboot

B) dnf (Fedora, RHEL, AlmaLinux, Rocky)

  • Enable the correct repo for your OS. Examples:

    • Fedora 40:
      sudo dnf -y install dnf-plugins-core
      sudo dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/fedora40/x86_64/cuda-fedora40.repo
    • RHEL 9 / AlmaLinux 9 / Rocky 9:
      sudo dnf -y install dnf-plugins-core
      sudo dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/cuda-rhel9.repo

  • Install CUDA Toolkit only:
    sudo dnf -y install cuda-toolkit-$CUDA_VER

  • Or install driver + toolkit:
    sudo dnf -y install cuda

  • Reboot:
    sudo systemctl reboot

C) zypper (openSUSE Leap 15.x)

  • Add NVIDIA’s repo for openSUSE Leap 15 (adjust for your release as needed):
    sudo zypper addrepo --refresh https://developer.download.nvidia.com/compute/cuda/repos/opensuse15/x86_64/cuda-opensuse15.repo
    sudo zypper refresh

  • Install CUDA Toolkit only:
    sudo zypper install -y cuda-toolkit-$CUDA_VER

  • Or install driver + toolkit:
    sudo zypper install -y cuda

  • Reboot:
    sudo reboot

Notes:

  • If the nouveau driver is loaded and prevents NVIDIA from binding, blacklist nouveau and rebuild initramfs. On some distros this is handled automatically by the NVIDIA packages; if not, consult your distro’s docs.

  • For other releases (Ubuntu 20.04, 24.04; Fedora N; SLES), swap the repo URL to match your OS version on the NVIDIA CUDA repo index.

4) Ubuntu-only quick install (distro packages)

If you’re on Ubuntu and want a fast, distro-native setup (often older than NVIDIA’s repo):

  • Install NVIDIA driver:
    sudo ubuntu-drivers autoinstall
    sudo reboot

  • Install CUDA Toolkit from Ubuntu repos:
    sudo apt update && sudo apt -y install nvidia-cuda-toolkit

This is convenient but may lag behind the latest CUDA versions. Always confirm compatibility with your AI framework.

5) Post-install: environment and verification

  • Optional: add CUDA to your shell PATH/LD_LIBRARY_PATH (useful if multi-version):
    echo 'export PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrc
    echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
    source ~/.bashrc

  • Verify the driver sees your GPU:
    nvidia-smi

  • Verify the CUDA compiler:
    nvcc --version

  • Build and run the CUDA samples (confirms toolchain works):
    cuda-install-samples-$CUDA_VER.sh $HOME
    cd ~/NVIDIA_CUDA-$CUDA_VER_Samples
    make -j$(nproc)
    ./bin/x86_64/linux/release/deviceQuery

You should see “Result = PASS” and device details for your GPU.

Real-world example: PyTorch with CUDA

  • Create a fresh environment and install a CUDA-enabled PyTorch build that matches your CUDA runtime (check PyTorch’s install page for the right index URL):
    python3 -m venv ~/venvs/torch && source ~/venvs/torch/bin/activate
    pip install --upgrade pip
    pip install torch --index-url https://download.pytorch.org/whl/cu124

  • Quick check:
    python -c "import torch; print('CUDA:', torch.cuda.is_available()); print('Device:', torch.cuda.get_device_name(0) if torch.cuda.is_available() else None)"

If that prints CUDA: True and your GPU name, you’re ready to accelerate training.

Bonus: performance and reliability tips

  • Persist mode for servers (keeps the driver initialized):
    sudo nvidia-smi -pm 1

  • Lock to a high-performance power state on data-center GPUs when appropriate:
    nvidia-smi -pl <WATTS>
    nvidia-smi -lgc <MIN,MAX>
    Check your GPU’s supported limits first: nvidia-smi -q -d SUPPORTED_CLOCKS

  • Monitor during training:
    nvidia-smi dmon -s pucvmt
    watch -n1 nvidia-smi

  • Keep kernel headers in sync after kernel updates to avoid module build failures.

Troubleshooting quick hits

  • “NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver”:
    Check Secure Boot (MOK enrollment), ensure nouveau is not binding, confirm kernel-devel/headers match uname -r, then reinstall the driver package and reboot.

  • “CUDA driver version is insufficient for CUDA runtime”:
    Your installed driver is too old for the CUDA runtime/framework. Upgrade the driver (ideally via the same NVIDIA repo you used for CUDA).

  • Build errors compiling samples:
    Ensure gcc, g++, and make are installed, and that your user has permissions to write in the samples directory.

Conclusion and next steps

With the driver and CUDA Toolkit in place, your Linux box is now GPU-accelerated and AI-ready. Next:

  • Install your framework of choice (PyTorch or TensorFlow) with a build that matches your CUDA version.

  • Containerize your stack for reproducibility, or script this setup for your team.

  • Benchmark a real model to confirm the speedup on your hardware.

Call to action: Pick a small model you know, run it once on CPU, once on GPU, and measure the difference. If you hit bumps, consult your framework’s CUDA compatibility matrix and align versions using the steps above. Your GPU is ready—put it to work.

Related posts