Artificial Intelligence Package Management Problems: Surviving Dependency Hell on Linux

You finally sit down to train that model. One pip install later and… everything breaks. Torch wants a different CUDA. Your distro’s Python fights with pip. A minor driver update silently changes ABI compatibility. Minutes become hours.

This post explains why AI package management is uniquely painful on Linux and gives you a sane, repeatable way out—using only Bash, your distro’s package manager, and a few discipline rules. You’ll leave with practical commands (apt, dnf, zypper included), version-pinning strategies, and a debugging checklist you can paste into your shell.

---

Why AI packaging really is harder than normal dev

• Heavy native code: AI frameworks ship big C/C++ and GPU-accelerated binaries. Binary compatibility is fragile across glibc, GCC/C++ ABI, and kernel/driver/toolkit versions.

• Rapid cadence: PyTorch, TensorFlow, CUDA/ROCm, cuDNN, and ONNX move fast and not always in sync.

• Multiple stacks: CPU-only, NVIDIA CUDA/cuDNN, AMD ROCm—each has its own wheel sets, build flags, and runtime expectations.

• Python vs system: Mixing sudo pip with distro Python, or installing random headers globally, is a recipe for “works today, segfault tomorrow.”

The fix isn’t magic; it’s process. Isolate, pin, prefer prebuilt binaries, and containerize when in doubt.

---

Step 0: Bootstrap your machine once

Install a minimal toolchain, Python basics, and BLAS. Use your distro packages for these base layers only.

Apt (Debian/Ubuntu):

sudo apt update
sudo apt install -y python3 python3-venv python3-pip build-essential git pkg-config libopenblas-dev

DNF (Fedora/RHEL/CentOS/Alma/Rocky):

sudo dnf -y groupinstall "Development Tools"
sudo dnf -y install python3 python3-pip python3-virtualenv git pkg-config openblas-devel

Zypper (openSUSE/SLE):

sudo zypper refresh
sudo zypper install -y -t pattern devel_basis
sudo zypper install -y python3 python3-pip python3-virtualenv git pkg-config openblas-devel

Notes:

• Never use sudo pip. Keep system Python clean; do AI work inside an isolated environment.

• If you build from source later, these dev packages save hours.

---

1) Isolate environments (and never sudo pip)

Use venv or Conda/Mamba. venv is enough for most Python-first workflows and plays nicely with your distro.

Create and activate a venv:

python3 -m venv ~/.venvs/ai
source ~/.venvs/ai/bin/activate
python -m pip install -U pip setuptools wheel

Give each project its own venv. Name them after the exact AI stack to avoid confusion:

python3 -m venv ~/.venvs/ai-py310-torch23-cu121

Deactivate with:

deactivate

---

2) Pin exact versions (Python and system)

Once you have a working stack, freeze it. Reproducibility beats “latest”.

Pin Python packages:

# inside your venv
pip install -r requirements.txt  # or install interactively first
pip freeze > requirements.lock

System-level pinning prevents accidental upgrades that break ABI.

Apt (hold a package, e.g., OpenBLAS):

sudo apt-mark hold libopenblas-dev
# to unhold:
sudo apt-mark unhold libopenblas-dev

DNF (requires plugins):

sudo dnf -y install dnf-plugins-core
sudo dnf versionlock add openblas*
# list/remove locks:
sudo dnf versionlock list
sudo dnf versionlock delete openblas*

Zypper:

sudo zypper addlock openblas-devel
# list/remove:
sudo zypper locks
sudo zypper removelock openblas-devel

Also pin Python itself per project if you can. Tools like pyenv or container images help here.

---

3) Prefer prebuilt wheels that match your hardware

Let the framework vendors ship you the right binary. Don’t mix system CUDA/ROCm unless you must.

Detect what you have:

# NVIDIA
command -v nvidia-smi && nvidia-smi || echo "No NVIDIA GPU detected."
# CUDA compiler (if installed system-wide)
command -v nvcc && nvcc --version || echo "No nvcc in PATH."

# AMD
command -v rocminfo && rocminfo || echo "No ROCm detected."

Install PyTorch wheels that bundle the right runtime (examples use 2.3.x—check pytorch.org for the latest matrix):

• CPU-only:

pip install "torch==2.3.1+cpu" "torchvision==0.18.1+cpu" "torchaudio==2.3.1+cpu" \
  --index-url https://download.pytorch.org/whl/cpu

• NVIDIA CUDA 12.1 bundled:

pip install "torch==2.3.1+cu121" "torchvision==0.18.1+cu121" "torchaudio==2.3.1+cu121" \
  --index-url https://download.pytorch.org/whl/cu121

• AMD ROCm 6.0 bundled:

pip install "torch==2.3.1+rocm6.0" \
  --index-url https://download.pytorch.org/whl/rocm6.0

Verify:

python -c "import torch; print('Torch', torch.__version__, 'CUDA:', torch.version.cuda, 'Is CUDA available?', torch.cuda.is_available())"

For TensorFlow, prefer the official install selector; TF’s GPU extras and CUDA/cuDNN bundling varies by release. CPU-only is usually:

pip install "tensorflow==2.15.*"

Why this works: these wheels carry the compatible runtime (CUDA/ROCm) so you don’t fight your distro’s toolkit. You still need a working GPU driver installed at the OS level for acceleration, but you avoid mismatched user-space libraries.

---

4) If you must build, prepare the system properly

Sometimes a wheel doesn’t exist for your arch/distro/Python. Then you need headers and compilers.

Apt:

sudo apt update
sudo apt install -y build-essential cmake ninja-build python3-dev pkg-config libopenblas-dev

DNF:

sudo dnf -y groupinstall "Development Tools"
sudo dnf -y install cmake ninja-build python3-devel pkg-config openblas-devel

Zypper:

sudo zypper install -y -t pattern devel_basis
sudo zypper install -y cmake ninja python3-devel pkg-config openblas-devel

Build example (generic Python package with native code):

export MAX_JOBS=$(nproc)
pip install --no-binary=:all: somepackage

Or for a local source tree:

pip install -U pip setuptools wheel
pip install -e ".[dev]"

Tip: When builds fail, inspect missing libs:

ldd $(python -c 'import sysconfig,sys; print(sysconfig.get_config_var("LIBDIR") or "")') 2>/dev/null || true

and search the missing .so with:

ldconfig -p | grep -E 'cudnn|cuda|mkl|openblas'

---

5) Contain the chaos with containers (especially for mixed stacks)

Containers keep conflicting stacks separate. Podman works rootless and is available in all major distros.

Install Podman:

Apt:

sudo apt update
sudo apt install -y podman

DNF:

sudo dnf -y install podman

Zypper:

sudo zypper install -y podman

Run a CPU-only workflow:

podman run --rm -it -v "$PWD":/work -w /work docker.io/python:3.11-slim bash -lc '
  python -m venv .venv && . .venv/bin/activate &&
  pip install -U pip &&
  pip install "torch==2.3.1+cpu" --index-url https://download.pytorch.org/whl/cpu &&
  python -c "import torch; print(torch.__version__, torch.cuda.is_available())"
'

GPU passthrough requires host drivers and vendor tooling. For NVIDIA, use the NVIDIA Container Toolkit; for AMD ROCm, pass the appropriate devices. Consult vendor docs for your distro and driver version.

---

Debugging checklist (paste-friendly)

• Check which Python/pip you’re using:

which -a python python3 pip
python -c "import sys; print(sys.version)"
pip debug --verbose | sed -n '1,120p'

• Confirm Torch sees your GPU/runtime:

python - <<'PY'
import torch, sys
print("Torch:", torch.__version__)
print("CUDA Runtime in wheel:", torch.version.cuda)
print("CUDA available?:", torch.cuda.is_available())
print("Devices:", [torch.cuda.get_device_name(i) for i in range(torch.cuda.device_count())] if torch.cuda.is_available() else [])
PY

• Find missing shared libs:

python - <<'PY'
import os, torch
libdir = os.path.dirname(torch.__file__)
target = [os.path.join(libdir, 'lib', x) for x in ('libtorch_cuda.so','libc10_cuda.so')]
print('\n'.join([t for t in target if os.path.exists(t)]))
PY
for f in $(python - <<'PY'
import os, torch
libdir = os.path.dirname(torch.__file__)
for x in ('libtorch_cuda.so','libc10_cuda.so'):
    p=os.path.join(libdir,'lib',x)
    print(p if os.path.exists(p) else '')
PY
); do
  [ -n "$f" ] && echo "Checking $f" && ldd "$f" | grep "not found" || true
done

• Check driver/toolkit versions:

nvidia-smi || true
nvcc --version || true
rocminfo || true

---

Real-world examples

1) PyTorch wants CUDA 12.1, host has CUDA 12.3 libs

• Symptom: ImportError mentioning libcudart.so.X mismatch.

• Fix: Ignore system CUDA; install the cu121-bundled wheels inside a venv:

python3 -m venv ~/.venvs/torch-cu121 && source ~/.venvs/torch-cu121/bin/activate
pip install -U pip
pip install "torch==2.3.1+cu121" --index-url https://download.pytorch.org/whl/cu121

2) ONNX Runtime vs protobuf mismatch

• Symptom: onnxruntime fails to load due to libprotobuf.so version error.

• Fix: Pin both in a clean venv:

python3 -m venv ~/.venvs/onnx && source ~/.venvs/onnx/bin/activate
pip install -U pip
pip install "onnxruntime==1.18.0" "protobuf==4.25.3"
pip freeze > requirements.lock

3) CPU build performance is terrible

• Symptom: Training/inference slow without MKL/OpenBLAS.

• Fix: Ensure OpenBLAS dev libs were present before building Numpy/Scipy, or use wheels:

# System libs (choose your distro)
# apt:
sudo apt install -y libopenblas-dev
# dnf:
sudo dnf -y install openblas-devel
# zypper:
sudo zypper install -y openblas-devel

# Then inside venv, force rebuild or install wheels:
pip install -U --force-reinstall --no-cache-dir numpy scipy

---

Conclusion and next step (CTA)

AI packaging pain is real—but manageable when you:

• Isolate every project’s Python environment.

• Pin versions (Python and system) once you’ve got a working matrix.

• Prefer vendor wheels that bundle the right CUDA/ROCm runtime.

• Containerize when projects or teams need different stacks.

• Debug like a sysadmin: verify paths, drivers, and shared libraries.

Your next step:

• Create a bootstrap script for your team containing the exact commands above for your distro and GPU stack.

• Freeze a known-good environment with pip freeze > requirements.lock and commit it.

• If you have recurring GPU/runtime mismatches, consider standardizing on containers for all training/inference nodes.

If you’d like, share your stack (distro, GPU, Python, framework versions) and the errors you see—I can help you map it to a clean, reproducible install plan.