Learning Artificial Intelligence Troubleshooting: A Bash-First Playbook for Linux

Ever shipped a model that works on your laptop but crashes on a server with “Killed”, “Illegal instruction”, or “CUDA error: device-side assert triggered”? AI stacks are tall—Python, native libraries, CUDA/cuDNN, kernels, drivers—and most failures are environmental, not algorithmic.

This post gives you a reproducible, Bash-first troubleshooting playbook you can run on any Linux distribution. You’ll get practical commands, real-world failure signatures, and clear next steps. By the end, you’ll know how to isolate the problem fast, fix it cleanly, and avoid it next time.

Why this matters

• Environment drift is the #1 source of breakage in AI workflows.

• Small mismatches (CUDA version, CPU instruction flags, BLAS threads) can cause crashes that look like “mystery bugs.”

• A few habits—snapshotting your system, isolating environments, and reading native errors—save hours.

Prerequisites: tools you’ll want on every box

Install core development tools, Python, and utilities. Choose your package manager and run the corresponding block.

Apt (Debian/Ubuntu):

sudo apt update
sudo apt install -y \
  python3 python3-venv python3-pip \
  git curl cmake pkg-config \
  build-essential \
  libopenblas-dev \
  strace gdb lsb-release pciutils

DNF (Fedora/RHEL/CentOS Stream):

sudo dnf -y groupinstall "Development Tools"
sudo dnf install -y \
  python3 python3-virtualenv python3-pip \
  git curl cmake pkgconf-pkg-config \
  openblas-devel \
  strace gdb redhat-lsb-core pciutils

Zypper (openSUSE/SLE):

sudo zypper refresh
sudo zypper in -y -t pattern devel_basis
sudo zypper in -y \
  python3 python3-virtualenv python3-pip \
  git curl cmake pkgconf-pkg-config \
  openblas-devel \
  strace gdb lsb-release pciutils

Optional (handy monitors):

• nvtop (GPU monitoring)

  • Apt: sudo apt install -y nvtop
  • DNF: sudo dnf install -y nvtop
  • Zypper: sudo zypper in -y nvtop

• htop

  • Apt: sudo apt install -y htop
  • DNF: sudo dnf install -y htop
  • Zypper: sudo zypper in -y htop

Note: GPU drivers/CUDA toolkits vary by vendor/distros. Prefer framework wheels that bundle CUDA (e.g., PyTorch cu121 wheels) unless you know you need system CUDA.

---

The 5-step AI troubleshooting playbook

1) Snapshot the system: facts before fixes

Collect OS, kernel, CPU flags, GPU/driver, Python and key env vars. Paste this into tickets/PRs to cut guesswork.

echo "=== OS / Kernel ==="
uname -a
command -v lsb_release >/dev/null && lsb_release -a || cat /etc/*release

echo "=== CPU ==="
grep -m1 "model name" /proc/cpuinfo
grep -m1 "flags" /proc/cpuinfo

echo "=== Memory ==="
free -h

echo "=== Python ==="
python3 -V
python3 -c "import sys, platform; print(sys.executable); print(platform.python_implementation())" 2>&1 || true

echo "=== Pip packages (top) ==="
python3 -m pip --version
python3 -m pip list | sed -n '1,40p'

echo "=== GPU / Drivers (if any) ==="
command -v nvidia-smi && nvidia-smi || echo "nvidia-smi not found"
command -v nvcc && nvcc --version || echo "nvcc not found"

echo "=== Environment ==="
echo "PATH=$PATH"
echo "LD_LIBRARY_PATH=${LD_LIBRARY_PATH:-<empty>}"

Real-world win:

• “Illegal instruction (core dumped)” frequently maps to missing CPU flags (e.g., AVX/AVX2). If grep flags lacks avx2 but the wheel requires it, switch to a compatible build (e.g., CPU-only or older baseline) rather than chasing ghost bugs.

2) Reproduce in a clean, isolated environment

Most issues disappear (or become obvious) in a minimal venv. Keep project deps pinned and verifiable.

Create a fresh venv:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip wheel

Install only what you need to reproduce:

pip install "torch" "torchvision"  # or tensorflow, transformers, etc.
pip check
pip freeze > requirements.lock

Helpful mechanics:

• Flush caches if wheels got corrupted: pip cache purge

• Verify pip environment: pip debug

• If using bundled-CUDA wheels for PyTorch (example index URL): pip install --index-url https://download.pytorch.org/whl/cu121 torch torchvision Adjust the index for your target CUDA version; prefer CPU-only if no GPU.

Real-world win:

• Build-time errors vanish when a project-specific venv isolates you from system-wide packages and conflicting libs.

3) GPU and CUDA sanity checks

Mismatch between driver, CUDA toolkit, and framework build is the most common GPU failure.

Quick checks:

# Driver and device visibility
nvidia-smi

# PyTorch CUDA binding
python - <<'PY'
import torch
print("is_available:", torch.cuda.is_available())
print("torch cuda:", torch.version.cuda)
print("device count:", torch.cuda.device_count())
PY

# TensorFlow CUDA binding
python - <<'PY'
import tensorflow as tf
print("built with cuda:", tf.test.is_built_with_cuda())
print("gpus:", tf.config.list_physical_devices("GPU"))
try:
    from tensorflow.python.platform import build_info as bi
    print("cuda:", bi.build_info.get("cuda_version"))
    print("cudnn:", bi.build_info.get("cudnn_version"))
except Exception as e:
    print("build info not available:", e)
PY

# Check common runtime libs
ldconfig -p | grep -E 'cudart|cublas|cudnn' || true

Typical errors and fixes:

• “CUDA error: invalid device ordinal”

  • Fix: export CUDA_VISIBLE_DEVICES to a valid index or clear it.

  export CUDA_VISIBLE_DEVICES=0
  

• “libcudart.so.11.x: cannot open shared object file”

  • Fix: ensure the framework wheel bundles the needed libs or that your LD_LIBRARY_PATH/system paths expose them. Prefer bundled-CUDA wheels to avoid chasing system CUDA.

• “Driver version is insufficient”

  • Fix: upgrade GPU driver to a version compatible with your framework’s CUDA runtime. Use the vendor-recommended method for your distro.

Real-world win:

• Teams standardize on a single wheel index (e.g., cu121) across machines; zero system CUDA churn, fewer “works on my machine” bugs.

4) Memory and performance triage (CPU/GPU)

Detect OOMs and tune threads/workers quickly.

Watch live:

watch -n1 'free -h; echo ""; nvidia-smi --query-gpu=memory.total,memory.used --format=csv,noheader 2>/dev/null || true'

When processes die:

dmesg | tail -n50 | grep -i -E 'out of memory|killed process' || true
journalctl -k -g "Out of memory" -n 20 || true

Common mitigations:

• Data loaders:

  # PyTorch DataLoader knobs
  num_workers=2   # start low; increase gradually
  pin_memory=True # if using GPU
  persistent_workers=True
  prefetch_factor=2
  

• Batch and precision:

  • Reduce batch size; try mixed precision (AMP).

• Fragmentation (GPU):

  export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
  

• Determinism and debugging:

  export CUDA_LAUNCH_BLOCKING=1           # clearer CUDA stack traces (slower)
  export TORCH_SHOW_CPP_STACKTRACES=1
  

• CPU threading (avoid oversubscription):

  export OMP_NUM_THREADS=$(nproc)
  export MKL_NUM_THREADS=$(nproc)
  # For PyTorch CPU:
  python - <<'PY'
  import torch, os
  torch.set_num_threads(max(1, os.cpu_count() or 1))
  print("threads:", torch.get_num_threads())
  PY
  

Real-world win:

• “Process killed” with no Python traceback was an OOM killer. Reducing batch size and enabling AMP fixed it immediately.

5) Native library and import errors: read the dynamic linker

When imports crash or segfault, ask the loader what it tried to do.

Find missing symbols/paths:

# Show what libraries are needed and from where
python - <<'PY'
import torch, os
print(torch.__file__)
PY
# Use the printed path to inspect:
ldd /path/to/site-packages/torch/lib/libc10.so

Trace library resolution:

LD_DEBUG=libs python -c "import torch" 2>&1 | head -n 80

Find file access errors near the crash:

strace -f -e trace=openat,access,execve -o trace.log python -c "import torch"
grep -E 'ENOENT|EACCES' trace.log | head

Symbols and relocations (advanced):

# Which symbols are missing?
readelf -Ws /path/to/lib.so | grep missing_symbol_name || true

Illegal instruction checks:

grep -m1 "flags" /proc/cpuinfo
# If 'avx2' is absent, avoid wheels requiring AVX2; choose CPU-only or older baseline builds.

Permissions and locales (sneaky culprits):

# Dataset path access
ls -l /data/path
# Fix if needed (careful with ownership):
sudo chown -R "$USER":"$USER" /data/path

# Locale
locale
export LC_ALL=C.UTF-8
export LANG=C.UTF-8

Real-world win:

• “ImportError: libcusparse.so.11 not found” turned out to be an LD_LIBRARY_PATH pointing to an older CUDA first. Unsetting it allowed the bundled library to load.

---

A tiny “AI SOS” script you can carry around

Drop this into ai-sos.sh to capture the essentials and run it before opening an issue.

#!/usr/bin/env bash
set -euo pipefail

echo "=== Time ==="; date
echo "=== OS/Kernel ==="; uname -a; (lsb_release -a 2>/dev/null || cat /etc/*release)
echo "=== CPU flags ==="; grep -m1 "flags" /proc/cpuinfo
echo "=== Mem/GPU ==="; free -h; (nvidia-smi || true)
echo "=== Python ==="; which python3; python3 -V; python3 -m pip --version
echo "=== PyTorch/TensorFlow ==="
python3 - <<'PY' || true
try:
    import torch
    print("torch:", torch.__version__, "cuda:", torch.version.cuda, "avail:", torch.cuda.is_available())
except Exception as e:
    print("torch import failed:", e)
try:
    import tensorflow as tf
    print("tf:", tf.__version__, "built_cuda:", tf.test.is_built_with_cuda(), "gpus:", tf.config.list_physical_devices('GPU'))
except Exception as e:
    print("tf import failed:", e)
PY
echo "=== Env vars (selected) ==="
env | grep -E 'CUDA|NCCL|LD_LIBRARY_PATH|OMP|MKL' || true

Make it executable:

chmod +x ai-sos.sh && ./ai-sos.sh

---

Conclusion and next steps (CTA)

Troubleshooting AI on Linux is mostly about disciplined environment management and a few sharp tools:

• Snapshot first, don’t guess.

• Isolate with venvs.

• Verify CUDA/driver/framework alignment.

• Triage memory and threads before touching code.

• Read what the dynamic linker tells you.

Your next step:

• Create a clean venv for your current project and pin dependencies.

• Save the ai-sos.sh script to your repos for reproducible diagnostics.

• Standardize your team on a single wheel index (CPU or a specific cuXX) to minimize drift.

If you found this useful, bookmark it, share with your team, and consider building a small internal “AI doctor” CLI that wraps these checks for your stack. Happy debugging!