Artificial Intelligence Docker Troubleshooting: A Linux Bash Playbook

You just pulled the latest AI container, hit enter, and… boom: “CUDA driver version is insufficient,” “illegal instruction (core dumped),” or your container silently exits. AI inside Docker is fantastic for reproducibility—but GPU passthrough, drivers, IPC, memory limits, and SELinux can turn a simple run into a head-scratcher.

This guide shows you how to quickly diagnose and fix the most common AI-in-Docker problems on Linux. You’ll get actionable steps, ready-to-run Bash, and package-manager-specific install instructions (apt, dnf, zypper) where relevant.

Why this matters

AI stacks are hardware-aware: drivers, GPU runtimes, and CPU instruction sets must align across host and container.
Containers add isolation: default network, filesystems, cgroups, and SELinux/AppArmor can block what AI workloads need (large shared memory, GPU devices, big model files).
A systematic approach prevents wild goose chases: a few checks identify 80% of problems in minutes.

Quick-start prerequisites

Make sure you have Docker Engine installed and your user is in the docker group.

Install Docker Engine:

apt (Ubuntu/Debian):

sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") $(. /etc/os-release; echo "$VERSION_CODENAME") stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker $USER
newgrp docker

dnf (Fedora/RHEL/CentOS):

sudo dnf -y install dnf-plugins-core
# Fedora:
sudo dnf config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo
# CentOS/RHEL (use centos repo):
# sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo systemctl enable --now docker
sudo usermod -aG docker $USER
newgrp docker

zypper (openSUSE/SLE):

sudo zypper refresh
# Use distribution packages (Docker CE repo doesn’t officially support openSUSE/SLE)
sudo zypper install -y docker docker-compose
sudo systemctl enable --now docker
sudo usermod -aG docker $USER
newgrp docker

Install pciutils for lspci (useful for GPU checks):

apt:

sudo apt-get update && sudo apt-get install -y pciutils

dnf:

sudo dnf install -y pciutils

zypper:

sudo zypper install -y pciutils

Five common AI-in-Docker failure modes (and fast fixes)

1) GPU not visible inside the container

Symptoms:

nvidia-smi: command not found or no GPUs listed in the container.
CUDA error: “driver version is insufficient” or “libcuda.so not found.”

Host checks:

lspci | grep -i nvidia
nvidia-smi

If nvidia-smi fails on the host, install or fix the NVIDIA driver first (outside Docker).

Install NVIDIA Container Toolkit (enables --gpus):

apt (Ubuntu/Debian):

distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

dnf (Fedora/RHEL/CentOS):

distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.repo | \
sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
sudo dnf install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

zypper (openSUSE/SLE):

distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.repo | \
sudo tee /etc/zypp/repos.d/nvidia-container-toolkit.repo
sudo zypper refresh
sudo zypper install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Verify GPU passthrough:

docker run --rm --gpus all nvidia/cuda:12.4.1-runtime-ubuntu22.04 nvidia-smi

Tip (Compose):

# docker-compose.yml
services:
  app:
    image: nvidia/cuda:12.4.1-runtime-ubuntu22.04
    gpus: all
    command: nvidia-smi

Version mismatch note:

Driver (host) must be >= minimum version required by the CUDA runtime (container). You can check CUDA-to-driver compatibility on NVIDIA’s matrix. If you see “CUDA driver version is insufficient,” update the host driver (not just the container).

2) OOM kills, CUDA out-of-memory, or training crashes

Symptoms:

Container exits abruptly; dmesg shows OOM kill.
PyTorch/TensorFlow reports CUDA OOM despite free VRAM.
DataLoader crashes or performance is poor due to tiny /dev/shm.

Fixes:

Give the container a bigger shared memory segment and unlock memlock:

docker run --rm --gpus all \
--shm-size=1g --ulimit memlock=-1:-1 --ipc=host \
pytorch/pytorch:2.3.1-cuda12.1-cudnn8-runtime \
python -c "import torch; print(torch.cuda.is_available())"

If your container has memory limits, consider removing them or increasing them:
```
docker run --memory=0 --memory-swap=0 ...  # no memory cgroup limits
```

Helpful PyTorch envs:

-e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
-e TORCH_SHOW_CPP_STACKTRACES=1

Check OOM evidence:

dmesg -T | tail -n 100 | grep -i -E 'kill|oom|out of memory' || true

Compose equivalents:

services:
  trainer:
    image: pytorch/pytorch:2.3.1-cuda12.1-cudnn8-runtime
    gpus: all
    shm_size: "1g"
    ulimits:
      memlock: -1
    ipc: host

3) “illegal instruction (core dumped)” on CPU

Symptoms:

Instant crash with “illegal instruction,” often when the image was built with AVX/AVX2/FMA while your CPU lacks them.

Check CPU flags:

grep -m1 -o -E 'avx512|avx2|avx|sse4_2|sse4_1' /proc/cpuinfo | tr ' ' '\n' | sort -u

Fixes:

Use containers built for older CPUs (e.g., many wheels distribute “+cpu” builds or “no-avx” variants).
Build from source inside your Dockerfile with conservative flags (e.g., for PyTorch, ONNX Runtime, or NumPy with baseline CPU).
Note: --platform=linux/amd64 won’t fix missing CPU instructions—it only changes architecture, not CPU feature baseline.

4) Volume permissions and SELinux denials

Symptoms:

“Permission denied” when writing model weights/checkpoints to a bind mount.
On Fedora/CentOS/RHEL with SELinux enforcing: “operation not permitted” even with correct Unix perms.

Fixes:

Match user IDs:

mkdir -p $PWD/models
chown $(id -u):$(id -g) $PWD/models
docker run --rm -it \
--user $(id -u):$(id -g) \
-v "$PWD/models:/models" \
pytorch/pytorch:2.3.1-cuda12.1-cudnn8-runtime bash

On SELinux systems, label the volume:

docker run --rm -it \
-v "$PWD/models:/models:Z" \
pytorch/pytorch:2.3.1-cuda12.1-cudnn8-runtime bash

Or in Compose:

services:
app:
  volumes:
    - ./models:/models:Z

For persistent labeling without :Z, you can change context:
```
sudo chcon -Rt svirt_sandbox_file_t $PWD/models
```

5) Network, proxies, and DNS weirdness

Symptoms:

pip install or model downloads time out in the container, but work on the host.
Behind corporate proxies, outbound connections fail.

Quick tests:

docker run --rm alpine sh -c "apk add --no-cache curl >/dev/null && curl -I https://pypi.org"

One-off proxy in docker run:

docker run -e HTTP_PROXY=http://proxy:3128 \
           -e HTTPS_PROXY=http://proxy:3128 \
           -e NO_PROXY=localhost,127.0.0.1,::1,*.cluster.local \
           ...

Configure Docker daemon-wide proxy (systemd):

sudo mkdir -p /etc/systemd/system/docker.service.d
cat <<'EOF' | sudo tee /etc/systemd/system/docker.service.d/proxy.conf
[Service]
Environment="HTTP_PROXY=http://proxy:3128"
Environment="HTTPS_PROXY=http://proxy:3128"
Environment="NO_PROXY=localhost,127.0.0.1,::1,*.local,registry.local"
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker

DNS override if name resolution fails:

docker run --dns=8.8.8.8 --dns=1.1.1.1 ...

Speed up pip/conda in containers:

Use wheels/conda caches via volumes: -v $HOME/.cache/pip:/root/.cache/pip -v $HOME/.conda:/root/.conda

Real-world troubleshooting flow (copy-paste friendly)

1) Verify Docker and runtime:

docker version
docker info --format '{{json .Runtimes}}'

2) Confirm host GPU and driver:

lspci | grep -i -E 'nvidia|amd|gpu'
nvidia-smi

3) Smoke test GPU container:

docker run --rm --gpus all nvidia/cuda:12.4.1-runtime-ubuntu22.04 nvidia-smi

4) Test your framework quickly:

docker run --rm --gpus all \
  pytorch/pytorch:2.3.1-cuda12.1-cudnn8-runtime \
  python -c "import torch;print('CUDA ok?', torch.cuda.is_available());print('GPU count:', torch.cuda.device_count())"

5) If the container exits:

# Check logs
docker ps -a
docker logs <container>

# Inspect runtime and mounts
docker inspect <container> | jq '.[0] | {HostConfig: .HostConfig, Mounts: .Mounts, State: .State}'

# Kernel messages for OOM or denials
dmesg -T | tail -n 200

Bonus: AMD ROCm containers (CPU/GPU)

If you’re on AMD GPUs with ROCm:

Ensure ROCm drivers are installed on the host.

Expose devices explicitly:

docker run --rm \
--device=/dev/kfd --device=/dev/dri \
--group-add video \
rocm/pytorch:latest \
python -c "import torch; print(torch.cuda.is_available(), torch.version.hip)"

SELinux users may also need :Z on volumes.

Package notes vary by distro—consult AMD ROCm docs for your OS version.

Common gotchas to remember

Host driver wins: updating the container won’t fix an outdated host GPU driver.
Shared memory matters: DataLoaders and certain frameworks need a large /dev/shm.
Compose vs Swarm: deploy.resources GPU reservations are for Swarm; in Compose use gpus: all.
Permissions are two-layered: Unix perms and SELinux/AppArmor both can block writes.
CPU features are not emulated: pick images built for your CPU baseline.

Conclusion and next steps

AI in Docker is reliable once the foundations are aligned: host driver, GPU runtime, memory/IPC, and volumes. Bookmark this playbook, wire the verification commands into your CI, and codify your container run/compose options so teammates don’t rediscover the same pitfalls.

Call to Action:

Run the GPU smoke test now and fix any red flags:

docker run --rm --gpus all nvidia/cuda:12.4.1-runtime-ubuntu22.04 nvidia-smi

Add --shm-size, --ulimit memlock, and gpus: all to your project’s Compose.
If you’re still stuck, paste your error and the outputs of docker info, nvidia-smi, and dmesg -T | tail -n 200 into your issue—most fixes fall out from these three.

Artificial Intelligence Docker Troubleshooting: A Linux Bash Playbook

Why this matters

Quick-start prerequisites

Five common AI-in-Docker failure modes (and fast fixes)

1) GPU not visible inside the container

2) OOM kills, CUDA out-of-memory, or training crashes

3) “illegal instruction (core dumped)” on CPU

4) Volume permissions and SELinux denials

5) Network, proxies, and DNS weirdness

Real-world troubleshooting flow (copy-paste friendly)

Bonus: AMD ROCm containers (CPU/GPU)

Common gotchas to remember

Conclusion and next steps

Related posts