Posted on
• Artificial Intelligence

Artificial Intelligence Infrastructure Labs

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence Infrastructure Labs: A Bash‑first Guide for Linux

You don’t need a hyperscale data center to build serious AI. With a couple of commodity Linux machines, some GPUs, and a handful of Bash commands, you can stand up a reliable “AI Infrastructure Lab” to prototype models, validate pipelines, and train at small to mid-scale—without breaking the bank.

This guide explains why building your own AI lab is valuable, then walks you through a practical, Bash-friendly setup. You’ll get distro-agnostic, command-ready steps and per-distro installs (apt, dnf, zypper) wherever tools are cited.

Why an AI Infrastructure Lab is worth it

  • Reproducibility beats “it works on my laptop”: Containers, versioned datasets, and scripted provisioning lock in your environment.

  • Cost control: Try, measure, and iterate locally before renting expensive cloud GPU hours.

  • Skills and ownership: Learn the full stack—from drivers to orchestration—so you can tune for performance and reliability.

  • Portability: A lab that’s built on Linux + containers moves easily to cloud or HPC later.

What you’ll build

  • A minimal, GPU-capable Linux stack with containers (Docker) and NVIDIA Container Toolkit

  • Optional multi-node orchestration with k3s

  • Shared storage via NFS

  • Basic monitoring and benchmarking

  • A repeatable, Bash-driven workflow that scales from one box to a small cluster

0) System prerequisites and base tools

Install common CLI tools and Python build essentials.

  • Debian/Ubuntu (apt):
sudo apt update
sudo apt install -y git curl wget tmux htop build-essential \
  python3 python3-venv python3-pip
  • Fedora/RHEL/CentOS Stream (dnf):
sudo dnf -y update
sudo dnf -y install git curl wget tmux htop gcc make \
  python3 python3-pip python3-virtualenv
  • openSUSE Leap/Tumbleweed (zypper):
sudo zypper refresh
sudo zypper install -y git curl wget tmux htop gcc make \
  python3 python3-pip python3-virtualenv

Optional sanity checks:

git --version
python3 --version
pip3 --version

1) Get containers running (CPU-only baseline and GPU-ready)

You can use Podman or Docker. For GPU workloads, Docker has the most straightforward path with NVIDIA Container Toolkit, so we’ll focus on Docker for GPU, and mention Podman as an alternative for CPU-only.

Install Docker

  • Debian/Ubuntu (apt):
sudo apt update
sudo apt install -y docker.io docker-compose-plugin
sudo systemctl enable --now docker
sudo usermod -aG docker $USER
# Re-login or run:
newgrp docker
  • Fedora/RHEL/CentOS Stream (dnf):
sudo dnf -y install moby-engine docker-compose-plugin
sudo systemctl enable --now docker
sudo usermod -aG docker $USER
newgrp docker
  • openSUSE Leap/Tumbleweed (zypper):
sudo zypper install -y docker docker-compose
sudo systemctl enable --now docker
sudo usermod -aG docker $USER
newgrp docker

Test:

docker run --rm hello-world

Optional: Install Podman (CPU workloads)

  • Debian/Ubuntu (apt):
sudo apt update
sudo apt install -y podman
  • Fedora/RHEL/CentOS Stream (dnf):
sudo dnf -y install podman
  • openSUSE Leap/Tumbleweed (zypper):
sudo zypper install -y podman

2) Make containers see your GPUs (NVIDIA Container Toolkit)

Ensure your NVIDIA driver is installed using your distro/vendor’s supported method. Verify:

nvidia-smi

If you see GPU info, proceed to the container toolkit.

Install NVIDIA Container Toolkit:

  • Debian/Ubuntu (apt):
distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -fsSL https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update
sudo apt install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
  • Fedora/RHEL/CentOS Stream (dnf):
distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.repo | \
  sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
sudo dnf -y install nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
  • openSUSE Leap/Tumbleweed (zypper):
distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.repo | \
  sudo tee /etc/zypp/repos.d/nvidia-container-toolkit.repo
sudo zypper refresh
sudo zypper install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

GPU test in a container:

docker run --rm --gpus all nvidia/cuda:12.3.2-base-ubuntu22.04 nvidia-smi

3) Build a reusable AI container image

Use CUDA base images and install your frameworks. Example: minimal PyTorch image that uses the GPU if available.

Create Dockerfile:

# Dockerfile.ai
FROM nvidia/cuda:12.3.2-cudnn-runtime-ubuntu22.04

ENV DEBIAN_FRONTEND=noninteractive \
    PIP_NO_CACHE_DIR=1 \
    PYTHONDONTWRITEBYTECODE=1

RUN apt-get update && apt-get install -y --no-install-recommends \
      python3 python3-pip python3-venv git ca-certificates && \
    rm -rf /var/lib/apt/lists/*

# Install PyTorch (adjust CUDA version/index URL as needed)
RUN pip3 install --upgrade pip && \
    pip3 install torch --index-url https://download.pytorch.org/whl/cu121

# Simple sanity script
RUN printf '%s\n' \
'python3 - <<EOF' \
'import torch' \
'print(\"Torch:\", torch.__version__)' \
'print(\"CUDA available:\", torch.cuda.is_available())' \
'print(\"Device count:\", torch.cuda.device_count())' \
'EOF' > /opt/sanity.sh && chmod +x /opt/sanity.sh

WORKDIR /workspace

Build and run:

docker build -t ai-lab:torch -f Dockerfile.ai .
docker run --rm --gpus all ai-lab:torch /opt/sanity.sh

Expected output shows Torch version, whether CUDA is available, and GPU count.

4) Share data across nodes with NFS

Centralize datasets/checkpoints so every machine sees the same path.

  • On the storage node (server):

    • Debian/Ubuntu (apt):
    sudo apt update
sudo apt install -y nfs-kernel-server nfs-common
    • Fedora/RHEL/CentOS Stream (dnf):
    sudo dnf -y install nfs-utils
    • openSUSE Leap/Tumbleweed (zypper):
    sudo zypper install -y nfs-kernel-server nfs-client

    Export a directory:

    sudo mkdir -p /srv/ai-data
sudo chown -R $USER:$USER /srv/ai-data
echo "/srv/ai-data 192.168.1.0/24(rw,sync,no_subtree_check)" | sudo tee -a /etc/exports
sudo exportfs -ra
sudo systemctl enable --now nfs-server || sudo systemctl enable --now nfs-kernel-server

  • On each worker/client:

    • Debian/Ubuntu (apt):
    sudo apt update
sudo apt install -y nfs-common
    • Fedora/RHEL/CentOS Stream (dnf):
    sudo dnf -y install nfs-utils
    • openSUSE Leap/Tumbleweed (zypper):
    sudo zypper install -y nfs-client

    Mount (replace 192.168.1.10 with your server IP):

    sudo mkdir -p /mnt/ai-data
sudo mount -t nfs 192.168.1.10:/srv/ai-data /mnt/ai-data
# Persist across reboots:
echo "192.168.1.10:/srv/ai-data /mnt/ai-data nfs defaults,_netdev 0 0" | sudo tee -a /etc/fstab

5) Orchestrate: single node today, cluster tomorrow (k3s)

For multi-node scheduling and reproducible deployments, k3s is a lightweight Kubernetes distribution that works well in labs. It installs with one command and includes containerd. You can still use Docker for your local development containers.

  • Control plane (master):
curl -sfL https://get.k3s.io | sh -
sudo kubectl get nodes
  • Get the join token on the control plane:
sudo cat /var/lib/rancher/k3s/server/node-token
  • Worker node(s), replace and :
curl -sfL https://get.k3s.io | K3S_URL=https://<SERVER_IP>:6443 K3S_TOKEN=<TOKEN> sh -

GPU workloads on Kubernetes require the NVIDIA device plugin. For labs, you can run experiments first with Docker-only to validate containers, then add the device plugin to k3s when ready.

6) Monitor and benchmark quickly

  • Install nvtop (GPU/VRAM live view):

    • Debian/Ubuntu (apt):
    sudo apt update
sudo apt install -y nvtop
    • Fedora/RHEL/CentOS Stream (dnf):
    sudo dnf -y install nvtop
    • openSUSE Leap/Tumbleweed (zypper):
    sudo zypper install -y nvtop

  • Quick GPU/container smoke test:

docker run --rm --gpus all nvidia/cuda:12.3.2-base-ubuntu22.04 bash -lc "nvidia-smi && nvcc --version || true"
  • Micro-benchmark with PyTorch:
cat > gemm_bench.py << 'EOF'
import torch, time
device = "cuda" if torch.cuda.is_available() else "cpu"
print("Device:", device)
A = torch.randn(8192, 8192, device=device)
B = torch.randn(8192, 8192, device=device)
torch.cuda.synchronize() if device=="cuda" else None
t0 = time.time()
C = A @ B
torch.cuda.synchronize() if device=="cuda" else None
print("GEMM seconds:", time.time() - t0)
print("Checksum:", float(C.mean()))
EOF

docker run --rm --gpus all -v "$PWD":/workspace -w /workspace ai-lab:torch \
  python3 gemm_bench.py

Real-world example: a two-node GPU lab in an afternoon

  • Node A (control + storage): 1× A6000, 64 GB RAM, 2 TB NVMe

  • Node B (worker): 1× RTX 4090, 64 GB RAM

  • Network: 2.5 GbE, static IPs

  • Steps we took: 1) Install base tools, Docker, NVIDIA Container Toolkit (sections 0–2) 2) Create NFS share on Node A, mount on Node B (section 4) 3) Build ai-lab:torch image and verify GPU inside container (section 3) 4) Bring up k3s on Node A, join Node B (section 5) 5) Run training jobs pointed at /mnt/ai-data, monitor with nvtop

Result: reproducible training environment, easy data sharing, and the ability to scale to more workers with minimal extra steps.

Frequently used Bash snippets

  • Create a Python venv for quick, local CPU experiments:
python3 -m venv .venv && . .venv/bin/activate
pip install --upgrade pip
pip install numpy scipy jupyter
  • Save and load Docker images to move between offline nodes:
docker save ai-lab:torch | gzip > ai-lab.tgz
scp ai-lab.tgz user@node-b:/tmp/
ssh user@node-b 'gunzip -c /tmp/ai-lab.tgz | docker load'
  • Tag and push to a private registry (adjust REGISTRY):
docker tag ai-lab:torch registry.local:5000/ai-lab:torch
docker push registry.local:5000/ai-lab:torch

Conclusion and next steps (CTA)

You now have the blueprint for a practical, Bash-driven AI Infrastructure Lab:

  • Containers for reproducibility

  • NVIDIA Container Toolkit for GPU acceleration

  • Optional k3s for lightweight orchestration

  • NFS for shared datasets

  • Simple monitoring and benchmarks

Next steps: 1) Script your setup end-to-end so a new node can join in minutes. 2) Add a model registry and experiment tracker (e.g., MLflow) to your stack. 3) Introduce CI to auto-build and scan your AI images. 4) Measure, then optimize: mixed precision, data pipeline, and IO.

Spin up your first node this weekend. Keep it small, keep it scripted, and grow as your workloads demand. If you want a follow-up post on k3s + NVIDIA device plugin and job scheduling patterns, say the word.

Related posts