Posted on
• Artificial Intelligence

Artificial Intelligence Virtualisation Checklist

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence Virtualization Checklist (for Linux Bash Users)

If your AI experiments live on bare metal today, you’re leaving portability, reproducibility, and scale on the table. Virtualization lets you snapshot a known‑good stack, pin performance where it matters, and ship a whole environment between machines—without breaking your GPU workflows. This guide gives you a practical, Bash‑first checklist to virtualize AI workloads on Linux with minimal fuss.

What you’ll get:

  • Why virtualization is worth it for AI

  • A 5‑part checklist with real‑world steps

  • Copy‑paste install commands (apt, dnf, zypper)

  • Sanity‑check scripts and example commands

Why virtualize AI?

  • Reproducibility: Freeze driver/toolchain versions inside a VM image. Share it with teammates or CI/CD with zero “works on my machine” surprises.

  • Isolation: Train or serve multiple models with strict resource boundaries. Ideal for multi‑tenant labs.

  • Portability: Move a tuned VM from workstation to server seamlessly.

  • Safety: Try new CUDA/ROCm stack versions in an isolated VM before touching your host.

1) Prepare the host (Hypervisor, Firmware, and Services)

First, validate CPU virtualization and IOMMU support, then install KVM/QEMU + libvirt.

Check CPU virtualization:

lscpu | grep -E 'Virtualization|Flags'
egrep -o 'vmx|svm' /proc/cpuinfo | sort -u

Check KVM modules:

lsmod | grep kvm

Install virtualization stack:

  • Ubuntu/Debian (apt):
sudo apt update
sudo apt install -y qemu-kvm libvirt-daemon-system libvirt-clients virt-manager virtinst bridge-utils ovmf cloud-init qemu-guest-agent pciutils cpu-checker virt-top iperf3
  • Fedora/RHEL/CentOS (dnf):
sudo dnf install -y qemu-kvm libvirt libvirt-client virt-install virt-manager bridge-utils edk2-ovmf cloud-init qemu-guest-agent pciutils virt-top iperf3
  • openSUSE (zypper):
sudo zypper refresh
sudo zypper install -y qemu-kvm libvirt libvirt-client virt-install virt-manager bridge-utils ovmf cloud-init qemu-guest-agent pciutils virt-top iperf3

Enable and start libvirt:

sudo systemctl enable --now libvirtd

Quick host validation:

sudo virt-host-validate

Optional: add your user to libvirt group so you can manage VMs without sudo:

sudo usermod -aG libvirt $(whoami)
newgrp libvirt

Tip: Ensure you have UEFI firmware for VMs installed (ovmf/edk2-ovmf) and available in virt-manager or virt-install.

2) Pick a GPU strategy: PCIe Passthrough vs vGPU/MIG

Most individual AI researchers opt for PCIe passthrough (VFIO). For multi‑tenant or slicing enterprise GPUs, consider NVIDIA MIG (A100/H100) or licensed vGPU.

Enable IOMMU in your host’s kernel parameters:

  • Intel:
sudo sed -i 's/GRUB_CMDLINE_LINUX="/GRUB_CMDLINE_LINUX="intel_iommu=on iommu=pt /' /etc/default/grub
  • AMD:
sudo sed -i 's/GRUB_CMDLINE_LINUX="/GRUB_CMDLINE_LINUX="amd_iommu=on iommu=pt /' /etc/default/grub

Update bootloader and reboot:

  • Debian/Ubuntu:
sudo update-grub
sudo reboot
  • Fedora/RHEL/openSUSE (grub2):
sudo grub2-mkconfig -o /boot/grub2/grub.cfg
sudo reboot

Find your GPU’s PCI address:

lspci -nn | egrep -i 'vga|3d|nvidia|amd'

Check IOMMU groups (ensure GPU and its audio function are isolated):

for d in /sys/kernel/iommu_groups/*/devices/*; do echo $d; done | sort

Bind GPU to VFIO (example BDFs 0000:65:00.0 and 0000:65:00.1):

GPU_VID_PID=$(lspci -n -s 65:00.0 | awk '{print $3}')
AUDIO_VID_PID=$(lspci -n -s 65:00.1 | awk '{print $3}')
echo "options vfio-pci ids=$GPU_VID_PID,$AUDIO_VID_PID" | sudo tee /etc/modprobe.d/vfio.conf
echo vfio | sudo tee /etc/modules-load.d/vfio.conf
echo vfio-pci | sudo tee -a /etc/modules-load.d/vfio.conf

Rebuild initramfs and reboot:

  • Debian/Ubuntu:
sudo update-initramfs -u
sudo reboot
  • Fedora/RHEL:
sudo dracut -f
sudo reboot
  • openSUSE:
sudo dracut -f
sudo reboot

After reboot, confirm VFIO binding:

lspci -k -s 65:00.0

Real‑world note:

  • Single‑GPU desktops used for passthrough need a second “host” GPU or iGPU for the host display to avoid black screens.

  • MIG (A100/H100/RTX 6000 Ada with supported drivers) can carve a GPU into multiple isolated instances—great for multi‑tenant inference.

3) VM configuration for AI: CPU flags, HugePages, Storage

Expose host CPU features (AVX/AVX2/AVX‑512) and tune memory + storage.

Use host‑passthrough CPU in your VM:

virt-install \
  --name ai-box \
  --memory 32768 \
  --vcpus 8 \
  --cpu host-passthrough,cache.mode=passthrough \
  --disk size=200,backing_store_format=raw,bus=virtio \
  --cdrom /var/lib/libvirt/images/ubuntu-22.04.iso \
  --network network=default,model=virtio \
  --os-variant ubuntu22.04 \
  --graphics none

Add your GPU to the VM (replace with your GPU BDF; add both GPU and audio functions):

virsh nodedev-list | grep pci_
virsh nodedev-dumpxml pci_0000_65_00_0

Then attach:

virt-xml ai-box --add-device --host-device 0000:65:00.0
virt-xml ai-box --add-device --host-device 0000:65:00.1

Enable HugePages on the host for large models:

echo "vm.nr_hugepages=4096" | sudo tee /etc/sysctl.d/99-hugepages.conf
sudo sysctl --system

Pin VM memory to HugePages (in XML: memoryBacking with hugepages) or via virt-xml:

virt-xml ai-box --edit --memorybacking clearxml=yes
virt-xml ai-box --memorybacking hugepages=on

Storage tips:

  • Prefer virtio‑blk/virtio‑scsi and raw/NVMe‑backed volumes for I/O‑heavy training data.

  • Put dataset cache (Hugging Face, pip wheels) on a fast disk.

  • Snapshot before driver/toolchain changes.

Install guest agent inside the VM (for clean shutdowns, IP reporting, file‑freeze):

  • Ubuntu/Debian (inside guest):
sudo apt update
sudo apt install -y qemu-guest-agent cloud-init
sudo systemctl enable --now qemu-guest-agent
  • Fedora/RHEL (inside guest):
sudo dnf install -y qemu-guest-agent cloud-init
sudo systemctl enable --now qemu-guest-agent
  • openSUSE (inside guest):
sudo zypper install -y qemu-guest-agent cloud-init
sudo systemctl enable --now qemu-guest-agent

4) Networking and data movement

Default NAT is fine to start. For higher throughput/latency‑sensitive training clusters, use a bridge or SR‑IOV.

Create a simple bridge on the host (example with iproute2; replace enp3s0):

sudo ip link add name br0 type bridge
sudo ip link set enp3s0 master br0
sudo ip addr flush dev enp3s0
sudo dhclient br0
sudo ip link set br0 up

Attach your VM NIC to br0 in virt-manager or:

virt-xml ai-box --edit --network clearxml=yes
virt-xml ai-box --add-device --network bridge=br0,model=virtio

Measure throughput:

iperf3 -s   # on one machine
iperf3 -c <server-ip> -P 4   # from the other

Data tips:

  • Keep datasets on a separate virtio disk and mount read‑only if possible.

  • For shared datasets across VMs, export via NFS/SMB or object storage and cache locally.

5) Automation, monitoring, and reproducibility

Automate provisioning with cloud‑init and keep images versioned.

Create a cloud‑init ISO with user + SSH key:

mkdir -p ~/cloudinit
cat > ~/cloudinit/user-data <<'EOF'
#cloud-config
users:
  - name: ai
    sudo: ALL=(ALL) NOPASSWD:ALL
    groups: users,adm
    shell: /bin/bash
    ssh_authorized_keys:
      - ssh-rsa AAAA...yourkey...
packages:
  - build-essential
  - python3-pip
  - git
runcmd:
  - pip3 install --upgrade pip
  - pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu121
EOF

touch ~/cloudinit/meta-data
genisoimage -output ~/cloudinit/seed.iso -volid cidata -joliet -rock ~/cloudinit/user-data ~/cloudinit/meta-data

Attach the seed ISO during virt-install:

--disk path=/home/$USER/cloudinit/seed.iso,device=cdrom

Monitor host and VMs:

virt-top
virsh domstats ai-box
nvidia-smi
nvidia-smi dmon -s pucvmet

Backups:

  • Use virsh snapshot-create-as before big driver changes.

  • Keep your cloud‑init files + virt-install commands in git for reproducibility.

Real‑world examples

  • Solo researcher with RTX 3090: One Ubuntu VM with PCIe passthrough GPU, HugePages, and host‑passthrough CPU runs nightly fine‑tunes. Snapshots protect CUDA upgrades.

  • Shared A100 server: MIG slices provide multiple isolated inference VMs. Each VM ships with pinned CUDA/cuDNN versions; ops can roll upgrades VM‑by‑VM.

  • CI for model builds: A headless VM template boots with cloud‑init, provisions Python + CUDA toolchain, runs tests, and exports artifacts. The host stays clean.

One‑shot install recap (host)

  • Ubuntu/Debian:
sudo apt update
sudo apt install -y qemu-kvm libvirt-daemon-system libvirt-clients virt-manager virtinst bridge-utils ovmf cloud-init qemu-guest-agent pciutils cpu-checker virt-top iperf3
sudo systemctl enable --now libvirtd
sudo virt-host-validate
  • Fedora/RHEL/CentOS:
sudo dnf install -y qemu-kvm libvirt libvirt-client virt-install virt-manager bridge-utils edk2-ovmf cloud-init qemu-guest-agent pciutils virt-top iperf3
sudo systemctl enable --now libvirtd
sudo virt-host-validate
  • openSUSE:
sudo zypper install -y qemu-kvm libvirt libvirt-client virt-install virt-manager bridge-utils ovmf cloud-init qemu-guest-agent pciutils virt-top iperf3
sudo systemctl enable --now libvirtd
sudo virt-host-validate

Quick self‑check script

#!/usr/bin/env bash
set -euo pipefail

echo "=== CPU virtualization ==="
lscpu | grep -E 'Virtualization|Flags' || true
egrep -o 'vmx|svm' /proc/cpuinfo | sort -u || true

echo "=== KVM modules ==="
lsmod | egrep 'kvm|vfio' || true

echo "=== Libvirt service ==="
systemctl is-active libvirtd || true

echo "=== UEFI firmware (OVMF) ==="
rpm -q edk2-ovmf 2>/dev/null || dpkg -l | grep -E 'ovmf|edk2-ovmf' || true

echo "=== IOMMU (kernel cmdline) ==="
grep -E 'intel_iommu|amd_iommu|iommu=pt' /proc/cmdline || true

echo "=== GPUs ==="
lspci | egrep -i 'vga|3d|nvidia|amd' || true

Save as ai-virt-check.sh, then:

chmod +x ai-virt-check.sh
./ai-virt-check.sh

Conclusion / Call to Action

Virtualizing your AI stack gives you reproducibility, safety, and portability—without sacrificing performance when tuned correctly. Start with the host prep, decide on your GPU strategy, and lock in CPU/memory/storage best practices. Then automate with cloud‑init and keep simple monitoring in place.

Your next steps: 1) Run the one‑shot install for your distro. 2) Reboot with IOMMU enabled and validate VFIO. 3) Create your first GPU‑backed VM with virt-install and cloud‑init. 4) Snapshot. Experiment. Iterate confidently.

Have a favorite tweak or a gotcha on your hardware? Share it—let’s make this checklist even better for the Linux and Bash AI community.

Related posts