Artificial Intelligence Virtualisation Best Practices (for Linux + Bash)

You finally scored a couple of GPUs… yet your training jobs still crawl, experiments aren’t reproducible, and moving workloads between machines is painful. Virtualisation can either supercharge your AI stack or quietly siphon performance if it’s set up incorrectly.

This post shows you how to do it right on Linux: why virtualising AI workloads is valuable, what to avoid, and the exact, copy-pasteable commands to install and tune a fast, reproducible virtualised AI environment. All examples are Bash-friendly and include apt, dnf, and zypper instructions.

---

Why virtualise AI workloads?

• Reproducibility: Snapshots, images, and declarative configs make experiments consistent across machines.

• Isolation: Keep conflicting CUDA/cuDNN/driver stacks or system packages from interacting.

• Portability: Move a tuned VM image to a workstation, a lab server, or a cloud host with minimal changes.

• Safety: Separate experiments and users; minimize “it worked on that node” failures.

Done right, the overhead is small (often single digits). Done poorly, you’ll burn 20–50% of your performance budget on IO, memory, and CPU scheduling inefficiencies.

---

Quick installs: the essentials

The fastest path to a good baseline is KVM/libvirt for VMs, plus containers (Docker or Podman) for per-project environments inside the VM or on bare metal.

Before you start, enable hardware virtualisation (Intel VT-x/AMD-V and VT-d/IOMMU) in your BIOS/UEFI.

Check on the host:

lscpu | grep -i virtualization
lsmod | grep kvm

1) KVM/QEMU, libvirt, virt-manager, OVMF (UEFI), cloud-init

• Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y qemu-kvm libvirt-daemon-system libvirt-clients virt-manager bridge-utils ovmf cloud-init cloud-image-utils
sudo systemctl enable --now libvirtd
sudo usermod -aG libvirt $USER
newgrp libvirt

• Fedora/RHEL/CentOS Stream (dnf):

sudo dnf -y install @virtualization virt-install virt-manager bridge-utils edk2-ovmf cloud-init
sudo systemctl enable --now libvirtd
sudo usermod -aG libvirt $USER
newgrp libvirt

• openSUSE/SLE (zypper):

sudo zypper refresh
sudo zypper install -y qemu-kvm libvirt virt-manager bridge-utils ovmf cloud-init
sudo systemctl enable --now libvirtd
sudo usermod -aG libvirt $USER
newgrp libvirt

Install the QEMU guest agent inside VMs for better integration:

• Inside Debian/Ubuntu guest:

sudo apt update && sudo apt install -y qemu-guest-agent
sudo systemctl enable --now qemu-guest-agent

• Inside Fedora/RHEL guest:

sudo dnf -y install qemu-guest-agent
sudo systemctl enable --now qemu-guest-agent

• Inside openSUSE/SLE guest:

sudo zypper install -y qemu-guest-agent
sudo systemctl enable --now qemu-guest-agent

2) Containers: Docker or Podman

• Docker Engine:

  • Debian/Ubuntu (apt):

  sudo apt update
  sudo apt install -y docker.io
  sudo systemctl enable --now docker
  sudo usermod -aG docker $USER
  newgrp docker
  

  • Fedora/RHEL/CentOS Stream (dnf):

  sudo dnf -y install moby-engine
  sudo systemctl enable --now docker
  sudo usermod -aG docker $USER
  newgrp docker
  

  • openSUSE/SLE (zypper):

  sudo zypper install -y docker
  sudo systemctl enable --now docker
  sudo usermod -aG docker $USER
  newgrp docker
  

• Podman:

  • Debian/Ubuntu (apt):

  sudo apt update && sudo apt install -y podman
  

  • Fedora/RHEL/CentOS Stream (dnf):

  sudo dnf -y install podman
  

  • openSUSE/SLE (zypper):

  sudo zypper install -y podman
  

3) NVIDIA Container Toolkit (for GPU containers)

Note: You still need a proper proprietary GPU driver on the host. Follow your distro/GPU vendor’s guide for that step.

• Debian/Ubuntu (apt):

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update
sudo apt install -y nvidia-container-toolkit
# Configure for Docker (or change --runtime=podman for Podman)
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

• Fedora/RHEL/CentOS Stream (dnf):

distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.repo | \
  sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
sudo dnf -y install nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

• openSUSE/SLE (zypper):

distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.repo | \
  sudo tee /etc/zypp/repos.d/nvidia-container-toolkit.repo
sudo zypper refresh
sudo zypper install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Test GPU container:

docker run --rm --gpus all nvidia/cuda:12.3.0-runtime-ubuntu22.04 nvidia-smi
# Podman (example; hooks may vary by distro/toolkit version)
podman run --rm --hooks-dir=/usr/share/containers/oci/hooks.d --device nvidia.com/gpu=all \
  nvidia/cuda:12.3.0-runtime-ubuntu22.04 nvidia-smi

---

5 Best Practices for Fast, Reproducible AI VMs

1) Use the right isolation: VM vs container vs both

• Containers (bare metal): Lowest overhead; ideal when host drivers/toolchains are stable. Combine with NVIDIA Container Toolkit.

• VM + containers: Best balance of isolation and portability. VM pins CPU/memory/IO, containers vary Python/CUDA/ML libs per project inside the VM.

• Full VM per team/experiment: Strongest boundaries; adds a few percent overhead when tuned.

Tip: If multiple users need different CUDA versions or kernels, prefer “VM + containers.”

2) Build a tuned KVM VM (UEFI, virtio, hugepages, NUMA/pinning)

Create a performance-oriented VM (example with cloud-init):

# Example assumes an Ubuntu cloud image is downloaded as jammy.img
virt-install \
  --name ai-vm \
  --memory 32768 --vcpus 8 \
  --cpu host-passthrough,topology.sockets=1,cores=8,threads=1 \
  --disk path=/var/lib/libvirt/images/ai-vm.qcow2,size=120,bus=virtio,cache=none,discard=unmap,format=qcow2 \
  --network network=default,model=virtio \
  --os-variant ubuntu22.04 \
  --graphics none \
  --features kvm_hidden=on \
  --boot uefi \
  --cloud-init user-data=cloud-init.yaml \
  --import --disk path=jammy.img,device=disk,bus=virtio

Enable hugepages on the host (reduces TLB misses and QEMU overhead):

# Example: reserve 4096 x 2MB hugepages (~8GB)
echo "vm.nr_hugepages=4096" | sudo tee /etc/sysctl.d/99-hugepages.conf
sudo sysctl --system

Then add to the VM XML (via virt-manager or virsh edit) to use hugepages:

<memoryBacking>
  <hugepages/>
</memoryBacking>

Pin vCPUs to physical cores and keep the emulator thread off hot cores:

# Pin VM vCPUs 0-7 to physical cores 0-7
virsh vcpupin ai-vm 0 0
virsh vcpupin ai-vm 1 1
virsh vcpupin ai-vm 2 2
virsh vcpupin ai-vm 3 3
virsh vcpupin ai-vm 4 4
virsh vcpupin ai-vm 5 5
virsh vcpupin ai-vm 6 6
virsh vcpupin ai-vm 7 7

# Keep QEMU emulator threads off those cores (example: move to cores 8-11)
virsh emulatorpin ai-vm 8-11

# NUMA affinity (strictly use node 0, as an example)
virsh numatune ai-vm --mode strict --nodeset 0

Use virtio devices, disable unnecessary emulation, and prefer cache=none + discard=unmap on disks for throughput and TRIM.

3) Treat GPUs as first-class citizens

Option A: Containers on bare metal

• Easiest way to drive GPUs at full tilt without passthrough complexity.

• Keep the host GPU driver correct; manage CUDA/toolchains at the container layer.

• Test:

docker run --rm --gpus all nvidia/cuda:12.3.0-runtime-ubuntu22.04 nvidia-smi

Option B: Full GPU passthrough to a VM (VFIO)

• Enable IOMMU on the host:

# Edit /etc/default/grub and add to GRUB_CMDLINE_LINUX:
# Intel: intel_iommu=on iommu=pt
# AMD:   amd_iommu=on iommu=pt
sudo sed -i 's/GRUB_CMDLINE_LINUX="/GRUB_CMDLINE_LINUX="intel_iommu=on iommu=pt /' /etc/default/grub || true
# Update GRUB (UEFI paths vary):
sudo update-grub 2>/dev/null || sudo grub2-mkconfig -o /boot/grub2/grub.cfg || sudo grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg
sudo reboot

• Identify GPU and audio function PCI IDs:

lspci -nn | grep -E "VGA|3D|Audio"

• Bind to vfio-pci:

# Example IDs; replace with your GPU: 10de:1db6 and audio: 10de:10f0
echo "options vfio-pci ids=10de:1db6,10de:10f0" | sudo tee /etc/modprobe.d/vfio.conf
echo -e "blacklist nouveau\nblacklist nvidiafb" | sudo tee /etc/modprobe.d/blacklist-nvidia.conf
sudo update-initramfs -u 2>/dev/null || sudo dracut -f
sudo reboot

• Attach GPU to the VM via libvirt XML (hostdev entries):

<hostdev mode='subsystem' type='pci' managed='yes'>
  <source><address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/></source>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
  <source><address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/></source>
</hostdev>

Install the GPU driver inside the guest VM and validate with nvidia-smi.

Notes:

• Single-GPU passthrough works well; sharing requires vendor tech (NVIDIA vGPU, AMD SR-IOV) and licensing/hardware support.

• Host and guest Secure Boot/UEFI settings may affect driver loading; OVMF helps.

4) Storage, IO, and datasets

• Put VMs on fast NVMe SSDs.

• Use qcow2 with metadata preallocation for speed and snapshots:

qemu-img create -f qcow2 -o preallocation=metadata /var/lib/libvirt/images/ai-vm.qcow2 200G

• Prefer virtio-scsi or virtio-blk, cache=none, discard=unmap, separate IO threads for heavy disk workloads.

• Datasets:

  • Read-mostly: virtiofs (fast, simple sharing from host to VM).
  • Multi-VM or multi-host: NFS or object storage; cache within the VM/container.

• Inside Linux guests, XFS or ext4 are safe defaults. Mount with noatime to trim metadata writes for large data scans.

Example virt-install disk flags already shown (cache=none,discard=unmap). For virtiofs, add a filesystem device in libvirt or mount via fstab when using newer libvirt/QEMU.

5) Automate the “known good” state

• Cloud-init to bootstrap users, SSH keys, packages, and conda environments:

# cloud-init.yaml (minimal example)
#cloud-config
users:
  - name: ai
    groups: sudo
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    shell: /bin/bash
    ssh_authorized_keys:
      - ssh-ed25519 AAAA...yourkey
package_update: true
packages:
  - git
  - build-essential
runcmd:
  - curl -fsSL https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -o /tmp/conda.sh
  - bash /tmp/conda.sh -b -p /opt/conda
  - /opt/conda/bin/conda init bash
  - echo 'export PATH=/opt/conda/bin:$PATH' >> /home/ai/.bashrc

• Snapshots before major changes:

virsh snapshot-create-as --domain ai-vm base-setup --atomic --disk-only --quiesce

• Image hygiene:

virt-sysprep -a /var/lib/libvirt/images/ai-vm.qcow2
qemu-img compact /var/lib/libvirt/images/ai-vm.qcow2

• Keep infra-as-code: store your cloud-init.yaml, virt-install scripts, and libvirt XML diffs in git.

---

Real‑world pattern: “One host, two tuned VMs, containers inside”

• Host: 16C/32T CPU, 128 GB RAM, 2× GPUs, 2× NVMe.

• VM1 (GPU0): 8 vCPU pinned to NUMA node 0, 48 GB RAM w/ hugepages, virtio-scsi on NVMe0, virtiofs mount for datasets.

• VM2 (GPU1): mirror of VM1 on NUMA node 1/NVMe1.

• Inside each VM: Docker or Podman with NVIDIA Container Toolkit. Team members run containers with project-specific CUDA/PyTorch/TensorFlow stacks.

• Benefits: Strong isolation per team, easy migrations, near bare-metal training speed, clean rollbacks.

---

Conclusion and next steps

Virtualisation is not the enemy of performance—misconfiguration is. With KVM/libvirt, virtio devices, hugepages, CPU/NUMA pinning, and a clear GPU strategy (containers or passthrough), you can get predictable, fast, portable AI environments.

Your next steps: 1) Install the essentials (KVM/libvirt + Docker/Podman + NVIDIA toolkit if needed). 2) Launch a tuned VM using the example virt-install and cloud-init files. 3) Choose your GPU approach: bare-metal containers or full passthrough. 4) Automate snapshots and keep your build scripts in git.

Have a favorite tuning trick or a gotcha others should know? Write it down, script it, and make it part of your baseline. Your future experiments—and teammates—will thank you.