Posted on
• Artificial Intelligence

Artificial Intelligence VM Case Studies

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence VM Case Studies: Practical Patterns for Linux Bash Users

Ever had a working AI stack break after a system update? Or wished you could sandbox a risky experiment without risking your main OS? With a Linux box, KVM, and a few Bash one-liners, you can spin up reliable, reproducible AI environments in minutes—complete with snapshots, isolation, and optional GPU passthrough. This post walks through real VM-based AI setups you can copy and adapt today.

What you’ll get:

  • Why VMs still matter for AI in 2026

  • A quick, distro-agnostic setup checklist

  • Three hands-on case studies (CPU farm, GPU passthrough dev box, and a reproducible research sandbox)

  • A tuning checklist for better performance

Why AI in VMs is worth it

  • Reproducibility: Pin OS, drivers, libraries. Snapshot. Roll back. Share the exact image with a teammate.

  • Isolation and security: Keep untrusted notebooks, data, or model evals cordoned off. Perfect for compliance or air-gapped work.

  • Controlled upgrades: Test CUDA, ROCm, kernels, and toolchains in a VM before touching the host.

  • Portability: Move your VM from workstation to cloud to lab server unchanged.

  • Resource pooling: Use CPU farms for batch inference; use passthrough for one powerful GPU VM when needed.

Prereqs: Verify virtualization and install the stack

1) Check hardware virtualization:

lscpu | grep -i virtualization
egrep -q 'vmx|svm' /proc/cpuinfo && echo "HW virt: OK" || echo "HW virt: MISSING"
lsmod | grep kvm

2) Install KVM/QEMU, libvirt, virt-install, virt-manager, UEFI firmware (OVMF), and cloud-init tools.

Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils virt-manager virtinst ovmf cloud-image-utils genisoimage qemu-guest-agent

Fedora/RHEL (dnf):

sudo dnf install -y @virtualization virt-install virt-manager edk2-ovmf cloud-utils genisoimage qemu-guest-agent
sudo systemctl enable --now libvirtd

openSUSE/SLES (zypper):

sudo zypper install -y qemu-kvm libvirt-daemon libvirt-client bridge-utils virt-manager virt-install ovmf cloud-init cloud-guest-utils genisoimage qemu-guest-agent

3) Enable libvirt, add your user, and activate the default network:

sudo systemctl enable --now libvirtd
sudo usermod -aG libvirt,kvm $USER
newgrp libvirt
virsh net-start default || true
virsh net-autostart default

4) Grab a base cloud image:

mkdir -p ~/vm-images && cd ~/vm-images
wget -c https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img

Note: Virt tools are distro-agnostic; your VMs can run any supported guest OS (Ubuntu, Rocky, openSUSE, etc.).

Case Study 1: CPU-only inference farm you can scale in minutes

Goal: Build several small VMs to run batched CPU inference services with FastAPI + ONNX Runtime. Easy to clone, snapshot, and autoscale by count.

1) Create cloud-init files:

cd ~/vm-images
cat > user-data.yaml <<'YAML'
#cloud-config
users:
  - name: ai
    groups: [sudo]
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    shell: /bin/bash
    ssh_authorized_keys:
      - ssh-ed25519 AAAA...your_ssh_key_here
package_update: true
packages:
  - python3
  - python3-venv
  - python3-pip
runcmd:
  - python3 -m pip install --upgrade pip onnxruntime uvicorn fastapi
  - |
    cat >/home/ai/app.py <<'PY'
    from fastapi import FastAPI
    import onnxruntime as ort
    app = FastAPI()
    @app.get("/health")
    def health():
        return {"ok": True, "providers": ort.get_available_providers()}
    PY
  - chown ai:ai /home/ai/app.py
  - |
    cat >/etc/systemd/system/inference.service <<'UNIT'
    [Unit]
    Description=Minimal inference API
    After=network-online.target
    [Service]
    User=ai
    WorkingDirectory=/home/ai
    ExecStart=/usr/bin/python3 -m uvicorn app:app --host 0.0.0.0 --port 8000
    Restart=always
    [Install]
    WantedBy=multi-user.target
    UNIT
  - systemctl daemon-reload
  - systemctl enable --now inference
YAML

cat > meta-data.yaml <<'YAML'
instance-id: ai-cpu-1
local-hostname: ai-cpu-1
YAML

Build a NoCloud seed ISO (use whichever tool is available):

Option A (cloud-localds):

cloud-localds ai-seed.iso user-data.yaml meta-data.yaml

Option B (genisoimage):

genisoimage -output ai-seed.iso -volid cidata -joliet -rock user-data.yaml meta-data.yaml

2) Create the first VM:

virt-install \
  --name ai-cpu-1 \
  --memory 4096 --vcpus 4 \
  --cpu host-passthrough \
  --disk path=$HOME/vm-images/ai-cpu-1.qcow2,backing_store=$HOME/vm-images/jammy-server-cloudimg-amd64.img,format=qcow2,size=20 \
  --disk path=$HOME/vm-images/ai-seed.iso,device=cdrom \
  --network network=default,model=virtio \
  --graphics none \
  --import --os-variant ubuntu22.04

3) Discover the IP and test:

virsh domifaddr ai-cpu-1
curl http://<VM-IP>:8000/health

4) Scale to N instances (update instance-id per VM for clean cloud-init runs):

for i in 2 3; do
  sed "s/ai-cpu-1/ai-cpu-$i/g" meta-data.yaml > meta-$i.yaml
  cloud-localds ai-seed-$i.iso user-data.yaml meta-$i.yaml
  virt-install \
    --name ai-cpu-$i \
    --memory 4096 --vcpus 4 --cpu host-passthrough \
    --disk path=$HOME/vm-images/ai-cpu-$i.qcow2,backing_store=$HOME/vm-images/jammy-server-cloudimg-amd64.img,format=qcow2,size=20 \
    --disk path=$HOME/vm-images/ai-seed-$i.iso,device=cdrom \
    --network network=default,model=virtio \
    --graphics none --import --os-variant ubuntu22.04
done

Optional (manual installs inside a guest, if not using cloud-init):

  • apt:
sudo apt update && sudo apt install -y python3 python3-venv python3-pip
  • dnf:
sudo dnf install -y python3 python3-pip
  • zypper:
sudo zypper install -y python3 python3-pip

Case Study 2: Single-GPU passthrough dev VM for training and prototyping

Goal: Give a VM full access to a discrete GPU for CUDA/ROCm-based work without polluting your host.

Warning: GPU passthrough can make a system unbootable if misconfigured. Back up, keep a rescue USB handy, and document each change.

1) Ensure IOMMU is enabled:

  • Edit /etc/default/grub and add one of:
    • Intel: intel_iommu=on iommu=pt
    • AMD: amd_iommu=on iommu=pt

Regenerate bootloader:

  • Debian/Ubuntu (apt):
sudo update-grub
sudo reboot
  • Fedora/RHEL (dnf):
sudo grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg  # UEFI
# or, for BIOS:
sudo grub2-mkconfig -o /boot/grub2/grub.cfg
sudo reboot
  • openSUSE/SLES (zypper):
sudo grub2-mkconfig -o /boot/grub2/grub.cfg
sudo reboot

Verify after reboot:

dmesg | egrep -i 'iommu|dmar'

2) Bind the GPU to vfio-pci on the host:

GPU=$(lspci -nn | awk '/VGA.*(NVIDIA|AMD)/{print $1}')
AUDIO=$(lspci -nn | awk '/Audio.*(NVIDIA|AMD)/{print $1}')
echo "GPU=$GPU AUDIO=$AUDIO"
lspci -nn -s $GPU
lspci -nn -s $AUDIO

echo "options vfio-pci ids=$(lspci -nn -s $GPU | sed -n 's/.*\[\(....:....\)\].*/\1/p'),$(lspci -nn -s $AUDIO | sed -n 's/.*\[\(....:....\)\].*/\1/p') disable_vga=1" | sudo tee /etc/modprobe.d/vfio.conf

NVIDIA hosts often need nouveau blacklisted:

echo -e "blacklist nouveau\noptions nouveau modeset=0" | sudo tee /etc/modprobe.d/blacklist-nouveau.conf

Rebuild initramfs and reboot:

  • Debian/Ubuntu (apt):
sudo update-initramfs -u && sudo reboot
  • Fedora/RHEL/openSUSE (dnf/zypper):
sudo dracut -f && sudo reboot

Confirm vfio-pci is bound:

lspci -k -s $GPU
lspci -k -s $AUDIO
# Look for "Kernel driver in use: vfio-pci"

3) Create a GPU VM with UEFI and hostdev mappings:

virt-install \
  --name ai-gpu-dev \
  --memory 16384 --vcpus 8 \
  --cpu host-passthrough,cache.mode=passthrough \
  --hugepages \
  --disk path=$HOME/vm-images/ai-gpu-dev.qcow2,size=100,format=qcow2,cache=none,discard=unmap \
  --disk path=$HOME/vm-images/ai-seed.iso,device=cdrom \
  --network network=default,model=virtio \
  --graphics spice --video virtio \
  --hostdev $GPU \
  --hostdev $AUDIO \
  --boot uefi \
  --os-variant ubuntu22.04

Inside the VM:

  • Install the vendor GPU driver and toolkit (CUDA/ROCm) appropriate for the guest OS and kernel following the vendor’s official instructions.

  • Then install your frameworks (e.g., PyTorch, TensorFlow) in a venv/conda environment.

Tip: For large datasets, mount a host directory with virtiofs for speed:

# On modern libvirt/QEMU, add a virtiofs share
virsh edit ai-gpu-dev
# Add within <devices>:
# <filesystem type='mount' accessmode='passthrough'>
#   <driver type='virtiofs'/>
#   <source dir='/data/datasets'/>
#   <target dir='datasets'/>
# </filesystem>

Then inside the VM:

sudo mkdir -p /mnt/datasets
sudo mount -t virtiofs datasets /mnt/datasets

Case Study 3: Reproducible research sandbox with Jupyter over SSH

Goal: A clean, versioned, and shareable environment for experiments. No open ports; access via SSH tunnel.

1) Create cloud-init to set up Jupyter Lab:

cd ~/vm-images
cat > research-user-data.yaml <<'YAML'
#cloud-config
users:
  - name: ai
    groups: [sudo]
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    shell: /bin/bash
    ssh_authorized_keys:
      - ssh-ed25519 AAAA...your_ssh_key_here
package_update: true
packages:
  - python3
  - python3-pip
runcmd:
  - python3 -m pip install --upgrade pip
  - python3 -m pip install jupyterlab numpy pandas matplotlib scikit-learn
  - |
    cat >/etc/systemd/system/jupyter.service <<'UNIT'
    [Unit]
    Description=Jupyter Lab (loopback only)
    After=network-online.target
    [Service]
    User=ai
    ExecStart=/usr/bin/python3 -m jupyter lab --ip 127.0.0.1 --no-browser --NotebookApp.token='' --NotebookApp.password=''
    Restart=on-failure
    [Install]
    WantedBy=multi-user.target
    UNIT
  - systemctl daemon-reload
  - systemctl enable --now jupyter
YAML

cat > research-meta.yaml <<'YAML'
instance-id: ai-research-1
local-hostname: ai-research-1
YAML

cloud-localds research-seed.iso research-user-data.yaml research-meta.yaml \
  || genisoimage -output research-seed.iso -volid cidata -joliet -rock research-user-data.yaml research-meta.yaml

2) Start the VM:

virt-install \
  --name ai-research-1 \
  --memory 8192 --vcpus 4 --cpu host-passthrough \
  --disk path=$HOME/vm-images/ai-research-1.qcow2,backing_store=$HOME/vm-images/jammy-server-cloudimg-amd64.img,format=qcow2,size=40 \
  --disk path=$HOME/vm-images/research-seed.iso,device=cdrom \
  --network network=default,model=virtio \
  --graphics none --import --os-variant ubuntu22.04

3) Tunnel to Jupyter securely:

VMIP=$(virsh domifaddr ai-research-1 | awk '/ipv4/{print $4}' | cut -d/ -f1)
ssh -L 8888:127.0.0.1:8888 ai@$VMIP
# Open http://localhost:8888 in your browser

Optional (inside the VM, extra tooling):

  • apt:
sudo apt update && sudo apt install -y git podman
  • dnf:
sudo dnf install -y git podman
  • zypper:
sudo zypper install -y git podman

Performance tuning checklist (copy/paste ready)

  • Use host CPU features for better vectorization:
virsh setvcpus <vm> <n> --config
virsh setmem <vm> <bytes> --config
# In virt-install: --cpu host-passthrough
  • Hugepages for better memory performance (host):
echo 4096 | sudo tee /proc/sys/vm/nr_hugepages
# Then add --hugepages to virt-install and ensure the VM memory fits in hugepages.

  • Paravirtualized drivers:

    • Use virtio for network and disks (default with virt-install above).
    • Use virtiofs to share large datasets read-mostly from host.

  • Disk format and caching:

# Use qcow2 for snapshots, raw for max performance.
# Consider cache=none, discard=unmap for better I/O semantics with SSD/NVMe.
  • NUMA and pinning (advanced):
virsh vcpupin <vm> 0 0
virsh vcpupin <vm> 1 1
# Pin vCPUs to host cores near your PCIe root complex if doing GPU passthrough.

Troubleshooting quick notes

  • Can’t get an IP? Ensure the default libvirt network is active:
virsh net-list --all
virsh net-start default

  • Cloud-init didn’t run as expected? Increment instance-id in meta-data and reboot, or recreate the VM.

  • GPU not bound to vfio-pci? Secure Boot, initramfs, or driver blacklists might be interfering. Confirm:

lspci -k -s $GPU
sudo dmesg | grep -i vfio

Conclusion and next steps

You don’t need a cluster to build robust AI workflows. Start with the CPU inference farm to get the hang of cloud-init and virt-install. Once you’re comfortable, graduate to GPU passthrough for heavier training workloads. Keep your research sandbox as a reproducible “golden image” you can share with your team.

Call to action:

  • Pick one case study and implement it today.

  • Snapshot your working VM.

  • Measure performance changes as you apply the tuning checklist.

  • Share your Bash and XML snippets with the team so everyone can spin up identical environments.

If you’d like a follow-up deep dive (multi-node VM clusters, SR-IOV vGPU, or CI-driven VM image builds), let me know which topic you want first.

Related posts