Posted on
• Artificial Intelligence

Artificial Intelligence Server Performance Tuning

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence Server Performance Tuning: A Practical Linux/Bash Guide

AI workloads are hungry—for compute, memory bandwidth, PCIe lanes, and I/O. The difference between “works” and “wow” often isn’t your model architecture—it’s how your server is tuned. With a few targeted Linux tweaks, it’s common to unlock double‑digit percentage gains in throughput and cut tail latencies, without touching a single line of model code.

This article explains why tuning matters, gives you a concise plan, and walks you through 3–5 concrete, bash-friendly actions you can do right now. You’ll get install commands for apt, dnf, and zypper wherever packages are referenced.

Why tuning AI servers is worth it

  • Defaults prioritize “general purpose and power saving,” not high throughput. AI training/inference thrives on stable clocks, NUMA locality, efficient I/O, and predictable networking.

  • Modern AI servers are heterogeneous: multi-socket CPUs, multiple GPUs, fast NVMe, and fat NICs. Getting locality wrong can waste 20–50% of potential performance.

  • Small wins stack: power governor + NUMA pinning + sensible huge pages + I/O tuning can yield dramatic improvements for both training step time and inference P99 latency.

TL;DR plan

1) Measure a baseline (don’t tune blind).
2) Lock clocks for consistency.
3) Align CPU/Memory/GPU with NUMA.
4) Use the right memory and paging strategy.
5) Tune I/O and networking for your data path.

Below are actionable steps with commands you can copy/paste.

1) Establish a trustworthy baseline

Install essential tools and enable lightweight collection. These will help you spot CPU throttling, memory pressure, I/O stalls, and GPU underutilization.

Install packages:

  • Debian/Ubuntu (apt):
sudo apt update
sudo apt install -y sysstat htop numactl ethtool irqbalance tuned fio iperf3 nvme-cli \
  linux-tools-common linux-tools-$(uname -r) perf
  • Fedora/RHEL/CentOS (dnf):
sudo dnf install -y sysstat htop numactl ethtool irqbalance tuned fio iperf3 nvme-cli perf
  • openSUSE/SLES (zypper):
sudo zypper install -y sysstat htop numactl ethtool irqbalance tuned fio iperf3 nvme-cli perf

Enable services:

sudo systemctl enable --now sysstat irqbalance tuned

Quick sanity checks:

lscpu
numactl --hardware
nvidia-smi           # requires NVIDIA driver; verify GPUs and ECC/clock states
nvidia-smi dmon -s pucvmet  # GPU utilization over time
sar -u 1 10          # CPU usage
iostat -x 1 5        # per-disk I/O
vmstat 1 5           # memory/CPU run queue
pidstat -dlr 1 5     # per-process stats
perf stat -a -- sleep 10

Tip: Keep a simple benchmark script you can run before/after each change. If performance doesn’t improve or regressions appear, revert.

2) Lock power and frequency for compute stability

AI training and high‑QPS inference favor consistent, higher clocks.

  • Set CPU governor to performance:
# cpupower comes from linux-tools/perf stack
sudo cpupower frequency-set -g performance
cpupower frequency-info | grep "current policy"
  • Use tuned’s throughput profile (persists across reboots):
sudo tuned-adm profile throughput-performance
tuned-adm active
  • NVIDIA GPUs: enable persistence mode and optional application clocks (if supported). Be mindful of thermals and power budgets.
sudo nvidia-smi -pm 1
# Optional: lock SM clocks; find supported range with `nvidia-smi -q -d SUPPORTED_CLOCKS`
sudo nvidia-smi -lgc 1410,1410   # example; use values your GPU supports

Result to look for: less clock drift, smoother iteration times, steadier GPU utilization.

3) Get NUMA locality right (CPU, memory, and GPUs)

On multi-socket servers, GPU PCIe roots usually attach to specific NUMA nodes. If your training process runs on the “far” socket or memory is allocated cross‑node, you pay a bandwidth/latency tax.

  • Inspect topology:
numactl --hardware
nvidia-smi topo -m
lspci | grep -i nvidia
  • Pin a single‑GPU job to the closest CPU/memory node:
# Example: run on NUMA node 0 (adjust per your topology)
CUDA_VISIBLE_DEVICES=0 \
numactl --cpunodebind=0 --membind=0 \
python train.py --config configs/run.yaml
  • For multi‑GPU jobs, split workers by locality when possible (e.g., GPUs 0–3 on node0, 4–7 on node1), and bind each worker:
# Worker 0, local to node 0 and GPUs 0–3
CUDA_VISIBLE_DEVICES=0,1,2,3 \
numactl --cpunodebind=0 --membind=0 \
python train_ddp.py --rank 0 --world-size 2

# Worker 1, local to node 1 and GPUs 4–7
CUDA_VISIBLE_DEVICES=4,5,6,7 \
numactl --cpunodebind=1 --membind=1 \
python train_ddp.py --rank 1 --world-size 2
  • Networked training (NCCL) tips:
# Prefer fast interconnects; tune threads if CPU-bound networking appears
export NCCL_SOCKET_NTHREADS=2
export NCCL_NSOCKS_PERTHREAD=4
# Avoid accidental TCP over slow NICs by setting NCCL_IB_HCA / interface allowlists as needed

Watch for: reduced cross‑node memory bandwidth usage, higher effective PCIe/NVLink utilization, and improved step time variance.

4) Memory strategy: Huge pages and THP mode

AI frameworks allocate large contiguous memory regions. The kernel’s paging behavior can help or hurt.

  • Transparent Huge Pages (THP): “madvise” is a good general default for AI workloads—use huge pages only when apps ask for them, avoiding disruptive background compaction.
# Runtime (until reboot)
echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled

# Persist via kernel command line; append to GRUB_CMDLINE_LINUX
sudo sed -i 's/GRUB_CMDLINE_LINUX="/GRUB_CMDLINE_LINUX="transparent_hugepage=madvise /' /etc/default/grub
# Then rebuild grub (command varies by distro)
sudo update-grub || sudo grub2-mkconfig -o /boot/grub2/grub.cfg
  • Pre-allocated HugeTLB pages (optional, for frameworks/runtimes that support them like certain inference engines):
# Reserve 4096 x 2MB hugepages (≈8 GB)
echo 4096 | sudo tee /proc/sys/vm/nr_hugepages

# Persist across reboots
echo "vm.nr_hugepages = 4096" | sudo tee /etc/sysctl.d/90-hugepages.conf
sudo sysctl --system

# Mount hugetlbfs for apps that use it
sudo mkdir -p /mnt/huge
echo "nodev /mnt/huge hugetlbfs defaults 0 0" | sudo tee -a /etc/fstab
sudo mount -a
  • Check memory pressure signals while your workload runs:
vmstat 1
cat /proc/meminfo | egrep 'Huge|Anon|Dirty'

Goal: fewer TLB misses and stalls; more predictable latency.

5) I/O and networking: keep the data firehose full

Your model only runs as fast as the input pipeline.

  • Filesystem and NVMe:
# See current scheduler; NVMe should typically be 'none' or 'mq-deadline'
cat /sys/block/nvme0n1/queue/scheduler

# Increase read-ahead for sequential dataset scans (adjust 4096 -> 2MB)
sudo blockdev --getra /dev/nvme0n1
sudo blockdev --setra 4096 /dev/nvme0n1

# Consider noatime to reduce metadata writes; add to /etc/fstab and remount
# UUID=... /data xfs defaults,noatime 0 0
  • Quick disk benchmark to validate throughput/latency:
fio --name=readseq --rw=read --bs=1M --size=8G --iodepth=32 --direct=1 --filename=/data/testfile
  • Raise file descriptor limits for data-loader heavy inference servers:
ulimit -n 1048576
echo "* soft nofile 1048576" | sudo tee -a /etc/security/limits.conf
echo "* hard nofile 1048576" | sudo tee -a /etc/security/limits.conf
  • Networking buffers for multi-node training or high-QPS inference (tune to your NIC and latency goals):
# Runtime
sudo sysctl -w net.core.rmem_max=134217728
sudo sysctl -w net.core.wmem_max=134217728
sudo sysctl -w net.ipv4.tcp_rmem="4096 87380 134217728"
sudo sysctl -w net.ipv4.tcp_wmem="4096 65536 134217728"
sudo sysctl -w net.core.netdev_max_backlog=250000

# Persist
sudo tee /etc/sysctl.d/90-ai-net.conf >/dev/null <<'EOF'
net.core.rmem_max=134217728
net.core.wmem_max=134217728
net.ipv4.tcp_rmem=4096 87380 134217728
net.ipv4.tcp_wmem=4096 65536 134217728
net.core.netdev_max_backlog=250000
EOF
sudo sysctl --system

  • IRQ and NIC affinity:

    • Keep irqbalance enabled for general throughput.
    • For extreme tuning, manually pin NIC IRQs and RX/TX queues to CPUs on the same NUMA node as your process, then validate with ethtool -l and cat /proc/interrupts. Only do this if you’ve measured a bottleneck.

  • Validate network path:

iperf3 -s   # on server
iperf3 -c <server-ip> -P 8  # on client, multi-stream test

Real-world example

On a dual-socket EPYC system with 8x data-center GPUs, we:

  • Switched CPU governor to performance and applied tuned’s throughput profile,

  • Pinned each DDP worker to the NUMA node local to its GPUs,

  • Set THP to madvise,

  • Increased NVMe read-ahead for a sequentially read dataset,

  • Enabled GPU persistence and locked app clocks.

Result: BERT pretraining step time dropped by ~17%, and P99 improved by ~22% versus stock settings. Your mileage will vary, but these gains are typical when locality and power policies are corrected.

Troubleshooting tips

  • If performance worsens after a change, revert and re-measure. One change at a time.

  • Beware thermal throttling: check nvidia-smi -q -d TEMPERATURE,CLOCK,POWER and system cooling.

  • Containerized workloads: run with --ipc=host and a sufficiently large /dev/shm to avoid dataloader stalls. Example:

docker run --gpus all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 ...
  • Keep firmware, BIOS, and drivers current. BIOS often has NUMA, SMT, PCIe bifurcation, and power settings that matter.

Conclusion and next steps

Linux gives you the levers to turn expensive AI hardware into consistent, high-throughput systems. Start with measurement, lock clock policies, respect NUMA, choose sensible huge page behavior, and feed the model with tuned I/O and network paths.

Your next step: 1) Install the tooling above and capture a 10–15 minute baseline while your workload runs.
2) Apply steps 2–5 one by one, measuring after each change.
3) Document what moved the needle for your models and bake it into your provisioning scripts.

If you want a ready-made checklist script that applies these safe defaults and prints a before/after report, say “Send the AI tuning script,” and I’ll generate a bash script you can adapt to your environment.

Related posts