- Posted on
- Featured Image
Most AI inference runs on CPUs; this hands-on Linux guide shows how to unlock 1.5–4x throughput or lower tail latency without changing your model using five Bash-friendly wins: pin threads, fix NUMA locality, tune math backends and thread counts, set performance governors and irqbalance, and quantise (INT8/BF16). Includes baseline measurement, apt/dnf/zypper installs, tuning loops, and a reusable NUMA-pinned service wrapper.