Posted on
• Artificial Intelligence

MCP Performance Tuning

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

MCP Performance Tuning: Squeeze More Speed from Linux with Memory, CPU, and Process Tweaks

Ever watched a perfectly capable Linux box stumble under a traffic spike or a heavy batch run—only to find CPU is “fine” but latency still climbs and throughput dives? That’s the gap between “it works” and “it’s tuned.”

In this guide, MCP stands for Memory, CPU, and Processes—the three levers most admins reach for first when they want practical, measurable performance wins without rewriting applications. We’ll explain why MCP tuning matters, how to measure before changing anything, and then walk through a small set of targeted, high-value tweaks you can roll out safely. You’ll also get copy-paste Bash to install the right tools and make changes persistent.

Why MCP Tuning Works

  • Memory, CPU, and Processes dominate most system-level bottlenecks. Poor defaults or mismatched policies (e.g., powersave governors on latency-sensitive nodes) quietly eat performance.

  • Linux is deliberately conservative and general-purpose. You can bias it toward low latency, high throughput, or better isolation with a handful of sysctls, schedulers, and service limits.

  • Small, validated changes compound. For example: CPU governor + IRQ affinity + THP policy + reasonable file descriptor limits can together cut p99 latency dramatically without new hardware.

Prerequisites: Install the right tools

You can’t tune what you can’t see. Install basic observability and tuning helpers first.

Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y sysstat htop numactl irqbalance ethtool tuned
# perf and cpupower live in kernel tools; install the matching version:
sudo apt install -y linux-tools-common linux-tools-generic linux-tools-$(uname -r)
# Optional (older systems): cpufrequtils
sudo apt install -y cpufrequtils
# Start tuned (if installed)
sudo systemctl enable --now tuned || true

RHEL/CentOS/Fedora (dnf):

sudo dnf install -y sysstat htop numactl irqbalance ethtool tuned perf kernel-tools
sudo systemctl enable --now tuned

SUSE/openSUSE (zypper):

sudo zypper install -y sysstat htop numactl irqbalance ethtool tuned perf cpupower
sudo systemctl enable --now tuned

Tip: On some distros sysstat collection is disabled by default. Enable it to build history:

# RHEL/Fedora/SUSE
sudo systemctl enable --now sysstat
# Debian/Ubuntu
sudo sed -i 's/ENABLED="false"/ENABLED="true"/' /etc/default/sysstat
sudo systemctl enable --now sysstat

Step 1: Measure before you change anything

  • CPU, load, run queue:
mpstat -P ALL 1 5
pidstat -u -p ALL 1 5
  • Process contention and hotspots:
top -H
perf top        # live hotspots (root)
perf stat -a -- sleep 10
  • I/O and disk queues:
iostat -xz 1 5
  • Memory pressure and NUMA topology:
free -h
numactl --hardware
  • Network:
sar -n DEV 1 5
ethtool -S eth0 | head

Capture a 10–15 minute baseline during real workload. Only then pick the most relevant knobs below.

Step 2: CPU policy and scheduling

1) Set an appropriate CPU governor
For latency-sensitive services, prefer performance; for throughput/batch, throughput-performance via tuned is great.

  • Quick check and set with cpupower:
cpupower frequency-info | grep "governor"
sudo cpupower frequency-set -g performance
  • Using tuned profiles (recommended and persistent):
sudo tuned-adm profile latency-performance
# or for bulk throughput:
sudo tuned-adm profile throughput-performance
sudo tuned-adm active

2) Keep IRQs balanced—and pin hot ones if needed
General case: leave irqbalance on. For extreme low-latency, pin a chatty NIC’s IRQs away from app cores.

  • See which IRQs are hot:
grep -E "CPU|eth|nvme" /proc/interrupts
  • Pin an IRQ to CPUs 2–3 (example):
echo 2-3 | sudo tee /proc/irq/123/smp_affinity_list

Caution: Test carefully; mis-pinning can hurt more than it helps.

3) Optionally isolate cores for critical work
Reserve a core set for your most sensitive service, and keep housekeeping off them (advanced).

  • Add to kernel cmdline (GRUB): isolcpus=2-7 nohz_full=2-7 rcu_nocbs=2-7

  • Then pin the service:

sudo taskset -c 2-7 chrt -f 1 /usr/bin/myservice

Use only if you understand the trade-offs.

Step 3: Memory and NUMA behavior

1) Right-size swappiness and dirty writeback
Lower swappiness on servers that shouldn’t swap during peaks; adjust dirty ratios to smooth writeback.

  • Test live (non-persistent):
sudo sysctl vm.swappiness=10
sudo sysctl vm.dirty_background_ratio=5
sudo sysctl vm.dirty_ratio=20
  • Make persistent:
sudo tee /etc/sysctl.d/90-mcp-tuning.conf >/dev/null <<'EOF'
vm.swappiness = 10
vm.dirty_background_ratio = 5
vm.dirty_ratio = 20
EOF
sudo sysctl --system

2) Transparent Huge Pages (THP) policy
Many databases prefer THP disabled or madvise to reduce latency spikes.

  • Check and adjust (runtime):
cat /sys/kernel/mm/transparent_hugepage/enabled
echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
  • Persist via tuned custom profile:
sudo mkdir -p /etc/tuned/mcp-thp
sudo tee /etc/tuned/mcp-thp/tuned.conf >/dev/null <<'EOF'
[main]
include=latency-performance

[sysfs]
/sys/kernel/mm/transparent_hugepage/enabled=madvise
EOF
sudo tuned-adm profile mcp-thp

3) NUMA locality for memory-heavy apps
Keep CPU and memory on the same NUMA node to reduce cross-node latency.

  • Discover layout:
numactl --hardware
  • Bind a process to node 0:
numactl --cpunodebind=0 --membind=0 /usr/bin/myservice

Step 4: Process limits and cgroups

1) File descriptors and process limits
Network daemons often need higher nofile.

  • Check and raise temporarily:
ulimit -n
ulimit -n 1048576
  • Persist for a systemd service:
sudo systemctl edit my.service

Then add:

[Service]
LimitNOFILE=1048576

And reload:

sudo systemctl daemon-reload
sudo systemctl restart my.service

2) Priorities and resource shares (cgroups v2 via systemd)
Protect critical services under load.

  • Boost CPU and IO weights; cap memory:
sudo systemctl set-property my.service CPUWeight=900 IOWeight=900 MemoryHigh=2G
  • Set realtime or higher scheduling where appropriate (careful!):
sudo systemctl set-property my.service CPUSchedulingPolicy=fifo CPUSchedulingPriority=1

Step 5: I/O and network fast-paths

1) Choose the right I/O scheduler and read-ahead

  • For NVMe: usually none. For SATA SSD: mq-deadline. For HDD: mq-deadline or bfq.

  • Check and set (runtime):

for d in /sys/block/nvme* /sys/block/sd*; do
  [ -e "$d/queue/scheduler" ] && echo "$d: $(cat $d/queue/scheduler)"
done
echo none | sudo tee /sys/block/nvme0n1/queue/scheduler
echo mq-deadline | sudo tee /sys/block/sda/queue/scheduler
  • Tune read-ahead (example 256 sectors = 128 KiB):
sudo blockdev --getra /dev/sda
sudo blockdev --setra 256 /dev/sda

Persist via udev rules or tuned custom profile if needed.

2) Enable modern TCP congestion control (BBR) for WAN throughput/latency
Available on most modern kernels.

  • Test live:
sudo sysctl net.core.default_qdisc=fq
sudo sysctl net.ipv4.tcp_congestion_control=bbr
sysctl net.ipv4.tcp_congestion_control
  • Persist:
sudo tee /etc/sysctl.d/90-mcp-net.conf >/dev/null <<'EOF'
net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr
EOF
sudo sysctl --system

3) NIC offload sanity and driver stats

  • Verify offloads (keep defaults unless you have a reason):
sudo ethtool -k eth0 | sed -n '1,20p'
  • Watch for drops:
sudo ethtool -S eth0 | egrep 'drop|err|miss'

Real-world examples

  • Low-latency API node
    Set tuned profile to latency-performance, ensure irqbalance is on, switch THP to madvise, and raise LimitNOFILE. Result: tighter p99 tail and fewer timeouts during GC or noisy-neighbor events.

  • Batch ETL server
    Use throughput-performance, increase dirty_ratio to keep disks busier, choose mq-deadline for SSDs, and bind the ETL to one NUMA node. Result: faster completion times and more consistent disk utilization.

  • Database host
    Disable or madvise THP per vendor docs, set swappiness low, pick mq-deadline for SATA SSD or none for NVMe, increase read-ahead modestly, and limit noisy sidecars with cgroup weights. Result: steadier latency under concurrent load.

Make it safe: persistence and rollback

  • Put sysctls in /etc/sysctl.d/*.conf and apply with sysctl --system.

  • Prefer tuned profiles for CPU/THP/IRQ/sysfs when available; it’s easy to switch back:

sudo tuned-adm profile balanced

  • Use systemctl edit drop-ins for service-level limits; revert by removing the drop-in and reloading.

  • Change one knob at a time. Keep baseline metrics and roll back what doesn’t help.

Conclusion and next steps

MCP tuning—Memory, CPU, and Processes—gives you high-impact control over Linux behavior with minimal risk. Start by measuring, apply one or two changes that match your workload, and validate with the same metrics you used for baseline. Bake the wins into tuned profiles, sysctl.d, and systemd drop-ins so they survive reboots and scale with your fleet.

Your next step:

  • Install the tools, record a 15-minute baseline during normal and peak load.

  • Apply Step 2 (CPU) and Step 3 (Memory) tweaks that fit your workload.

  • Validate, then move on to process limits and I/O/network.

If you want, share your before/after metrics and the knobs you used—I’ll help you refine the profile further.

Related posts