AI-Powered Linux Capacity Planning with Bash: From Metrics to Predictions

Ever been paged at 2 a.m. because a server hit 100% CPU, or found out you’ve been paying for idle capacity you don’t use? Traditional capacity planning is reactive and expensive. The good news: you can turn your Linux telemetry into forward-looking insights with a few Bash commands and a tiny bit of machine learning. This guide shows you how to collect the right metrics, build an AI-ready dataset, forecast your needs, and automate action—entirely from the command line.

What you’ll get:

• A minimal, portable metrics pipeline in Bash (no heavy agents)

• Actionable forecasts (e.g., “You’ll need 12 vCPUs next Monday at noon”)

• Early anomaly detection to prevent outages

Why AI for Linux capacity planning?

• Precision beats guesswork: AI can capture seasonality (work-hours vs nights/weekends), spikes, and growth trends better than static thresholds.

• Cost and risk: Right-sizing reduces overprovisioning costs and helps you avoid downtime from saturation.

• Linux-friendly: You don’t need a data lake. Careful Bash collection + a small Python model is enough to start.

Prerequisites: install the basics

These packages are used in the steps below. Install them with your distro’s package manager.

• Python 3 and pip (for the tiny ML scripts)

• cron (for scheduled metric collection)

• Optional: sysstat (if you prefer sar/iostat over /proc-based sampling)

Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y python3 python3-pip cron
# Optional
sudo apt install -y sysstat

Fedora/RHEL/CentOS (dnf):

sudo dnf install -y python3 python3-pip cronie
# Optional
sudo dnf install -y sysstat
# Enable/Start cron service if needed:
sudo systemctl enable --now crond

openSUSE/SLE (zypper):

sudo zypper refresh
sudo zypper install -y python3 python3-pip cron
# Optional
sudo zypper install -y sysstat
# Enable/Start cron service if needed:
sudo systemctl enable --now cron

Python packages (user install):

pip3 install --user pandas scikit-learn numpy

Tip: Ensure ~/.local/bin is on your PATH if using --user.

---

1) Decide what you’re planning for (KPIs and targets)

You can’t model what you don’t define. Pick a small set of KPIs and a headroom policy:

• CPU: target average under 70–75%; track spikes (95th percentile).

• Memory: keep enough headroom to avoid swapping; track MemAvailable%.

• Disk: watch utilization (% busy time) and latency; sustained > 80–90% busy is risky.

• Network: track sustained RX/TX bandwidth and burst behavior.

• Headroom rule of thumb:

  • Capacity_needed = Peak_predicted / Target_utilization
  • Example: If you expect 560% CPU across cores at peak and target 70% utilization, you need ceil(560/70) = 8 vCPUs.

Sanity checks you can run right now:

uptime             # quick load snapshot
cat /proc/meminfo  # MemAvailable
cat /proc/loadavg  # 1/5/15-min load

---

2) Collect the right telemetry every minute (pure Bash)

The script below samples once per minute and writes a CSV with:

• Timestamp

• CPU used (%)

• Load average (1 min)

• Memory used (%)

• Disk busy time (%)

• Network RX/TX (kbps)

It uses /proc and /sys for portability and avoids locale-dependent parsing.

Create metrics.sh:

#!/usr/bin/env bash
set -euo pipefail

OUT_DIR="${OUT_DIR:-/var/log/capacity}"
OUT_CSV="${OUT_CSV:-$OUT_DIR/metrics.csv}"

# Auto-detect root block device if DISK is not set (e.g., sda, nvme0n1)
detect_disk() {
  local src dev base
  src=$(findmnt -n -o SOURCE / || true)          # e.g., /dev/sda2 or /dev/nvme0n1p1
  dev=${src#/dev/}
  base=$(sed -E 's/p?[0-9]+$//' <<< "$dev")      # strip partition suffix
  echo "${base:-sda}"
}

# Auto-detect primary net dev if NETDEV is not set
detect_netdev() {
  ip route 2>/dev/null | awk '/default/ {print $5; exit}' || echo "eth0"
}

DISK="${DISK:-$(detect_disk)}"
NETDEV="${NETDEV:-$(detect_netdev)}"

mkdir -p "$OUT_DIR"
if [ ! -s "$OUT_CSV" ]; then
  echo "ts,cpu_used_pct,load1,mem_used_pct,disk_util_pct,net_rx_kbps,net_tx_kbps" > "$OUT_CSV"
fi

read_cpu() {
  # outputs: total idle
  # /proc/stat: cpu user nice system idle iowait irq softirq steal guest guest_nice
  local arr total idle
  read -r _ user nice system idle iowait irq softirq steal guest guest_nice < /proc/stat
  total=$((user+nice+system+idle+iowait+irq+softirq+steal))
  echo "$total $((idle+iowait))"
}

read_disk_io_ticks() {
  # field 10 in /sys/block/$DISK/stat is "io_ticks" in ms
  awk '{print $10}' "/sys/block/$DISK/stat"
}

read_net_bytes() {
  local rx tx
  read -r rx < "/sys/class/net/$NETDEV/statistics/rx_bytes"
  read -r tx < "/sys/class/net/$NETDEV/statistics/tx_bytes"
  echo "$rx $tx"
}

# Snapshots before interval
ts=$(date -u +"%Y-%m-%d %H:%M:%S")
read -r cpu_total1 cpu_idle1 < <(read_cpu)
disk_ticks1=$(read_disk_io_ticks)
read -r rx1 tx1 < <(read_net_bytes)

# Sleep for sampling interval
sleep 1

# Snapshots after interval
read -r cpu_total2 cpu_idle2 < <(read_cpu)
disk_ticks2=$(read_disk_io_ticks)
read -r rx2 tx2 < <(read_net_bytes)

# CPU used %
cpu_delta=$((cpu_total2 - cpu_total1))
idle_delta=$((cpu_idle2 - cpu_idle1))
cpu_used_pct=$(awk -v c="$cpu_delta" -v i="$idle_delta" 'BEGIN{ if(c>0){print (100*(c - i)/c)} else {print 0}}')

# Load 1-min
load1=$(awk '{print $1}' /proc/loadavg)

# Memory used %
mem_total_kb=$(awk '/MemTotal/ {print $2}' /proc/meminfo)
mem_avail_kb=$(awk '/MemAvailable/ {print $2}' /proc/meminfo)
mem_used_pct=$(awk -v t="$mem_total_kb" -v a="$mem_avail_kb" 'BEGIN{ if(t>0){print (100*(1 - a/t))} else {print 0}}')

# Disk util % over 1s (io_ticks ms / 10)
disk_io_ms=$((disk_ticks2 - disk_ticks1))
disk_util_pct=$(awk -v ms="$disk_io_ms" 'BEGIN{v=ms/10.0; if(v>100) v=100; if(v<0) v=0; print v}')

# Network kbps
rx_kbps=$(awk -v d="$((rx2 - rx1))" 'BEGIN{ if(d<0) d=0; print (8*d/1000.0) }')
tx_kbps=$(awk -v d="$((tx2 - tx1))" 'BEGIN{ if(d<0) d=0; print (8*d/1000.0) }')

echo "$ts,$cpu_used_pct,$load1,$mem_used_pct,$disk_util_pct,$rx_kbps,$tx_kbps" >> "$OUT_CSV"

Make it executable and schedule it:

sudo install -m 0755 metrics.sh /usr/local/bin/metrics.sh
sudo mkdir -p /var/log/capacity
# Run every minute
( crontab -l 2>/dev/null; echo '* * * * * /usr/local/bin/metrics.sh' ) | crontab -

Let this run for at least a few hours (ideally a week) to capture patterns.

Real-world example:

• A fintech team found that Monday 09:00–11:00 CPU peaked 35% higher than other weekdays due to customer logins. A 70% target utilization implied moving from 8 to 12 vCPUs on the front-end tier during that window. Auto-scaling policy was updated accordingly.

---

3) Forecast your CPU needs with a tiny model (lag-based regression)

We’ll train a lightweight model to predict the next 60 minutes of CPU. It uses only pandas and scikit-learn and runs in seconds.

Install Python deps (if you haven’t):

pip3 install --user pandas scikit-learn numpy

Create forecast_capacity.py:

#!/usr/bin/env python3
import argparse, math
import pandas as pd
import numpy as np
from sklearn.linear_model import LinearRegression

def make_lag_features(series, lags=5):
    df = pd.DataFrame({"y": series})
    for i in range(1, lags+1):
        df[f"lag_{i}"] = df["y"].shift(i)
    return df.dropna()

def main():
    p = argparse.ArgumentParser(description="Forecast CPU usage and recommend vCPU capacity.")
    p.add_argument("--csv", default="/var/log/capacity/metrics.csv")
    p.add_argument("--horizon", type=int, default=60, help="Minutes to forecast")
    p.add_argument("--vcpu", type=int, required=True, help="Current total vCPUs for this node/tier")
    p.add_argument("--target-util", type=float, default=0.70, help="Target utilization (0-1)")
    args = p.parse_args()

    df = pd.read_csv(args.csv)
    df["ts"] = pd.to_datetime(df["ts"])
    df = df.sort_values("ts").set_index("ts")

    # Resample to 1-minute, fill small gaps
    df = df.resample("1min").mean().interpolate(limit=5)

    y = df["cpu_used_pct"].clip(lower=0, upper=100)
    feat = make_lag_features(y, lags=5)
    y_aligned = feat["y"].copy()
    X = feat.drop(columns=["y"])

    # Train using all but last 'horizon' points
    if len(X) <= args.horizon + 5:
        raise SystemExit("Not enough data to forecast. Collect more minutes of metrics.")

    X_train = X.iloc[:-args.horizon]
    y_train = y_aligned.iloc[:-args.horizon]

    model = LinearRegression()
    model.fit(X_train, y_train)

    # Recursive forecast for next horizon minutes
    history = list(y.values)
    preds = []
    for _ in range(args.horizon):
        feats = [history[-i] for i in range(1, 6)]
        pred = float(model.predict([feats])[0])
        pred = min(max(pred, 0.0), 100.0)  # clamp
        preds.append(pred)
        history.append(pred)

    last_ts = df.index[-1]
    future_idx = pd.date_range(last_ts + pd.Timedelta(minutes=1), periods=args.horizon, freq="1min")
    pred_series = pd.Series(preds, index=future_idx, name="pred_cpu_pct")

    peak_pred = float(pred_series.max())
    rec_vcpu = max(1, math.ceil(args.vcpu * (peak_pred/100.0) / args.target_util))

    print("Forecast summary")
    print("----------------")
    print(f"Horizon: {args.horizon} min")
    print(f"Peak predicted CPU: {peak_pred:.1f}%")
    print(f"Current vCPU: {args.vcpu}")
    print(f"Target utilization: {args.target_util*100:.0f}%")
    print(f"Recommended vCPU: {rec_vcpu}")
    print()
    print("Top 5 predicted peaks:")
    print(pred_series.sort_values(ascending=False).head(5).to_string())

if __name__ == "__main__":
    main()

Run it:

python3 forecast_capacity.py --vcpu 8 --horizon 60

Output example:

Forecast summary
----------------
Horizon: 60 min
Peak predicted CPU: 83.4%
Current vCPU: 8
Target utilization: 70%
Recommended vCPU: 10

Top 5 predicted peaks:
2026-07-09 09:59:00    83.4
2026-07-09 10:00:00    82.7
2026-07-09 09:58:00    82.3
2026-07-09 09:57:00    80.9
2026-07-09 09:56:00    79.5

Action: Scale to 10 vCPUs before the 10:00 surge, or shift load.

---

4) Detect anomalies before they become outages (MAD-based)

Catch sudden disk or CPU spikes using robust statistics (Median Absolute Deviation). This flags “weird” points without being fooled by regular peaks.

Create detect_anomalies.py:

#!/usr/bin/env python3
import argparse
import pandas as pd
import numpy as np

def mad_based_flags(series, window=21, z=6.0):
    x = series.copy()
    roll_med = x.rolling(window, center=True, min_periods=5).median()
    roll_mad = (x - roll_med).abs().rolling(window, center=True, min_periods=5).median() * 1.4826
    score = (x - roll_med).abs() / (roll_mad.replace(0, np.nan))
    return score > z

def main():
    p = argparse.ArgumentParser(description="Detect resource anomalies from metrics CSV")
    p.add_argument("--csv", default="/var/log/capacity/metrics.csv")
    p.add_argument("--window", type=int, default=21)
    p.add_argument("--z", type=float, default=6.0)
    args = p.parse_args()

    df = pd.read_csv(args.csv, parse_dates=["ts"]).set_index("ts").sort_index()
    for col in ["cpu_used_pct", "disk_util_pct", "mem_used_pct"]:
        if col not in df.columns:
            continue
        flags = mad_based_flags(df[col], window=args.window, z=args.z) | (df[col] > 95)
        anomalies = df[flags].tail(10)
        if not anomalies.empty:
            print(f"Anomalies for {col}:")
            print(anomalies[[col]].to_string())
            print()

if __name__ == "__main__":
    main()

Run it:

python3 detect_anomalies.py

Example remediation:

• If disk_util_pct anomalies align with a backup job, reschedule it or move it to a disk-optimized node.

• If cpu_used_pct anomalies correlate with a deploy, profile the new code path.

---

5) Automate and act

Wire it up so the system helps you before you’re in trouble.

• Cron the forecast for a daily planning snapshot:

# 07:30 every day; write summary to syslog
30 7 * * * /usr/bin/python3 /usr/local/bin/forecast_capacity.py --vcpu 8 --horizon 180 | /usr/bin/logger -t capacity-forecast

• Gate deploys on available headroom:

  • If predicted peak > 85% with current vCPU, require manual approval.

• Auto-scale or schedule:

  • For VMs: pre-scale before predicted peak, downscale after.
  • For K8s: feed forecasts into HPA/VPA or a custom scaler.

• Close the loop:

  • Store forecasts and realized usage; review misses weekly; adjust lags, horizon, or add features (e.g., day-of-week, hour-of-day).

---

Real-world patterns to watch for

• Busy Monday mornings: user login spikes, API traffic surges.

• Nightly batch jobs: backups/ETL saturate disk and network; stagger them.

• Month-end closes: finance/reporting loads. Pre-scale capacity and cache warm-ups.

• Release days: higher CPU from new features or warm caches; throttle concurrency.

---

Conclusion and next steps

You don’t need a data science team to get practical, AI-assisted capacity planning: 1) Capture clean, minute-level metrics with a 30-line Bash script. 2) Forecast with a simple, transparent model. 3) Detect anomalies early and automate actions.

Your next step:

• Deploy metrics.sh, let it run for a week, then run forecast_capacity.py and detect_anomalies.py.

• Use the forecast to update your scaling policy or a change window.

• Iterate: add features (hour-of-day, day-of-week), expand to memory and network planning, and plug into your CI/CD gates.

If this helped, consider standardizing it across your fleet and building a small internal “capacity report” that leadership can check every Monday morning. Your on-call rotation will thank you.