Posted on
• Artificial Intelligence

AI-Based Bandwidth Analysis

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

AI-Based Bandwidth Analysis on Linux with Bash: From Packets to Anomalies

Ever looked at a graph that showed “100% bandwidth utilized” and still couldn’t answer why? Or noticed those mysterious 2 A.M. spikes that vanish before you can tcpdump them? This guide shows how to build a lightweight, AI-assisted bandwidth analyzer that runs from your Linux shell. You’ll collect packet metadata with tshark, turn it into time-windowed features, train an anomaly detector, and monitor in real time—all scriptable, reproducible, and friendly to encrypted traffic.

What you’ll get:

  • A capture-to-features Bash pipeline using tshark + awk

  • A tiny Python model (Isolation Forest) that flags outliers in bandwidth patterns

  • Ready-to-run commands and scripts, including install instructions for apt, dnf, and zypper

Why this works:

  • You don’t need payloads to understand bandwidth behavior—headers and simple counts (bytes, packets, protocol mix) are enough to spot unusual periods.

  • AI baselines your normal, then highlights unusual bursts, drops, or protocol shifts—even when your averages look “fine.”

  • It’s lightweight and respects least privilege (no long-running root processes).

Prerequisites and Install

We’ll use:

  • tshark (the Wireshark CLI) for packet metadata

  • vnstat for quick interface stats

  • iperf3 for controlled test traffic

  • Python 3 + pip + a few ML libs

  • setcap to capture without root

Install with your package manager:

  • Debian/Ubuntu (apt)
sudo apt update
sudo apt install -y tshark iperf3 vnstat python3 python3-pip libcap2-bin
  • Fedora/RHEL/CentOS (dnf)
sudo dnf install -y wireshark-cli iperf3 vnstat python3 python3-pip libcap
  • openSUSE (zypper)
sudo zypper refresh
sudo zypper install -y wireshark-cli iperf3 vnstat python3 python3-pip libcap-progs

Grant non-root capture safely:

sudo groupadd -f pcap
sudo usermod -a -G pcap $USER
sudo chgrp pcap /usr/bin/dumpcap
sudo chmod 750 /usr/bin/dumpcap
sudo setcap cap_net_raw,cap_net_admin=eip /usr/bin/dumpcap
# Log out and back in so new group applies

Python packages:

python3 -m pip install --user --upgrade pip
python3 -m pip install --user pandas scikit-learn joblib

Overview of the Pipeline

1) Baseline your link quickly (vnstat, iperf3)
2) Capture and aggregate metadata in fixed windows (tshark + awk)
3) Train an Isolation Forest on “normal” data
4) Monitor live and alert on anomalies
5) Iterate (tune thresholds, filters, and windows)

1) Baseline: Know Your Normal

  • Quick live view with vnstat (replace eth0 with your interface):
IFACE=eth0
vnstat -l -i "$IFACE"
  • Optional controlled test with iperf3:
    • On a test server: iperf3 -s
    • From a client: iperf3 -c <server-ip> -t 20 -R This helps validate that your capture/analysis sees expected traffic levels.

2) Collect Features from Packets (Bash-only)

We’ll capture IP headers only and aggregate into 5-second bins. Features per bin:

  • bytes, packets

  • tcp packets, udp packets (protocol mix)

Capture ~15 minutes of training data:

IFACE=eth0
BIN=5          # seconds
DUR=900        # training capture duration (seconds)
OUT=traffic_5s.csv

tshark -i "$IFACE" -a duration:$DUR -f "ip" -s 96 \
  -T fields -E separator=\t \
  -e frame.time_epoch -e frame.len -e ip.proto 2>/dev/null | \
awk -v BIN="$BIN" 'BEGIN{FS="\t"; OFS=","; print "ts,bytes,pkts,tcp,udp"}
{
  t=int($1/BIN)*BIN
  if (curr=="") curr=t
  bytes[t]+=$2; pkts[t]++
  if($3==6) tcp[t]++; else if($3==17) udp[t]++
  if (t>curr) {
    print curr, bytes[curr]+0, pkts[curr]+0, tcp[curr]+0, udp[curr]+0
    delete bytes[curr]; delete pkts[curr]; delete tcp[curr]; delete udp[curr]
    curr=t
  }
}
END{
  if (curr!="") print curr, bytes[curr]+0, pkts[curr]+0, tcp[curr]+0, udp[curr]+0
}' | tee "$OUT"

Notes:

  • We use a small snap length (-s 96) to reduce overhead (header-only).

  • If the link is busy, consider BPF filters to narrow scope (e.g., exclude known noisy ports).

3) Train an AI Model (Isolation Forest)

We’ll train on the 5-second bins you just captured.

Save as train_bandwidth_model.py:

#!/usr/bin/env python3
import sys, joblib, pandas as pd
from sklearn.ensemble import IsolationForest
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline

CSV = sys.argv[1] if len(sys.argv) > 1 else "traffic_5s.csv"
BIN = 5.0

df = pd.read_csv(CSV)
# Build features
X = pd.DataFrame({
    "bytes_per_s": df["bytes"]/BIN,
    "pkts_per_s": df["pkts"]/BIN,
    "tcp_frac": (df["tcp"]/df["pkts"]).fillna(0.0),
    "udp_frac": (df["udp"]/df["pkts"]).fillna(0.0),
})

pipe = Pipeline([
    ("scaler", StandardScaler()),
    ("iso", IsolationForest(
        n_estimators=200,
        contamination=0.02,   # ~2% of intervals flagged as anomalies
        random_state=42
    )),
])

pipe.fit(X)
joblib.dump(pipe, "bw_iso.joblib")
print("Model saved to bw_iso.joblib; trained on", len(X), "windows")

Train it:

python3 train_bandwidth_model.py traffic_5s.csv

Tuning tips:

  • contamination controls how “sensitive” the detector is; start 0.01–0.05.

  • BIN can be 1s for bursty traffic or 10–30s for slower links.

4) Monitor Live and Alert on Anomalies

We’ll reuse the same live aggregation concept and score each bin as it closes.

Save as score_bandwidth.py:

#!/usr/bin/env python3
import sys, joblib
from datetime import datetime

if len(sys.argv) < 2:
    print("Usage: score_bandwidth.py <model.joblib>", file=sys.stderr)
    sys.exit(1)

model_path = sys.argv[1]
pipe = joblib.load(model_path)
BIN = 5.0

def parse(line):
    # ts,bytes,pkts,tcp,udp
    parts = line.strip().split(",")
    if parts[0] == "ts" or len(parts) < 5:
        return None
    ts = int(float(parts[0]))
    bytes_, pkts, tcp, udp = map(float, parts[1:5])
    x = [
        bytes_/BIN,
        pkts/BIN,
        (tcp/pkts if pkts>0 else 0.0),
        (udp/pkts if pkts>0 else 0.0),
    ]
    return ts, x, (bytes_, pkts, tcp, udp)

for line in sys.stdin:
    item = parse(line)
    if not item:
        continue
    ts, x, raw = item
    pred = pipe.predict([x])[0]              # 1 normal, -1 anomaly
    score = pipe.decision_function([x])[0]   # higher is more normal
    if pred == -1 or score < -0.1:
        iso8601 = datetime.utcfromtimestamp(ts).strftime("%Y-%m-%d %H:%M:%S")
        bytes_, pkts, tcp, udp = raw
        msg = f"ALERT {iso8601}Z score={score:.3f} bytes={bytes_:.0f} pkts={pkts:.0f} tcp={tcp:.0f} udp={udp:.0f}"
        print(msg, flush=True)
        # Optional: send to syslog
        # os.system(f'logger -t ai-bw "{msg}"')

Live watcher (ai-bw-watch.sh):

#!/usr/bin/env bash
set -euo pipefail
IFACE="${1:-eth0}"
BIN=5
MODEL="${2:-bw_iso.joblib}"

tshark -i "$IFACE" -f "ip" -s 96 \
  -T fields -E separator=$'\t' \
  -e frame.time_epoch -e frame.len -e ip.proto 2>/dev/null | \
awk -v BIN="$BIN" 'BEGIN{FS="\t"; OFS=","; print "ts,bytes,pkts,tcp,udp"}
{
  t=int($1/BIN)*BIN
  if (curr=="") curr=t
  bytes[t]+=$2; pkts[t]++
  if($3==6) tcp[t]++; else if($3==17) udp[t]++
  if (t>curr) {
    print curr, bytes[curr]+0, pkts[curr]+0, tcp[curr]+0, udp[curr]+0
    delete bytes[curr]; delete pkts[curr]; delete tcp[curr]; delete udp[curr]
    curr=t
  }
}' | python3 score_bandwidth.py "$MODEL"

Run it:

chmod +x ai-bw-watch.sh
./ai-bw-watch.sh eth0 bw_iso.joblib

You’ll see lines like:

ALERT 2026-07-06 14:23:10Z score=-0.341 bytes=8448790 pkts=6840 tcp=5830 udp=982

Tip: Integrate with logging/monitoring

  • Syslog: uncomment the logger line in score_bandwidth.py

  • Systemd: run as a service or use tmux/screen

  • Grafana/Prometheus: write metrics to a textfile exporter if desired

5) Real-World Ways to Use It

  • Catch unplanned saturation

    • Backups or large file sync jobs that start early can push normal limits. The model flags those windows even if your rolling averages look fine.

  • Spot protocol shifts

    • A sudden jump in UDP relative to TCP can indicate streaming, tunneling, or misconfigurations.

  • Validate QoS or ISP changes

    • Train a model before QoS rules or ISP switch; compare anomaly rates after. If anomaly frequency climbs, something changed.

  • Lab test it

    • While ai-bw-watch.sh runs, generate bursts with iperf3:
    iperf3 -c <server-ip> -u -b 200M -t 30

    You should see alerts during the burst windows.

Why This Approach Is Valid

  • Encryption-friendly: We only analyze metadata and counts; no payload inspection required.

  • Lightweight: Header-only capture and 5-second feature windows are cheap to compute.

  • Adaptive: Isolation Forest infers “normal” from your environment, not a static threshold.

  • Scriptable: The entire workflow is shell-first; easy to automate and version-control.

Tips, Tuning, and Safety

  • Reduce load:

    • Keep snaplen small (-s 96), and consider filters like -f "ip and not port 22 and not broadcast"

  • Tune sensitivity:

    • Adjust contamination in the trainer (0.01–0.05). Too many false positives? Lower it.
    • Change BIN to match your network’s “tempo.”

  • Permissions and policy:

    • Only capture metadata you’re authorized to observe. Follow org policy and local law.

Conclusion and Next Steps

In a few commands, you built an AI-assisted bandwidth analyzer that runs from Bash: capture, aggregate, baseline, and alert. Next steps:

  • Let the trainer run over a representative day (weekday + weekend if applicable).

  • Deploy ai-bw-watch.sh under systemd and log to syslog/Grafana.

  • Iterate filters and thresholds until alerts match your intuition.

When your next 2 A.M. spike hits, you won’t just see it—you’ll know it’s unusual, and you’ll have the data to prove it.

Related posts