Posted on
• Artificial Intelligence

Artificial Intelligence Network Security Reviews

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

AI-Assisted Network Security Reviews on Linux (with Bash)

What if your Linux box could draft a weekly network security review while you sleep—surfacing the scariest connections, odd DNS bursts, and stealthy exfiltration patterns? That’s the promise of AI-assisted network security reviews: pairing open-source sensors (Zeek, Suricata) with lightweight anomaly detection to prioritize what deserves your eyes first.

This post shows you how to stand up a Bash-friendly pipeline that:

  • Collects and summarizes network activity

  • Trains a simple, explainable anomaly model

  • Generates a repeatable “AI network security review” you can automate

You’ll get actionable commands, install steps for apt/dnf/zypper, and a minimal script you can tweak for your environment.

Why AI-Assisted Reviews Are Worth Your Time

  • Signal-to-noise: Analysts drown in logs and alerts. Unsupervised anomaly detection can spotlight outliers (weird ports, unusual byte ratios, sudden domain bursts) without perfect signatures.

  • Evasive adversaries: Encryption and living-off-the-land tactics reduce traditional IDS visibility; metadata analysis still reveals “shape of traffic” anomalies.

  • Faster triage: Even a basic model can cut review time by ranking hosts/flows by how unusual they are compared to your baseline.

  • Open-source, on your terms: No vendor lock-in. You own the data, models, and review cadence.

1) Install the toolchain

We’ll use Zeek (rich network metadata), Suricata (IDS + JSON events), tshark/tcpdump (capture), Python (anomaly model), jq (JSON parsing), and Git.

Run the commands for your distro:

  • Debian/Ubuntu (apt)
sudo apt update
sudo apt install -y zeek suricata tshark python3 python3-pip python3-venv jq git
  • Fedora/RHEL/CentOS Stream (dnf)
# On RHEL/Alma/Rocky, you may need EPEL for some packages:
# sudo dnf install -y epel-release

sudo dnf install -y zeek suricata wireshark-cli python3 python3-pip jq git
  • openSUSE Leap/Tumbleweed (zypper)
sudo zypper refresh
sudo zypper install -y zeek suricata wireshark python3 python3-pip jq git

Create a Python virtual environment and install minimal ML libs:

python3 -m venv ~/.venvs/ainet
source ~/.venvs/ainet/bin/activate
pip install --upgrade pip
pip install pandas scikit-learn numpy

Notes:

  • The tshark binary is provided by tshark (apt), wireshark-cli (dnf), or wireshark (zypper).

  • You’ll need root or proper capabilities to capture packets. For offline PCAPs, you can run tools as a normal user.

2) Capture a slice and generate network summaries

Grab a short capture window (e.g., 5 minutes) and build logs you can feature-engineer.

  • Capture 5 minutes of traffic to a PCAP:
sudo tcpdump -i eth0 -w capture.pcap -G 300 -W 1
  • Produce Suricata JSON events (EVE):
mkdir -p suri-logs
sudo suricata -r capture.pcap -l suri-logs
# Output: suri-logs/eve.json
  • Produce Zeek logs (connection metadata, DNS, etc.):
mkdir -p zeek-logs
zeek -r capture.pcap Log::default_writer=Log::WRITER_ASCII Log::dir=zeek-logs
# Output includes zeek-logs/conn.log, dns.log, http.log (if present), etc.

Quick sanity checks:

head -n 3 zeek-logs/conn.log
jq -c '.[0]' suri-logs/eve.json

3) Extract features for a simple anomaly model

Start with Zeek’s connection log (concise and broadly available) and craft a CSV of numeric features. You can expand later with Suricata EVE or Zeek DNS/HTTP.

  • Make a lightweight features CSV from conn.log:
# Convert Zeek's tab-separated log into CSV-like output with selected fields
# -d prints UNIX epoch for timestamps
zeek-cut -d ts id.orig_h id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state < zeek-logs/conn.log \
  | awk 'BEGIN{FS="\t"; OFS=","; print "ts,src,dst,dst_port,proto,service,duration,orig_bytes,resp_bytes,conn_state"}{print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10}' \
  > features.csv

Optional: add DNS NXDOMAIN ratios from Suricata (can help flag DGA-like traffic):

# Count NXDOMAINs per source IP in the Suricata EVE
jq -r 'select(.event_type=="dns") | [.src_ip, .dns.rcode] | @tsv' suri-logs/eve.json \
 | awk 'BEGIN{FS="\t"} {total[$1]++; if($2=="NXDOMAIN") nxd[$1]++} END{print "src,nx_ratio"; for (s in total) {nx=(nxd[s]+0)/total[s]; print s","nx}}' \
 > nxratio.csv

You can join nxratio.csv into your model workflow later. Let’s keep the first pass simple: use features.csv.

4) Train and score with a minimal Isolation Forest

Create a tiny Python script to:

  • Load features.csv

  • Build numeric features

  • Train an Isolation Forest (unsupervised)

  • Output a ranked CSV of anomalies

Save as ai_review.py:

#!/usr/bin/env python3
import os
import sys
import pandas as pd
import numpy as np
from sklearn.ensemble import IsolationForest
from sklearn.preprocessing import StandardScaler

in_csv = sys.argv[1] if len(sys.argv) > 1 else "features.csv"
out_csv = sys.argv[2] if len(sys.argv) > 2 else "ai_findings.csv"
contamination = float(os.environ.get("CONTAMINATION", "0.02"))

df = pd.read_csv(in_csv)

# Basic numeric features (extend as needed)
df["b_ratio"] = (df["orig_bytes"] + 1.0) / (df["resp_bytes"] + 1.0)
df["log_duration"] = np.log1p(df["duration"])
df["log_ob"] = np.log1p(df["orig_bytes"])
df["log_rb"] = np.log1p(df["resp_bytes"])

X = df[["dst_port", "log_duration", "log_ob", "log_rb", "b_ratio"]].fillna(0).values
scaler = StandardScaler()
Xn = scaler.fit_transform(X)

model = IsolationForest(n_estimators=200, contamination=contamination, random_state=42)
model.fit(Xn)
scores = -model.score_samples(Xn)  # higher = more anomalous
labels = model.predict(Xn)         # -1 anomalous, 1 normal

df["anomaly_score"] = scores
df["anomaly_label"] = labels
df_sorted = df.sort_values("anomaly_score", ascending=False)

# Keep a compact report
cols = ["ts","src","dst","dst_port","proto","service","duration","orig_bytes","resp_bytes","b_ratio","anomaly_score","anomaly_label"]
df_sorted[cols].to_csv(out_csv, index=False)

print(f"Wrote {out_csv}. Top 5:")
print(df_sorted[cols].head(5).to_string(index=False))

Run it:

source ~/.venvs/ainet/bin/activate
python3 ai_review.py features.csv ai_findings.csv

You now have ai_findings.csv with the most unusual flows at the top.

5) One-command review: Bash wrapper you can cron

Bundle the steps (capture → logs → features → model → report) into a single script.

Save as ai-net-review.sh:

#!/usr/bin/env bash
set -euo pipefail

# Config
IFACE="${IFACE:-eth0}"
DUR_SEC="${DUR_SEC:-300}"      # capture window
OUTDIR="${OUTDIR:-/tmp/ai-review}"
CONTAMINATION="${CONTAMINATION:-0.02}"

mkdir -p "$OUTDIR"
cd "$OUTDIR"

echo "[*] Capturing ${DUR_SEC}s on ${IFACE} ..."
sudo tcpdump -i "$IFACE" -w capture.pcap -G "$DUR_SEC" -W 1 >/dev/null 2>&1

echo "[*] Running Suricata and Zeek ..."
mkdir -p suri zeek
sudo suricata -r capture.pcap -l suri >/dev/null 2>&1 || true
zeek -r capture.pcap Log::default_writer=Log::WRITER_ASCII Log::dir=zeek >/dev/null 2>&1

echo "[*] Building features ..."
zeek-cut -d ts id.orig_h id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state < zeek/conn.log \
  | awk 'BEGIN{FS="\t"; OFS=","; print "ts,src,dst,dst_port,proto,service,duration,orig_bytes,resp_bytes,conn_state"}{print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10}' \
  > features.csv

echo "[*] Scoring anomalies ..."
source ~/.venvs/ainet/bin/activate
CONTAMINATION="$CONTAMINATION" python3 - <<'PYCODE'
import os, pandas as pd, numpy as np
from sklearn.ensemble import IsolationForest
from sklearn.preprocessing import StandardScaler

df = pd.read_csv("features.csv")
df["b_ratio"] = (df["orig_bytes"] + 1.0) / (df["resp_bytes"] + 1.0)
df["log_duration"] = np.log1p(df["duration"])
df["log_ob"] = np.log1p(df["orig_bytes"])
df["log_rb"] = np.log1p(df["resp_bytes"])
X = df[["dst_port","log_duration","log_ob","log_rb","b_ratio"]].fillna(0).values
Xn = StandardScaler().fit_transform(X)
model = IsolationForest(n_estimators=200, contamination=float(os.environ.get("CONTAMINATION","0.02")), random_state=42)
df["anomaly_score"] = -model.score_samples(Xn)
df = df.sort_values("anomaly_score", ascending=False)
df.to_csv("ai_findings.csv", index=False)
PYCODE

echo "[*] Creating a quick Markdown report ..."
{
  echo "# AI Network Security Review"
  echo
  date +"Generated: %F %T %Z"
  echo
  echo "Capture: ${DUR_SEC}s on ${IFACE}"
  echo
  echo "Top 10 anomalous flows:"
  echo
  awk -F, 'NR==1{for(i=1;i<=NF;i++)h[i]=$i} NR>1 && NR<=12 {printf "- %s:%s -> %s:%s proto=%s dur=%.2f ob=%s rb=%s score=%.3f\n", $2,"", $3,$4,$5,$7,$8,$9,$11 }' ai_findings.csv
} > report.md

echo "[*] Done. See $OUTDIR/report.md and $OUTDIR/ai_findings.csv"

Make it executable and run:

chmod +x ai-net-review.sh
./ai-net-review.sh

Automate weekly (e.g., Sundays at 02:00):

crontab -e
# Add:
0 2 * * 0 IFACE=eth0 DUR_SEC=600 OUTDIR=/var/log/ai-review /path/to/ai-net-review.sh >> /var/log/ai-review/cron.log 2>&1

6) What to look for (real-world patterns and quick pivots)

  • Sharp port-scans or lateral movement
    • Symptom: single host hitting many dst_port values rapidly.
    • Pivot:
awk -F, 'NR>1 {k=$2","$3; cnt[k","$4]++} END{for (c in cnt) print c","cnt[c]}' ai_findings.csv | sort -t, -k5,5nr | head
  • Data exfiltration or unusual egress
    • Symptom: very large orig_bytes to an external dst.
    • Pivot:
awk -F, 'NR==1{for(i=1;i<=NF;i++)h[i]=$i} NR>1 && $8>10000000 {print}' ai_findings.csv | sort -t, -k8,8nr | head
  • DGA/suspicious DNS activity (if you parsed Suricata EVE)
    • Symptom: high NXDOMAIN ratio per source.
    • Pivot:
jq -r 'select(.event_type=="dns") | [.src_ip, .dns.rrname, .dns.rcode] | @tsv' suri-logs/eve.json | head
  • Protocol/service mismatch or odd services
    • Symptom: unknown service on common port or vice versa (Zeek’s service + proto fields).
    • Pivot:
awk -F, 'NR>1 && $6=="" {print}' ai_findings.csv | head

Always validate top anomalies with packet inspection if needed:

tshark -r capture.pcap -Y "ip.src==X.X.X.X && ip.dst==Y.Y.Y.Y && tcp.dstport==PORT" -V | less

Guardrails and tuning

  • False positives happen. Start with a small contamination (1–3%), review top N, and tune features.

  • Baselines matter. Save and retrain on known-good periods or exclude maintenance windows to reduce noise.

  • Privacy and compliance. Network metadata can reveal sensitive patterns; set retention and access controls.

  • Defense-in-depth. Use AI review to prioritize, not to replace, signature IDS or blocklists.

Conclusion: Your next 60 minutes

  • Install the toolchain with your package manager.

  • Run the wrapper script on a short capture.

  • Skim the top anomalies; add one or two features (e.g., NXDOMAIN ratio).

  • Schedule a weekly run and iterate.

If this helped, turn it into a team ritual: a Monday “AI network security review” stand-up. As you enrich features and baselines, your signal will only get sharper—without drowning in raw logs.

Related posts