Posted on
• Artificial Intelligence

Artificial Intelligence Threat Hunting

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

AI-Powered Threat Hunting on Linux: A Bash-First Playbook

If you’re drowning in alerts, you’re not alone. Modern environments generate more telemetry than most teams can triage. The result: missed weak signals, alert fatigue, and dwell time that stretches into days or weeks. Artificial Intelligence can change the game for defenders—not by replacing analysts, but by accelerating detection of subtle, low-and-slow behaviors.

This post shows you how to use AI techniques for threat hunting from your Linux terminal. You’ll get practical, reproducible steps using open-source tools, complete with install commands for apt, dnf, and zypper. No black boxes—just Bash, logs, and a bit of Python.

Why AI for Threat Hunting?

  • Scale and subtlety: Adversaries hide in normals. AI-based anomaly detection can sift millions of events and surface the rare-but-meaningful outliers.

  • Behavior over signatures: When signatures and IOCs lag or are absent, behavior-based models catch beacons, DGAs, or abnormal process trees.

  • Human-in-the-loop: AI prioritizes leads. Analysts validate the output and refine hunts. The combination is faster and more accurate than either alone.

What you’ll build

  • Collect network telemetry with Zeek and Suricata.

  • Extract features from logs (with Bash, jq, and zeek-cut).

  • Run a quick anomaly detector (IsolationForest) to surface weird connections.

  • Hunt for DGAs and beaconing with simple, explainable ML logic.

  • Automate hunts with cron/systemd.

1) Install the tooling

We’ll use Zeek (rich network metadata), Suricata (IDS with JSON logs), TShark (packet CLI), jq (JSON), Python for ML, and auditd (optional, for process hunts). Use the commands for your distro:

Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y zeek suricata tshark jq python3-venv python3-pip auditd

Fedora (dnf):

sudo dnf install -y zeek suricata wireshark-cli jq python3-virtualenv python3-pip audit

RHEL/CentOS Stream (dnf) — enable EPEL first:

sudo dnf install -y epel-release
sudo dnf install -y zeek suricata wireshark-cli jq python3-virtualenv python3-pip audit

openSUSE (zypper):

sudo zypper refresh
sudo zypper install -y zeek suricata wireshark-cli jq python3-virtualenv python3-pip audit

Notes:

  • Some platforms may require enabling additional repositories for Zeek. If a package is missing, consult your distro’s security or network repos or the project’s official packages.

  • To run TShark without sudo, ensure your user is in the wireshark group and dumpcap has proper capabilities.

Optional Wireshark capability setup:

sudo usermod -aG wireshark "$USER"
sudo chgrp wireshark /usr/bin/dumpcap
sudo chmod 750 /usr/bin/dumpcap
sudo setcap cap_net_raw,cap_net_admin=eip /usr/bin/dumpcap

2) Collect and normalize telemetry

  • Start Suricata with JSON (EVE) logs:
sudo suricata -i eth0 -l /var/log/suricata -k none -D --set outputs.eve-log.enabled=yes
  • Quick Zeek run on an interface:
sudo zeek -i eth0 local
  • Or process a PCAP if you prefer offline analysis:
sudo tshark -i eth0 -a duration:60 -w capture.pcap
sudo zeek -r capture.pcap local
  • Verify logs:
ls -1 /var/log/suricata/eve.json
ls -1 /var/log/zeek */conn.log 2>/dev/null

3) Feature engineering with Bash

We’ll turn Zeek’s conn.log into a CSV for simple ML.

  • Locate the latest conn.log:
CONN_LOG=$(ls -1 /var/log/zeek/*/conn.log 2>/dev/null | tail -n1)
echo "$CONN_LOG"
  • Export core features:
echo "ts,orig_h,resp_h,resp_p,proto,service,dur,orig_b,resp_b,conn_state" > conn.csv
zeek-cut -d ts id.orig_h id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state < "$CONN_LOG" >> conn.csv
  • Peek at Suricata DNS queries:
jq -r 'select(.event_type=="dns") | [.timestamp, .src_ip, .dns.rrname, .dns.rcode] | @tsv' /var/log/suricata/eve.json | head

Why this matters: Good features make or break your model. Duration, byte asymmetry, service, and connection state are highly useful for catching C2, exfil, and scanning.

4) Run a fast anomaly detector (IsolationForest)

We’ll fit a basic model to highlight outliers in network behavior.

  • Create a Python virtual environment:
python3 -m venv ~/hunt-venv
source ~/hunt-venv/bin/activate
pip install --upgrade pip
pip install pandas scikit-learn numpy
  • Score anomalies:
python - <<'PY'
import pandas as pd
from sklearn.ensemble import IsolationForest
from sklearn.preprocessing import StandardScaler
import numpy as np

df = pd.read_csv('conn.csv')
num = df[['dur','orig_b','resp_b']].fillna(0)
X = StandardScaler().fit_transform(num)

model = IsolationForest(n_estimators=200, contamination=0.01, random_state=42)
pred = model.fit_predict(X)
df['anomaly'] = pred == -1

suspicious = df[df['anomaly']].sort_values('dur', ascending=False)
print(suspicious.head(30).to_csv(index=False))
PY

Interpretation tips:

  • Long duration with low bytes can indicate beacons.

  • High byte asymmetry (orig_b >> resp_b) can signal exfil or odd uploads.

  • Rare services or ports outside your baseline stand out quickly.

5) Hunt hypotheses with explainable AI

Actionable hunts you can run today:

A) DGA-like domain detection (entropy + digits)

jq -r 'select(.event_type=="dns" and .dns.type=="query") | .dns.rrname' /var/log/suricata/eve.json \
  | sort -u \
  | python - <<'PY'
import sys, math
from collections import Counter

def entropy(s):
    p = [c/len(s) for c in Counter(s).values()]
    return -sum(pi*math.log2(pi) for pi in p)

def suspicious(domain):
    name = domain.split('.')[0].lower()
    if len(name) < 8: return False
    e = entropy(name)
    digits = sum(ch.isdigit() for ch in name)
    return (e > 3.5) and (digits/len(name) > 0.3) and (len(name) > 12)

for dom in map(str.strip, sys.stdin):
    if dom and suspicious(dom):
        print(dom)
PY

Why it works: Many DGAs produce high-entropy, digit-heavy labels. This isn’t perfect, but it’s fast, transparent, and surfaces good leads.

B) Beaconing detection (regular inter-arrival times)

CONN_LOG=$(ls -1 /var/log/zeek/*/conn.log 2>/dev/null | tail -n1)
zeek-cut -d ts id.resp_h < "$CONN_LOG" | awk '{print $1","$2}' > beacons.csv

python - <<'PY'
import pandas as pd
df = pd.read_csv('beacons.csv', names=['ts','dst'])
df['ts'] = pd.to_datetime(df['ts'], unit='s')
out = []
for dst, g in df.sort_values('ts').groupby('dst'):
    s = g['ts'].astype('int64')//10**9
    if len(s) < 6:
        continue
    diffs = s.diff().dropna()
    mean = diffs.mean()
    std = diffs.std()
    cv = (std / mean) if mean else 0
    # Low CV -> regular intervals; require a minimum mean to filter noise
    if cv < 0.2 and mean > 10:
        out.append((dst, len(g), mean, cv))
for dst, n, mu, cv in sorted(out, key=lambda x: (-x[1], x[2]))[:15]:
    print(f"{dst}\tconnections={n}\tavg_interval={mu:.1f}s\tcv={cv:.2f}")
PY

Why it works: Many C2 frameworks beacon on steady timers. Low coefficient of variation (CV) hints at machine-like periodicity.

C) Rare process ancestry (auditd)

  • Start live execve capture:
sudo auditctl -a always,exit -F arch=b64 -S execve -k execs
  • Surface rare parent PIDs or commands:
sudo ausearch -k execs -i \
  | awk -F'ppid=' '/ppid=/ {split($2,a," "); print a[1]}' \
  | sort | uniq -c | sort -n | head

Extend this by building feature vectors of parent->child relationships and scoring rarity over time.

6) Automate and alert

Run hunts on a schedule and send results to your SIEM or chat.

  • Systemd unit (example for beacon hunt):
sudo tee /etc/systemd/system/hunt-beacons.service >/dev/null <<'UNIT'
[Unit]
Description=AI Hunt - Beaconing

[Service]
Type=oneshot
User=root
WorkingDirectory=/root
ExecStart=/bin/bash -lc '
CONN_LOG=$(ls -1 /var/log/zeek/*/conn.log 2>/dev/null | tail -n1);
zeek-cut -d ts id.resp_h < "$CONN_LOG" | awk "{print \$1\",\"\$2}" > /root/beacons.csv;
python /root/beacon_hunt.py > /root/beacon_findings.tsv || true
'
UNIT
  • Timer:
sudo tee /etc/systemd/system/hunt-beacons.timer >/dev/null <<'TIMER'
[Unit]
Description=Run AI Beacon Hunt every 15 minutes

[Timer]
OnCalendar=*:0/15
Persistent=true

[Install]
WantedBy=timers.target
TIMER

sudo systemctl daemon-reload
sudo systemctl enable --now hunt-beacons.timer
  • Cron alternative:
*/15 * * * * /usr/bin/bash -lc '/root/run_hunts.sh' >/var/log/hunts.log 2>&1

Best practices:

  • Version-control your hunt scripts and keep model parameters in config.

  • Store outputs with timestamps and hashes for repeatability.

  • Tag findings with hypotheses and outcomes (true/false positive) to refine future hunts.

Real-world scenarios you’ll catch

  • Stealth beacons to rare IPs with uniform intervals (flagged by CV-based beacon hunt).

  • DGA domains resolving briefly during initial access (flagged by entropy/digit ratio).

  • Data egress via unusual services or asymmetric flows (IsolationForest outliers).

  • Suspicious process trees (new or rare parent->child combos via auditd).

Caveats and guardrails

  • False positives happen. Calibrate thresholds and whitelist known-good behavior.

  • Data quality matters. Garbage in, garbage out—ensure clocks, fields, and interfaces are correct.

  • Privacy and compliance. Avoid retaining sensitive payloads; prefer metadata; follow policy and law.

  • Model drift. Reassess regularly; today’s anomalies can become tomorrow’s normal.

Conclusion and Call to Action

You don’t need a massive data science platform to start AI-assisted hunting. With Zeek, Suricata, Bash, and a few lines of Python, you can elevate your visibility and surface the signals that matter.

Your next steps: 1) Install the tooling and enable logging on one sensor. 2) Run the three hunts above (anomaly, DGA, beaconing) on a day of traffic. 3) Triage the top 20 results with your team; whitelist and iterate. 4) Automate the hunts and feed results to your SIEM or ticketing.

When you’re ready, extend with more features (JA3/JA4, HTTP/2 metadata, user/process context), add supervised models where you have labels, and measure precision/recall of your hunts. Small, steady improvements will compound into a powerful detection program.

Have a favorite hunt or dataset? Share it and let’s keep sharpening the blue team toolkit—one Bash script at a time.

Related posts