Posted on
• Artificial Intelligence

Artificial Intelligence Linux Forensics

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence + Linux Forensics: Practical Bash-Friendly Workflows You Can Run Today

Incidents move fast. Logs pile up faster. When seconds count, analysts don’t have time to read every SSH failure, every socket, or every noisy service log.

This is where AI is genuinely useful in Linux forensics: not as sci‑fi “magic,” but as math that helps you find needles in very big haystacks. With a few Bash-friendly commands and lightweight Python models, you can flag anomalies in auth logs, highlight suspicious networked processes, and cluster noisy logs into meaningful storylines—all on a single Linux host with no cloud dependencies.

Below you’ll find why this approach works, three actionable, real‑world workflows (with copy/paste code), package installation instructions for apt, dnf, and zypper, and a short CTA to help you automate it.

Why AI belongs in your Linux forensics toolkit

  • Scale and speed: Linux hosts produce millions of log lines. Unsupervised models (e.g., Isolation Forest, KMeans) rank outliers fast so you can triage first and deep-dive second.

  • Novelty and drift: Signature and rule-based detections miss weird-but-new behavior. Anomaly detection catches “things that don’t look like the others,” even if you don’t have a rule yet.

  • Endpoint-first, privacy-friendly: Everything below runs locally in user space. No data leaves the box.

  • Reproducible: You can wrap each workflow in Bash, check it into git, and run it the same way every time. That’s vital for defensibility and chain-of-custody.

Note: AI augments, not replaces, forensic rigor. Preserve evidence, record commands, and keep originals read-only.

Prerequisites (install once)

Python 3 and pip:

# Debian/Ubuntu
sudo apt update && sudo apt install -y python3 python3-pip

# Fedora/RHEL/CentOS
sudo dnf install -y python3 python3-pip

# openSUSE
sudo zypper install -y python3 python3-pip

Python libraries:

python3 -m pip install --user pandas scikit-learn numpy

If your shell doesn’t see user-installed tools, add this to your shell rc:

export PATH="$HOME/.local/bin:$PATH"

“ss” for socket listing (usually preinstalled; install if missing):

# Debian/Ubuntu
sudo apt install -y iproute2

# Fedora/RHEL/CentOS
sudo dnf install -y iproute

# openSUSE
sudo zypper install -y iproute2

You’ll also use built-in tools like journalctl and ps (present on most systems). Some commands below need sudo to access full data (e.g., process owners, sockets).

Workflow 1: Flag suspicious SSH sources with anomaly detection

Goal: Rapidly highlight remote IPs with abnormal SSH behavior (e.g., many failures, many distinct usernames, odd hours).

Step 1 — export recent sshd logs:

# Prefer journalctl if systemd is present
sudo journalctl --since "7 days ago" -t sshd -o short-iso > sshd.log

# If your distro logs to files instead:
# Debian/Ubuntu: /var/log/auth.log
# RHEL/CentOS/Fedora: /var/log/secure
# Example fallback:
sudo grep -E "sshd" /var/log/auth.log /var/log/secure 2>/dev/null | sed 's/^/1970-01-01T00:00:00 /' > sshd.log

Step 2 — run the Python analyzer (Isolation Forest):

python3 - <<'PY'
import re, sys, math
from datetime import datetime
import numpy as np
import pandas as pd
from sklearn.ensemble import IsolationForest

log_path = "sshd.log"
rx_ip    = re.compile(r'\bfrom (?P<ip>\d+\.\d+\.\d+\.\d+)\b')
rx_user  = re.compile(r'\bfor (?:invalid user )?(?P<user>[A-Za-z0-9._-]+)\b')
rx_time  = re.compile(r'^(?P<ts>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2})')

events = []
with open(log_path, 'r', errors='ignore') as f:
    for line in f:
        if 'sshd' not in line: 
            continue
        status = 'Failed' if 'Failed password' in line else ('Accepted' if 'Accepted password' in line else None)
        if not status:
            continue
        m_ip = rx_ip.search(line)
        m_usr = rx_user.search(line)
        m_ts = rx_time.search(line)
        ip = m_ip.group('ip') if m_ip else None
        user = m_usr.group('user') if m_usr else 'unknown'
        hour = None
        if m_ts:
            try:
                hour = datetime.fromisoformat(m_ts.group('ts')).hour
            except:
                pass
        if ip:
            events.append((ip, user, status, hour))

if not events:
    print("No SSH auth events parsed. Ensure sshd.log contains sshd lines.")
    sys.exit(0)

# Aggregate per source IP
from collections import defaultdict
agg = defaultdict(lambda: {"total":0,"failed":0,"users":set(),"hours":set()})
for ip, user, status, hour in events:
    a = agg[ip]
    a["total"] += 1
    if status == "Failed":
        a["failed"] += 1
    a["users"].add(user)
    if hour is not None: a["hours"].add(hour)

rows = []
for ip, a in agg.items():
    total = a["total"]
    failed = a["failed"]
    failed_ratio = failed/total if total else 0
    uniq_users = len(a["users"])
    hours_active = len(a["hours"]) if a["hours"] else 0
    rows.append((ip, total, failed, failed_ratio, uniq_users, hours_active))

df = pd.DataFrame(rows, columns=["ip","total","failed","failed_ratio","uniq_users","hours_active"])
if len(df) < 5:
    print(df.sort_values(["failed_ratio","total"], ascending=[False,False]).to_string(index=False))
    sys.exit(0)

X = df[["total","failed_ratio","uniq_users","hours_active"]].values
clf = IsolationForest(n_estimators=200, contamination="auto", random_state=42)
scores = clf.fit_predict(X)
decision = clf.decision_function(X)

df["anomaly"] = scores
df["score"] = -decision  # higher => more anomalous
sus = df[df["anomaly"]==-1].sort_values("score", ascending=False)

print("\nTop suspicious SSH sources (by anomaly score):\n")
print(sus.head(10).to_string(index=False))
print("\nTip: Investigate high failed_ratio, many uniq_users, and odd hours_active.")
PY

Why this works: Attackers often spray passwords from a few IPs, hit many usernames, and show up at odd hours. Unsupervised models surface those quickly—even if you’ve never seen that IP before.

Workflow 2: Spot odd processes making network connections

Goal: Identify processes whose network behavior is unlike their peers (e.g., root-owned binary talking to rare ports or too many unique IPs).

Step 1 — capture IPv4 TCP/UDP sockets with owning processes:

# Needs sudo to reveal process owners and PIDs
sudo ss -H -4 -tunp > sockets.txt

Step 2 — run the Python analyzer:

python3 - <<'PY'
import subprocess, re, pwd, os, statistics
import numpy as np, pandas as pd
from sklearn.ensemble import IsolationForest

# Load ss output (captured earlier)
with open("sockets.txt","r",errors="ignore") as f:
    lines = [l.strip() for l in f if l.strip()]

# Parse lines like:
# ESTAB 0 0 10.0.2.15:53660 172.217.169.36:443 users:(("curl",pid=1234,fd=3))
conns = []
rx_proc = re.compile(r'users:\(\("([^"]+)",pid=(\d+)')
for ln in lines:
    if 'users:(' not in ln: 
        continue
    # Tokens: State ... Local ... Peer ... users:(...)
    parts = ln.split()
    # Peer typically second-to-last token before users:(...)
    # Safer: everything before 'users:(', last token is Peer
    try:
        before_users = ln[:ln.index('users:(')].strip()
        peer = before_users.split()[-1]
        local = before_users.split()[-2]
    except Exception:
        continue
    m = rx_proc.search(ln)
    if not m: 
        continue
    name, pid = m.group(1), int(m.group(2))
    # Extract remote IP and port (IPv4 only due to -4)
    if ':' not in peer or ':' not in local:
        continue
    rip, rport = peer.rsplit(':',1)
    lip, lport = local.rsplit(':',1)
    # Keep only numeric ports
    try:
        rport_i = int(rport)
    except:
        continue
    conns.append((pid, name, lip, int(lport), rip, rport_i))

if not conns:
    print("No sockets parsed. Run: sudo ss -H -4 -tunp > sockets.txt")
    raise SystemExit

# Build per-pid stats
from collections import defaultdict
by_pid = defaultdict(lambda: {"name":None,"rports":[], "rips":set(), "count":0})
for pid, name, lip, lport, rip, rport in conns:
    a = by_pid[pid]
    a["name"] = name
    a["rports"].append(rport)
    a["rips"].add(rip)
    a["count"] += 1

# Enrich with user and command
def ps_info():
    out = subprocess.check_output(["ps","-eo","pid=,ppid=,uid=,comm=,cmd="], text=True, errors="ignore")
    info = {}
    for line in out.splitlines():
        try:
            pid, ppid, uid, comm, cmd = line.strip().split(None,4)
            info[int(pid)] = {"ppid":int(ppid), "uid":int(uid), "comm":comm, "cmd":cmd}
        except ValueError:
            continue
    return info

pinfo = ps_info()

rows = []
for pid, a in by_pid.items():
    if pid not in pinfo: 
        continue
    uid = pinfo[pid]["uid"]
    is_root = 1 if uid == 0 else 0
    rports = a["rports"]
    med_port = statistics.median(rports) if rports else 0
    uniq_ips = len(a["rips"])
    cmd = pinfo[pid]["cmd"]
    rows.append((
        pid, pinfo[pid]["ppid"], uid, a["name"], cmd, a["count"], uniq_ips, med_port, is_root
    ))

df = pd.DataFrame(rows, columns=["pid","ppid","uid","name","cmd","conn_count","uniq_rips","median_rport","is_root"])
if len(df) < 5:
    print(df.sort_values(["conn_count","uniq_rips"], ascending=[False,False]).to_string(index=False))
    raise SystemExit

X = df[["conn_count","uniq_rips","median_rport","is_root"]].values
clf = IsolationForest(n_estimators=300, contamination="auto", random_state=1337)
scores = clf.fit_predict(X)
decision = clf.decision_function(X)

df["anomaly"] = scores
df["score"] = -decision
sus = df[df["anomaly"]==-1].sort_values("score", ascending=False)

print("\nTop suspicious networked processes:\n")
for _, r in sus.head(10).iterrows():
    user = "root" if r["uid"]==0 else str(r["uid"])
    print(f"[score={r['score']:.3f}] pid={int(r['pid'])} user={user} conn={int(r['conn_count'])} uniq_ips={int(r['uniq_rips'])} med_port={int(r['median_rport'])} name={r['name']} cmd={r['cmd']}")
print("\nTip: Investigate unexpected root-owned outbound connections, rare ports, and short-lived utilities calling out to the internet.")
PY

Why this works: Malware and hands-on-keyboard activity leave telltale network traces. Even without threat intel, processes that connect in unusual ways (or with unusual volume) bubble to the top.

Workflow 3: Cluster noisy system logs to reveal “what happened”

Goal: Summarize the last 24 hours of logs into thematic clusters so you can quickly see major activities, errors, and outliers.

Step 1 — export last day of logs:

sudo journalctl --since "24 hours ago" -o short-iso > syslog-24h.log

Step 2 — cluster and summarize with TF‑IDF + KMeans:

python3 - <<'PY'
import re, random
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.cluster import MiniBatchKMeans

# Load logs
with open("syslog-24h.log","r",errors="ignore") as f:
    lines = [l.strip() for l in f if l.strip()]

if not lines:
    print("No logs found in syslog-24h.log")
    raise SystemExit

# Remove leading timestamps to focus on content
def strip_ts(s):
    return re.sub(r'^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d+)?\s+\S+\s+','',s)

docs = [strip_ts(l) for l in lines]

# Vectorize and cluster
k = 8 if len(docs) >= 800 else max(3, len(docs)//100 or 3)
vec = TfidfVectorizer(
    lowercase=True, 
    token_pattern=r'[a-zA-Z0-9_.:/-]{3,}',
    max_features=20000,
    ngram_range=(1,2)
)
X = vec.fit_transform(docs)
km = MiniBatchKMeans(n_clusters=k, random_state=42, batch_size=1024, n_init='auto')
labels = km.fit_predict(X)

terms = vec.get_feature_names_out()
order_centroids = km.cluster_centers_.argsort()[:, ::-1]

print(f"\nClustered {len(docs)} log lines into {k} groups:\n")
for i in range(k):
    top_terms = [terms[ind] for ind in order_centroids[i, :8]]
    idxs = [j for j,l in enumerate(labels) if l==i]
    sample = [docs[j] for j in random.sample(idxs, min(3, len(idxs)))]
    print(f"\n=== Cluster {i} ===")
    print("Top terms:", ", ".join(top_terms))
    print("Examples:")
    for s in sample:
        print("-", s[:200])
print("\nTip: Use clusters to identify repeated errors, service restarts, authentication bursts, and unusual kernel messages.")
PY

Why this works: TF‑IDF emphasizes “what’s unique,” while KMeans groups similar messages together. You get a quick map of the day’s activity without reading thousands of lines.

Real-world tips for defensible AI-assisted forensics

  • Preserve first, analyze second: Make read-only copies where possible. Record commands with script -a session.log.

  • Baseline: Save today’s model outputs and compare to tomorrow’s. Stable hosts should produce stable metrics.

  • Tune contamination: Isolation Forest’s sensitivity is adjustable. If you see too many false positives, raise the expected inlier fraction or engineer richer features.

  • Correlate: Anomalies are leads, not verdicts. Confirm with packet captures, binary hashes, and known-good inventories.

Automate it

Wrap each workflow in a simple runner script and schedule with cron/systemd. Example skeleton:

#!/usr/bin/env bash
set -euo pipefail

OUTDIR="$HOME/ai-forensics/$(date +%F-%H%M%S)"
mkdir -p "$OUTDIR"

# Collect artifacts
sudo journalctl --since "7 days ago" -t sshd -o short-iso > "$OUTDIR/sshd.log"
sudo journalctl --since "24 hours ago" -o short-iso > "$OUTDIR/syslog-24h.log"
sudo ss -H -4 -tunp > "$OUTDIR/sockets.txt"

# Run analyses (assuming the Python one-liners are saved as scripts)
python3 ssh_anomaly.py "$OUTDIR/sshd.log" > "$OUTDIR/ssh_report.txt" || true
python3 net_outliers.py "$OUTDIR/sockets.txt" > "$OUTDIR/net_report.txt" || true
python3 log_clusters.py "$OUTDIR/syslog-24h.log" > "$OUTDIR/log_clusters.txt" || true

echo "Reports in: $OUTDIR"

Conclusion and next steps

AI won’t investigate for you, but it will shine a light where you should look first. With a handful of Bash-friendly commands and small Python models, you can:

  • Flag suspicious SSH sources

  • Surface odd networked processes

  • Summarize noisy logs into actionable themes

Your next step: run the three workflows on a lab box today, tune them to your environment, and then automate them on a staging or low-risk fleet node. From there, baseline, iterate, and integrate the outputs into your existing incident response playbooks.

If you want a follow-up post with ready-to-run scripts in a git repo and systemd timers, say the word.

Related posts