Posted on
• Artificial Intelligence

Artificial Intelligence Linux Security Audits

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence Linux Security Audits: A Bash-First Playbook

Your Linux hosts are already telling you what’s wrong—through auditd, journald, file-integrity checks, and hardening scanners. The problem: humans can’t read it all fast enough. AI can help surface signal from the noise, prioritize fixes, and spot anomalies humans miss. This post shows you a practical, Bash-centric workflow to combine classic Linux audit tools with AI-driven insights—without handing your crown jewels to the cloud.

What you’ll get:

  • A quick, repeatable baseline audit using Lynis and OpenSCAP

  • Actionable telemetry from auditd and AIDE

  • A safe, AI-ready data pipeline (JSON + redaction)

  • Lightweight anomaly detection you can run locally

  • Optional LLM summarization for remediation guidance

Note: All code runs on the host you’re auditing. Use sudo or root where required.

Why AI for Linux security audits?

  • Volume: A single busy host emits thousands of events per minute. AI helps triage at scale.

  • Variance: Attackers live off the land. The weird combination of an otherwise “normal” command run by an unusual user at an odd hour is exactly what anomaly detection catches.

  • Velocity: AI can prioritize remediation by impact, not alphabetical order, so you fix what matters first.

Prerequisites: Install the core auditing toolchain

The following packages are in major distro repos. If a package name differs in your environment, use your package search (apt search, dnf search, zypper se).

Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y lynis openscap-scanner scap-security-guide aide auditd jq python3 python3-pip

Fedora / RHEL / CentOS Stream (dnf):

# On RHEL/CentOS Stream, enable EPEL if needed for some content
# sudo dnf install -y epel-release

sudo dnf install -y lynis openscap-scanner scap-security-guide aide audit jq python3 python3-pip

openSUSE Leap/Tumbleweed (zypper):

sudo zypper refresh
sudo zypper install -y lynis openscap scap-security-guide aide audit jq python3-pip

Optional (for local LLM summarization via Ollama):

  • Install instructions: https://ollama.com/download

  • Example after install: ollama pull mistral

1) Run a hardened baseline with Lynis and OpenSCAP

Create an initial, machine-readable posture snapshot to compare against later.

Lynis:

sudo lynis audit system --report-file /var/log/lynis-report.dat
# Quick view of top suggestions:
grep "warning\|suggestion" -i /var/log/lynis-report.dat | sed 's/^\s\+//'

OpenSCAP (auto-discover the data stream file and use a general profile):

DS_FILE=$(find /usr/share/xml/scap -name '*-ds.xml' | head -n1)
echo "Using SCAP data stream: $DS_FILE"
sudo oscap xccdf eval \
  --profile xccdf_org.ssgproject.content_profile_standard \
  --report /var/log/openscap-report.html \
  "$DS_FILE"
echo "Report: /var/log/openscap-report.html"

Why both? Lynis gives broad, OS-aware checks and tips; OpenSCAP provides standards-based benchmarking (CIS/STIG/Standard profile depending on content).

2) Turn on rich telemetry: auditd, journald, and AIDE

You can’t detect what you don’t log. Enable core sources that AI can learn from.

Enable and start auditd:

sudo systemctl enable --now auditd
sudo auditctl -s   # verify it’s running

Add high-signal audit rules (examples: sudoers changes and process execution). Create /etc/audit/rules.d/10-highsignal.rules:

-w /etc/sudoers -p wa -k sudoers
-w /etc/sudoers.d -p wa -k sudoers

# Track process execution (64-bit and 32-bit)
-a always,exit -F arch=b64 -S execve,execveat -k exec
-a always,exit -F arch=b32 -S execve,execveat -k exec

Load rules:

sudo augenrules --load

Initialize AIDE (file integrity):

sudo aideinit
sudo mv /var/lib/aide/aide.db.new /var/lib/aide/aide.db
# Later (cron it daily):
sudo aide --check | tee /var/log/aide-check.log

Journald already collects system events; we’ll query it as JSON next.

3) Build an AI-ready, privacy-aware data pipeline

Normalize your logs to JSON and redact sensitive data before any AI analysis or sharing.

Export last 24 hours of journal and audit logs to JSON:

sudo journalctl -o json --since "24 hours ago" > /var/log/journal-24h.json

# Audit events via systemd-journald (transport=audit)
sudo journalctl _TRANSPORT=audit -o json --since "24 hours ago" > /var/log/audit-24h.json

Redact hostnames, IPs, and some identifiers (tune as needed):

jq '
  del(.hostname?, ._HOSTNAME?, .SYSLOG_IDENTIFIER?, ._MACHINE_ID?) |
  .MESSAGE |= gsub("([0-9]{1,3}\\.){3}[0-9]{1,3}"; "<IP>") |
  .MESSAGE |= gsub("([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}"; "<MAC>")
' /var/log/audit-24h.json > /var/log/audit-24h.redacted.json

Tip:

  • Keep raw logs local; only process redacted data with external tools.

  • Use project- or enclave-specific salts if you hash identifiers.

4) Detect anomalies with a lightweight model (Isolation Forest)

We’ll turn audit events into categorical features (exe, uid, hour) and use Isolation Forest to surface outliers. This is fast, local, and works on a single host.

Install Python libs:

python3 -m pip install --user pandas scikit-learn

Run the detector on redacted audit JSON:

python3 - <<'PY'
import json, sys, re, math
from datetime import datetime, timezone
import pandas as pd
from sklearn.ensemble import IsolationForest
from sklearn.feature_extraction import FeatureHasher

src = "/var/log/audit-24h.redacted.json"
rows = []
kv_rx = re.compile(r'(\w+)=(".*?"|\S+)')

def parse_kv(msg):
    d={}
    for k,v in kv_rx.findall(msg):
        d[k]=v.strip('"')
    return d

with open(src, 'r', errors='ignore') as f:
    for line in f:
        try:
            j = json.loads(line)
        except Exception:
            continue
        msg = j.get("MESSAGE","")
        if not msg or "type=SYSCALL" not in msg:  # focus on exec/syscalls
            continue
        kv = parse_kv(msg)
        exe = kv.get("exe") or kv.get("comm") or "unknown"
        uid = kv.get("uid") or "?"
        auid = kv.get("auid") or "?"
        ts_us = int(j.get("__REALTIME_TIMESTAMP","0")) if j.get("__REALTIME_TIMESTAMP") else 0
        if ts_us:
            ts = datetime.fromtimestamp(ts_us/1_000_000, tz=timezone.utc)
            hour = ts.hour
        else:
            hour = -1
        rows.append({"exe":exe, "uid":uid, "auid":auid, "hour":str(hour), "raw":msg[:500]})

if not rows:
    print("No syscall/audit events parsed. Ensure auditd is logging and journald captured audit transport.")
    sys.exit(0)

df = pd.DataFrame(rows)
tokens = []
for _,r in df.iterrows():
    toks = [f"exe={r['exe']}", f"uid={r['uid']}", f"auid={r['auid']}", f"hour={r['hour']}"]
    tokens.append(toks)

hasher = FeatureHasher(n_features=128, input_type='string')  # small, fast
X = hasher.transform(tokens)

iso = IsolationForest(n_estimators=200, contamination='auto', random_state=42)
iso.fit(X)
scores = -iso.score_samples(X)  # higher = more anomalous
df["_anomaly"] = scores

top = df.sort_values("_anomaly", ascending=False).head(15)
print("Top suspicious (exe, uid, auid, hour, score):")
for _,r in top.iterrows():
    print(f"{r['exe']:<30} uid={r['uid']:<6} auid={r['auid']:<6} hour={r['hour']:<2} score={r['_anomaly']:.3f}")
    print(f"  raw: {r['raw']}")
PY

What to look for:

  • Rare executables (e.g., binaries from /tmp or /dev/shm)

  • Unusual users running high-privilege commands

  • Off-hours spikes for specific executables

Real-world style example:

  • An outlier shows exe=/bin/nc run by a service account at 03:14 UTC. That combination (exe + user + time) was never seen in the last 24 hours—worth immediate review.

5) Summarize and prioritize with an LLM (optional)

Use an LLM to distill long audit outputs into prioritized actions. Keep it local if possible or redact thoroughly before using a cloud API.

Summarize Lynis findings with a local model via Ollama:

# Ensure you've installed Ollama and pulled a model, e.g. mistral
# ollama pull mistral

PROMPT="$(printf 'Summarize and prioritize fixes for these Lynis findings. Group by risk, cite items, and suggest concrete, distro-appropriate remediations:\n\n%s' \
"$(sudo sed -n '1,400p' /var/log/lynis-report.dat)")"
ollama run mistral -p "$PROMPT"

Tips:

  • Chunk long files (e.g., first 400 lines, then next 400) and ask for a final combined summary.

  • Never feed secrets, private keys, or unredacted logs to external services.

  • Keep a text copy of the AI’s remediation plan alongside the original reports for audits.

Putting it together: A daily, safe AI-assisted audit loop

  • Nightly

    • Run Lynis and OpenSCAP; archive reports
    • Rotate AIDE checks
    • Export the last 24h of journald/auditd JSON
    • Redact identifiers
    • Run the anomaly detector; alert on the top N

  • Weekly

    • Use an LLM to summarize drift and remediation priorities
    • Track “time-to-fix” on repeated findings

  • Monthly

    • Tighten audit rules based on what produced the best signal
    • Update models and re-baseline

Example crontab entry (root) for daily export/anomaly detection:

# m h dom mon dow command
15 1 * * * /usr/bin/journalctl _TRANSPORT=audit -o json --since "24 hours ago" > /var/log/audit-24h.json && \
  /usr/bin/jq 'del(.hostname?, ._HOSTNAME?, .SYSLOG_IDENTIFIER?, ._MACHINE_ID?) | .MESSAGE |= gsub("([0-9]{1,3}\\.){3}[0-9]{1,3}"; "<IP>")' \
  /var/log/audit-24h.json > /var/log/audit-24h.redacted.json && \
  /usr/bin/python3 /root/bin/audit_iforest.py > /var/log/audit-anomalies.txt

Common pitfalls and how to avoid them

  • Data leakage: Always redact and minimize data before AI analysis. Keep raw logs offline.

  • Thin signals: If anomaly detection returns “nothing interesting,” expand features (e.g., add tty, success/failure, parent PID) or extend the lookback window.

  • Tool drift: After kernel or policy updates, re-check that audit rules are loaded and SCAP profiles match your OS version.

  • Vendor lock-in: Prefer local or open tools for triage; if you must use a cloud LLM, wrap it behind strict redaction and approval gates.

Conclusion and next steps (CTA)

AI won’t replace your Linux hardening tools—it makes them more effective. Start small: 1) Install the toolchain (Lynis, OpenSCAP, auditd, AIDE, jq, Python).
2) Generate a baseline and fix the top 5 issues.
3) Turn on the JSON+redaction pipeline.
4) Run the anomaly detector and investigate the top outliers.
5) Optionally, use a local LLM for remediation summaries.

Want a ready-to-run kit? Package the commands above into a repo with:

  • scripts/baseline.sh (Lynis + OpenSCAP)

  • scripts/export_redact.sh (journal/audit JSON + jq)

  • scripts/anomaly.py (Isolation Forest)

  • docs/playbooks.md (per-finding fixes)

Then run it nightly. Your future incident-retrospective self will thank you.

Related posts