Posted on
• Artificial Intelligence

Summarising Linux Reports with Artificial Intelligence

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Summarising Linux Reports with Artificial Intelligence

If you’ve ever stared at a 20,000-line journalctl dump at 3 a.m., you know the pain: the signal is there, but buried under a mountain of noise. What if you could ask a Linux-friendly AI to read the noise, surface the incidents, and propose next steps, all from your terminal?

This article shows you practical, privacy-aware ways to summarise Linux reports with AI—locally (no data leaves your box) or via a cloud API—plus the Bash glue to make it reliable.

Why this matters

  • Linux emits a lot of textual data: logs (journalctl, dmesg), performance reports (sar), inventories (df -h), and more. Manually triaging this costs time and attention.

  • AI summarisation can compress multi-megabyte reports into an actionable brief (key incidents, probable causes, and remediation steps).

  • Done carefully, this reduces MTTR, helps with postmortems, and eases handoffs—without replacing your judgment or the need to verify.

Tools you’ll need

We’ll use common CLI tools for pre-filtering and redaction, plus two summarisation options:

  • Local/offline: Ollama with an open model (e.g., Llama 3 8B)

  • Cloud API: OpenAI via curl (bring your API key)

Install the basics (curl, jq, ripgrep, sysstat) with your package manager:

  • apt (Debian/Ubuntu and derivatives):
sudo apt update
sudo apt install -y curl jq ripgrep sysstat
  • dnf (Fedora/RHEL/CentOS Stream):
sudo dnf install -y curl jq ripgrep sysstat
  • zypper (openSUSE/SLE):
sudo zypper refresh
sudo zypper install -y curl jq ripgrep sysstat

Optional: enable sysstat collection so sar has data:

sudo systemctl enable --now sysstat

Local AI (Ollama) install:

curl -fsSL https://ollama.com/install.sh | sh
# Example model
ollama pull llama3:8b

Cloud API (OpenAI):

  • Set your API key:
export OPENAI_API_KEY="sk-..."

Note: Keep sensitive logs on-prem. Prefer the local model path if privacy or compliance is a concern.

Step 1: Collect and normalise the reports

Create a working directory:

mkdir -p ~/ai-summaries/{logs,summaries}
cd ~/ai-summaries

Examples of useful report collections:

  • System logs (last 6 hours):
journalctl --since "6 hours ago" --no-pager > logs/journal_6h.log
  • Kernel ring buffer:
dmesg -T > logs/dmesg.log
  • Disk inventory:
df -hT > logs/df.log
  • Performance (CPU/memory, last 8 hours; requires sysstat history):
sar -u -r -S -n DEV -s 08:00:00 -e now > logs/sar_8h.log

Optional: Focus logs on likely-problem lines first (failures, OOMs, panics, segfaults):

rg -i -N -e 'fail|error|panic|oom|segfault|critical|timeout' logs/journal_6h.log > logs/journal_6h_errors.log

Real-world example:

  • On a noisy web node, journalctl often contains flapping units, OOM kills, and SSH auth spam. The ripgrep filter immediately shrinks the haystack, and AI can summarise patterns (e.g., “nginx restarted 7 times; suspect low file descriptors”).

Step 2: Redact and chunk before sending to AI

Redact obvious PII like emails and IPv4 addresses:

sed -E '
  s/([0-9]{1,3}\.){3}[0-9]{1,3}/<IP>/g;
  s/[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}/<EMAIL>/g
' logs/journal_6h_errors.log > logs/journal_6h_errors.redacted.log

Split large files into AI-friendly chunks (about 100 KB each):

split -b 100k -d -a 3 logs/journal_6h_errors.redacted.log logs/journal_6h_errors.part.

Tip: Favor smaller chunks for better grounding and lower token usage. We’ll later consolidate partial summaries into a final report.

Step 3: Summarise locally with Ollama (offline)

Use an open model like Llama 3 8B:

Install/pull (if you haven’t):

curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3:8b

Bash script to summarise each chunk, then consolidate:

Create summarize_local.sh:

#!/usr/bin/env bash
set -euo pipefail

MODEL="${MODEL:-llama3:8b}"
INPUT_GLOB="${1:-logs/journal_6h_errors.part.*}"
OUT_DIR="summaries"
PARTIAL="$OUT_DIR/journal_6h.partial.md"
FINAL="$OUT_DIR/journal_6h.final.md"

mkdir -p "$OUT_DIR"
: > "$PARTIAL"

for f in $INPUT_GLOB; do
  echo "Summarising $f ..."
  PROMPT=$(cat <<'EOF'
You are a Linux SRE assistant. Summarise the following logs.
Output sections:

- TL;DR (1–3 sentences)

- Key incidents (timestamp, unit/component, short message)

- Probable causes (with evidence)

- Action items (shell commands or config changes)
Be concise and accurate. Do not invent data not present.
EOF
)
  CONTENT=$(cat "$f")
  ollama run "$MODEL" -p "$PROMPT

<log>
$CONTENT
</log>
" >> "$PARTIAL"
  echo -e "\n---\n" >> "$PARTIAL"
done

echo "Consolidating partials into final report..."
CONSOLIDATE=$(cat <<'EOF'
Combine the following partial summaries into a single concise report with the same sections:

- TL;DR

- Key incidents

- Probable causes

- Action items
Merge duplicates, keep timestamps and units when available, and prioritise recurring issues and severity.
EOF
)

ollama run "$MODEL" -p "$CONSOLIDATE

<partials>
$(cat "$PARTIAL")
</partials>
" > "$FINAL"

echo "Done. See $FINAL"

Run it:

chmod +x summarize_local.sh
./summarize_local.sh

Real-world example:

  • Frequent “Out of memory: Kill process … java” lines across chunks will roll up into “Repeated OOM kills affecting Java service; increase cgroup memory limit or tune heap; add swap or reduce concurrency.”

Step 4: Summarise via OpenAI (cloud API)

This is useful if you prefer hosted models. Requires an API key:

export OPENAI_API_KEY="sk-..."

Script summarize_api.sh:

#!/usr/bin/env bash
set -euo pipefail

MODEL="${MODEL:-gpt-4o-mini}"
INPUT_GLOB="${1:-logs/journal_6h_errors.part.*}"
OUT_DIR="summaries"
PARTIAL="$OUT_DIR/journal_6h.api.partial.md"
FINAL="$OUT_DIR/journal_6h.api.final.md"

mkdir -p "$OUT_DIR"
: > "$PARTIAL"

for f in $INPUT_GLOB; do
  echo "Summarising $f via OpenAI ..."
  CONTENT=$(jq -Rs . < "$f")
  DATA=$(jq -n \
    --arg content "Summarise the following Linux logs. Return sections: TL;DR (1–3 sentences), Key incidents (timestamp/unit/message), Probable causes (with evidence), Action items (commands/config). Be precise and avoid fabrications.\n\n$(cat "$f")" \
    '{model:$ENV.MODEL, temperature:0.2, messages:[
      {role:"system", content:"You are a Linux SRE assistant."},
      {role:"user", content:$content}
    ]}'
  )

  curl -sS https://api.openai.com/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer '"$OPENAI_API_KEY"'" \
    -d "$DATA" | jq -r '.choices[0].message.content' >> "$PARTIAL"

  echo -e "\n---\n" >> "$PARTIAL"
done

# Consolidate partial summaries
DATA_FINAL=$(jq -n \
  --arg parts "$(cat "$PARTIAL")" \
  '{model:$ENV.MODEL, temperature:0.2, messages:[
    {role:"system", content:"You are a Linux SRE assistant."},
    {role:"user", content:("Combine the following partial summaries into a single concise report with sections: TL;DR, Key incidents, Probable causes, Action items. Merge duplicates; preserve timestamps and units where possible.\n\n" + $parts)}
  ]}'
)

curl -sS https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer '"$OPENAI_API_KEY"'" \
  -d "$DATA_FINAL" | jq -r '.choices[0].message.content' > "$FINAL"

echo "Done. See $FINAL"

Run it:

chmod +x summarize_api.sh
./summarize_api.sh

Note:

  • You’re sending content to a third party; ensure redaction and compliance.

  • Control cost/latency by chunking and keeping temperature low.

Step 5: Automate and iterate

  • Cron, hourly:
crontab -e
# Top of the hour: collect, filter, redact, chunk, summarise locally
0 * * * * cd /home/you/ai-summaries && journalctl --since "1 hour ago" --no-pager > logs/journal_1h.log && rg -i -N -e 'fail|error|panic|oom|segfault|critical|timeout' logs/journal_1h.log | sed -E 's/([0-9]{1,3}\.){3}[0-9]{1,3}/<IP>/g; s/[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}/<EMAIL>/g' > logs/journal_1h.redacted.log && split -b 100k -d -a 3 logs/journal_1h.redacted.log logs/journal_1h.part. && ./summarize_local.sh > /var/log/ai-summaries.log 2>&1

  • Save final summaries somewhere durable (git repo, wiki, artifact store), or notify via your preferred channel.

  • Improve prompts over time: ask for specific sections you care about (e.g., “Top 5 recurring errors with counts” or “Suspected misconfigurations and files to inspect”).

Practical patterns and examples

  • Incident triage: “Show me services that restarted more than 3 times and why.” Filter with ripgrep, summarise with AI, then run suggested systemctl status or journalctl -u commands.

  • Capacity trending: Summarise sar -r and sar -u outputs to spot time windows with sustained pressure; AI points you to “memory pressure 14:00–16:00, swapin spikes.”

  • Disk pressure: Collate df -hT across nodes; AI highlights filesystems <10% free, proposes cleanup paths or LVM extension steps.

Guardrails and good habits

  • Redact first. Err on the side of removing secrets, tokens, emails, IPs, and hostnames if needed.

  • Verify key claims. Treat AI outputs as a briefing; follow up with the suggested commands.

  • Keep prompts and outputs in source control; they make excellent postmortem artifacts.

  • Prefer local models for sensitive environments; tune model size vs. speed on your hardware.

Conclusion and next steps

AI can turn a haystack of Linux logs into a focused action plan in minutes—without changing your tooling or workflow. Start simple: 1) Collect last 6 hours of logs 2) Redact and chunk 3) Run the local or cloud summariser 4) Review, verify, and act

Try the provided scripts on a non-production box today. Once you’re comfortable, automate the pipeline, commit the summaries to your team’s knowledge base, and iterate on the prompts that best fit your environment.

Related posts