Posted on
• Artificial Intelligence

Artificial Intelligence Linux Reporting

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

AI-Powered Linux Reporting with Bash: Turn Noisy Logs Into Clear Morning Briefs

If your day starts with tailing logs, skimming sar graphs, and skirting pager anxiety, you’re not alone. Linux gives us oceans of telemetry, but turning it into concise, actionable health reports takes time you don’t have. What if Bash could collect the right signals and an AI model turned them into a short, prioritized brief—every morning, on schedule?

This post shows how to build an “AI reporting” pipeline with standard Linux tools and a ChatGPT-compatible endpoint. You’ll get:

  • A Bash-first snapshot of key system signals

  • A prompt that guides an AI model to summarize risks and remediation steps

  • Automation via cron/systemd and optional chat delivery

All with minimal moving parts and easy package-manager installs.

Why AI for Linux reporting?

  • Signal overload: Journald, /proc, sar, ps and friends can overwhelm during incidents.

  • Summarization + prioritization: LLMs excel at condensing sprawling data into bite-sized findings and next steps.

  • Continuous hygiene: A daily “morning brief” surfaces issues before they become outages, improving MTTP and team awareness.

What you’ll build

  • A Bash script that:
    • Gathers OS, CPU, memory, disk, network, top processes, and recent critical logs
    • Optionally uses sar (sysstat) if present
    • Constructs a strict, safe prompt for an AI model
    • Calls a ChatGPT-compatible API via curl + jq
    • Writes a neat Markdown report and (optionally) posts it to Slack

Prerequisites (install with your package manager)

  • curl

  • jq

  • sysstat (for sar, optional but recommended)

  • cron service (or use systemd timers)

Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y curl jq sysstat cron
# Enable sysstat collection
sudo sed -i 's/ENABLED="false"/ENABLED="true"/' /etc/default/sysstat || true
sudo systemctl enable --now sysstat
sudo systemctl enable --now cron

RHEL/CentOS/Fedora (dnf):

sudo dnf install -y curl jq sysstat cronie
sudo systemctl enable --now sysstat
sudo systemctl enable --now crond

SUSE (zypper):

sudo zypper refresh
sudo zypper install -y curl jq sysstat cron
sudo systemctl enable --now sysstat
sudo systemctl enable --now cron

Note: journalctl requires systemd. You may need sudo to access all logs.

Step 1: Create the snapshot and prompt script

Save as /usr/local/bin/ai-report.sh and make it executable.

#!/usr/bin/env bash
set -euo pipefail

# Configuration
MODEL="${MODEL:-gpt-4o-mini}"                           # Any OpenAI-compatible model name
BASE_URL="${OPENAI_BASE_URL:-https://api.openai.com/v1}"# Or your self-hosted / local endpoint
API_KEY="${OPENAI_API_KEY:-}"                           # export OPENAI_API_KEY=...
SLACK_WEBHOOK_URL="${SLACK_WEBHOOK_URL:-}"              # Optional

if [[ -z "${API_KEY}" ]]; then
  echo "ERROR: Set OPENAI_API_KEY (and optionally OPENAI_BASE_URL) before running." >&2
  exit 1
fi

# Paths
TMPDIR="${TMPDIR:-/tmp}"
STAMP="$(date -u +"%Y-%m-%dT%H:%M:%SZ")"
HOST="$(hostname -f 2>/dev/null || hostname)"
SNAP="$TMPDIR/linux_snapshot_${HOST}_$STAMP.txt"
PROMPT="$TMPDIR/ai_prompt_${HOST}_$STAMP.txt"
REPORT="$TMPDIR/ai_report_${HOST}_$STAMP.md"

# Basic redaction for safety (minimize sensitive leakage)
redact() {
  sed -E \
    -e 's/([Aa]uthorization: )[^\r\n ]+/\1REDACTED/g' \
    -e 's/([Pp]assword=)[^ &]+/\1REDACTED/g' \
    -e 's/([Tt]oken=)[^ &]+/\1REDACTED/g' \
    -e 's/([Kk]ey=)[^ &]+/\1REDACTED/g'
}

section() { echo -e "\n==== $1 ====\n" >> "$SNAP"; }

: > "$SNAP"

section "Context"
{
  echo "Host: $HOST"
  echo "Date (UTC): $STAMP"
  echo "Kernel: $(uname -srmo)"
} >> "$SNAP"

section "Uptime and Load"
{
  uptime || true
  cat /proc/loadavg 2>/dev/null || true
} >> "$SNAP"

section "CPU"
{
  grep -m1 -E 'model name' /proc/cpuinfo 2>/dev/null || true
  nproc 2>/dev/null || true
} >> "$SNAP"

section "Memory"
{
  free -h || true
  vmstat 1 2 2>/dev/null | tail -n 1 || true
} >> "$SNAP"

section "Disk"
{
  df -h -x tmpfs -x devtmpfs || true
  lsblk -o NAME,FSTYPE,SIZE,TYPE,MOUNTPOINT -e 7 2>/dev/null || true
} >> "$SNAP"

section "Top processes"
{
  ps -eo pid,ppid,comm,%cpu,%mem --sort=-%cpu | head -n 12
} >> "$SNAP"

section "Network"
{
  ss -s 2>/dev/null || true
  ip -brief addr 2>/dev/null || true
} >> "$SNAP"

section "Recent critical logs (last 2h)"
{
  journalctl -p 3 -S -2h --no-pager -n 200 2>/dev/null | redact || echo "No permission or no critical logs."
} >> "$SNAP"

section "sar (if available)"
{
  if command -v sar >/dev/null 2>&1; then
    sar -q 1 1 2>/dev/null || true
    sar -r 1 1 2>/dev/null || true
    sar -n DEV 1 1 2>/dev/null || true
  else
    echo "sysstat/sar not installed"
  fi
} >> "$SNAP"

# Build the analysis prompt
cat > "$PROMPT" <<'P_END'
Write a concise Linux system health report for SREs:

- Summarize notable findings and risks.

- Call out CPU/mem/disk saturation, I/O, network anomalies, and recent errors.

- Suggest the top 3 remediation actions with concrete commands.

- Include only what the data supports. Use bullet points, keep it under ~3000 characters.
P_END

cat "$SNAP" >> "$PROMPT"

# Prepare JSON and call model
USER_CONTENT_JSON="$(jq -Rs . < "$PROMPT")"
read -r -d '' DATA <<JSON || true
{"model":"$MODEL","temperature":0.2,
 "messages":[
   {"role":"system","content":"You are a Linux SRE assistant producing clear, actionable health reports."},
   {"role":"user","content":$USER_CONTENT_JSON}
 ]}
JSON

RESPONSE="$(curl -sS "$BASE_URL/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $API_KEY" \
  -d "$DATA")"

# Extract the content
jq -r '.choices[0].message.content' <<<"$RESPONSE" > "$REPORT" || {
  echo "Model response parsing failed. Raw response:" >&2
  echo "$RESPONSE" >&2
  exit 1
}

echo "AI report written to: $REPORT"
echo
cat "$REPORT"

# Optional: Post to Slack
if [[ -n "$SLACK_WEBHOOK_URL" ]]; then
  jq -n --arg text "$(cat "$REPORT")" '{text:$text}' \
  | curl -sS -X POST -H 'Content-type: application/json' --data @- "$SLACK_WEBHOOK_URL" >/dev/null \
  && echo "Posted report to Slack."
fi

Make it executable:

sudo install -m 0755 ai-report.sh /usr/local/bin/ai-report.sh

Export your API variables (replace with your values). Supports any OpenAI-compatible endpoint.

export OPENAI_API_KEY="sk-************************"
# Optional: point to your own compatible endpoint (e.g., local gateway)
# export OPENAI_BASE_URL="http://localhost:11434/v1"
# Optional: pick a model
# export MODEL="gpt-4o-mini"

Run it:

/usr/local/bin/ai-report.sh

It will print a Markdown report and save it under /tmp/ai_report_<host>_<timestamp>.md.

Step 2: Automate it (cron or systemd)

cron (Debian/Ubuntu/SUSE):

# Ensure cron is running (see install section above)
crontab -l 2>/dev/null; echo "0 7 * * * OPENAI_API_KEY=$OPENAI_API_KEY /usr/local/bin/ai-report.sh >> /var/log/ai-report.log 2>&1" | crontab -

cronie (RHEL/Fedora):

# Ensure crond is running (see install section above)
crontab -l 2>/dev/null; echo "0 7 * * * OPENAI_API_KEY=$OPENAI_API_KEY /usr/local/bin/ai-report.sh >> /var/log/ai-report.log 2>&1" | crontab -

Prefer systemd timers? Example:

# /etc/systemd/system/ai-report.service
[Unit]
Description=AI Linux Health Report

[Service]
Type=oneshot
Environment=OPENAI_API_KEY=YOUR_KEY_HERE
ExecStart=/usr/local/bin/ai-report.sh

# /etc/systemd/system/ai-report.timer
[Unit]
Description=Run AI Linux Health Report daily at 07:00

[Timer]
OnCalendar=*-*-* 07:00:00
Persistent=true

[Install]
WantedBy=timers.target

Enable:

sudo systemctl daemon-reload
sudo systemctl enable --now ai-report.timer

Step 3: Keep cost, privacy, and noise in check

  • Minimize tokens:

    • Trim logs by time: journalctl -S -2h -n 200
    • Focused metrics over raw dumps; keep sections short

  • Redact secrets:

    • The script includes a simple redact filter; adapt it to your environment and logging patterns

  • Principle of least privilege:

    • If you don’t need full logs, remove the journald section or run as a least-privileged user

  • Local endpoints:

    • If you run a self-hosted, OpenAI-compatible gateway, set OPENAI_BASE_URL to keep data in-network

Step 4: Example output (what “good” looks like)

Linux Health Report — host: db01.prod

Key Observations

- CPU load within capacity (1m/5m/15m: 0.42/0.36/0.31 on 8 cores).

- Memory pressure low; swap activity negligible.

- Disk: /var at 84% used — approaching threshold.

- 3 critical log entries in last 2h: repeated auth failures from 10.2.3.45; systemd unit flapped (nginx); rsyslog rotation completed.

Top Remediations

- Free disk on /var:
  sudo journalctl --vacuum-size=500M
  sudo du -sh /var/log/* | sort -h | tail

- Harden SSH against brute-force:
  sudo fail2ban-client status sshd
  sudo firewall-cmd --add-rich-rule='rule family=ipv4 source address=10.2.3.45 drop' --permanent && sudo firewall-cmd --reload

- Stabilize nginx unit:
  sudo systemctl status nginx --no-pager
  sudo tail -n 200 /var/log/nginx/error.log

Step 5: Extend it when you’re ready

  • Add application-specific checks (e.g., database status, queue depth)

  • Post reports to Slack/Teams (webhook) or create Git commits for historical diffs

  • Track a simple health score and alert when it crosses a threshold

  • Split into “daily” vs “incident” modes with different prompt instructions

Troubleshooting

  • No sar output: Ensure sysstat is enabled and collecting (see install section).

  • Permission errors on journalctl: Run with sudo or drop that section.

  • Empty AI response: Check OPENAI_API_KEY, BASE_URL, network egress, and model name. Inspect raw JSON in the terminal error output.

Conclusion and next step

You now have a Bash-native pipeline that converts Linux telemetry into an AI-curated brief—reducing toil and surfacing risks before they become incidents. Start by running the script manually on one host, then:

  • Enable the daily timer/cron

  • Wire a Slack webhook for team visibility

  • Iterate on the prompt and sections to fit your stack

When you’re ready, roll it across your fleet. Your future self (and your on-call rotation) will thank you.

Related posts