Posted on
• Artificial Intelligence

Artificial Intelligence SLA Reporting

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence SLA Reporting with Bash: From Logs to Guarantees

Your AI features are fast—until they aren’t. A spike in 429s, a provider hiccup, or a 2x latency regression can turn “wow” into “whoa” for your users. If you ship AI to production, you need Service Level Agreements (SLAs) you can actually measure, report, and act on—without waiting for a vendor dashboard.

This post shows how to build practical, vendor-agnostic AI SLA reporting using nothing but Linux, Bash, and a few standard CLI tools. You’ll define AI-specific SLIs/SLOs, instrument logs, generate daily reports, and automate them via cron or systemd. Everything runs locally and integrates with your existing ops toolchain.

What problem are we solving?

  • AI systems introduce new failure modes: rate limits, model saturation, token-based timeouts, and vendor-side safety blocks.

  • Traditional web SLAs (availability, error rate, latency) still apply, but need AI-aware metrics (token usage, rate-limit pressure, policy violation flags).

  • Vendor dashboards are useful, but you still need your own ground truth tied to your traffic, regions, and models.

This approach gives you:

  • A reproducible, version-controlled reporting pipeline

  • Independence from any single provider’s UI or definitions

  • Extensible metrics that track what matters to your users

Validity: Why measure AI SLAs this way?

  • It’s the same proven practice SRE teams use for web services (SLIs/SLOs/error budgets), adapted for AI-specific signals.

  • Logs are your most portable, long-term artifact. They let you reconstruct incidents and drill down by model, region, or customer.

  • Bash + jq pipelines are transparent and reviewable. No black boxes—just text processing you can audit and improve.

Prerequisites (Linux)

We’ll use these tools:

  • curl (HTTP), jq (JSON), gawk (awk), bc (math), mailx (optional email)

Install them with your package manager:

  • Debian/Ubuntu (apt):

    sudo apt update
sudo apt install -y curl jq gawk bc mailutils

    Note: mailutils provides the mail command. Optional; only needed if you’ll email reports.

  • Fedora/RHEL/CentOS (dnf):

    sudo dnf install -y curl jq gawk bc mailx

  • openSUSE (zypper):

    sudo zypper install -y curl jq gawk bc mailx

Step 1: Define AI SLIs and SLOs

Start with a small, high-signal set:

  • Availability

    • Experience availability: 2xx / total requests
    • Provider availability: 2xx / (total - 4xx excluding 429). This removes clear client errors but still counts vendor rate-limiting.
    • Example SLO: provider availability ≥ 99.9%

  • Latency

    • P95 latency for successful requests (ms)
    • Example SLO: P95 ≤ 1000 ms

  • Rate limits

    • Percentage of 429 responses
    • Target: keep under 0.2% or what your plan supports

  • Token usage

    • Total tokens in/out as a leading indicator for capacity and cost
    • Not an SLO by itself, but crucial for planning

  • Policy/safety blocks (optional)

    • Share of responses flagged by your policy layer (policy_violation == true)
    • Target derived from your risk tolerance

Write down your SLOs explicitly; you’ll evaluate PASS/FAIL in the script.

Step 2: Instrument logs for AI traffic

Emit one JSON line per request, capturing enough to reconstruct SLIs:

Example JSONL (one line per request):

{"ts":"2026-07-08T14:31:22Z","service":"ai-gw","provider":"openai","model":"gpt-4o-mini","status":200,"latency_ms":423,"tokens_in":118,"tokens_out":201,"policy_violation":false,"rate_limited":false,"region":"us-east-1"}
{"ts":"2026-07-08T14:31:23Z","service":"ai-gw","provider":"anthropic","model":"claude-3-haiku","status":429,"latency_ms":0,"tokens_in":41,"tokens_out":0,"policy_violation":false,"rate_limited":true,"region":"us-west-2"}

Store them at:

  • /var/log/ai_gateway/requests.jsonl

Optional logrotate snippet (/etc/logrotate.d/ai_gateway):

/var/log/ai_gateway/requests.jsonl {
  daily
  rotate 14
  compress
  missingok
  notifempty
  copytruncate
}

Tip: Keep timestamps in UTC ISO-8601 with Z (lexicographic comparisons in jq then “just work”).

Step 3: Generate a daily SLA report with Bash

Save the script below as ai-sla-report.sh, make it executable, and run it. It computes availability, latency percentiles, token totals, and flags SLO PASS/FAIL for a time window (defaults to “yesterday UTC”).

#!/usr/bin/env bash
set -euo pipefail

# Configuration (overrides via environment)
LOG_FILE="${LOG_FILE:-/var/log/ai_gateway/requests.jsonl}"
SINCE="${SINCE:-$(date -u -d 'yesterday 00:00:00' +%FT%TZ)}"
UNTIL="${UNTIL:-$(date -u -d 'yesterday 23:59:59' +%FT%TZ)}"
SLO_AVAILABILITY="${SLO_AVAILABILITY:-99.9}"         # Provider availability target
SLO_P95_LATENCY_MS="${SLO_P95_LATENCY_MS:-1000}"     # Target in ms

# Dependencies check
for cmd in jq awk bc date; do
  command -v "$cmd" >/dev/null 2>&1 || { echo "Missing dependency: $cmd"; exit 1; }
done

TMPDIR="$(mktemp -d)"
trap 'rm -rf "$TMPDIR"' EXIT

FILTER='.ts >= env(SINCE) and .ts <= env(UNTIL)'

# Filter once
if [ ! -f "$LOG_FILE" ]; then
  echo "Log file not found: $LOG_FILE" >&2
  exit 1
fi
jq -r "select($FILTER)" "$LOG_FILE" > "$TMPDIR/subset.jsonl"

total=$(wc -l < "$TMPDIR/subset.jsonl" | awk '{print $1}')
if [ "$total" -eq 0 ]; then
  echo "AI SLA Report ($SINCE to $UNTIL, UTC)"
  echo "No records found in the selected time window."
  exit 0
fi

# Status buckets
succ=$(jq -r 'select(.status>=200 and .status<300) | 1' "$TMPDIR/subset.jsonl" | wc -l | awk '{print $1}')
fourxx_no429=$(jq -r 'select(.status>=400 and .status<500 and .status!=429) | 1' "$TMPDIR/subset.jsonl" | wc -l | awk '{print $1}')
rate429=$(jq -r 'select(.status==429) | 1' "$TMPDIR/subset.jsonl" | wc -l | awk '{print $1}')
fivexx=$(jq -r 'select(.status>=500) | 1' "$TMPDIR/subset.jsonl" | wc -l | awk '{print $1}')

# Availabilities
avail_all=$(printf 'scale=4; 100*%s/%s\n' "$succ" "$total" | bc)
denom_no4xx=$(( total - fourxx_no429 ))
if [ "$denom_no4xx" -gt 0 ]; then
  avail_no4xx=$(printf 'scale=4; 100*%s/%s\n' "$succ" "$denom_no4xx" | bc)
else
  avail_no4xx="100"
fi

# Latencies for successful responses
jq -r 'select(.status>=200 and .status<300 and .latency_ms!=null) | .latency_ms' "$TMPDIR/subset.jsonl" | sort -n > "$TMPDIR/latencies.txt"
nlat=$(wc -l < "$TMPDIR/latencies.txt" | awk '{print $1}')
if [ "$nlat" -gt 0 ]; then
  p_index() { # nearest-rank percentile
    local p=$1
    local rank
    rank=$(awk -v p="$p" -v n="$nlat" 'BEGIN{r=int((p/100.0)*n+0.999999); if(r<1) r=1; print r}')
    sed -n "${rank}p" "$TMPDIR/latencies.txt"
  }
  p50=$(p_index 50)
  p95=$(p_index 95)
  p99=$(p_index 99)
else
  p50=0; p95=0; p99=0
fi

# Tokens
tin=$(jq -r 'select(.tokens_in!=null) | .tokens_in' "$TMPDIR/subset.jsonl" | awk '{s+=$1} END{print s+0}')
tout=$(jq -r 'select(.tokens_out!=null) | .tokens_out' "$TMPDIR/subset.jsonl" | awk '{s+=$1} END{print s+0}')
ttok=$(( tin + tout ))

# Policy violations (optional field)
pviol=$(jq -r 'select(.policy_violation==true) | 1' "$TMPDIR/subset.jsonl" | wc -l | awk '{print $1}')

# Throughput
start_epoch=$(date -u -d "$SINCE" +%s)
end_epoch=$(date -u -d "$UNTIL" +%s)
duration_sec=$(( end_epoch - start_epoch + 1 ))
rps=$(printf 'scale=3; %s/%s\n' "$total" "$duration_sec" | bc)

# SLO checks
status_slo_avail=$(awk -v a="$avail_no4xx" -v s="$SLO_AVAILABILITY" 'BEGIN{if (a+0 >= s+0) print "PASS"; else print "FAIL"}')
status_slo_p95=$(awk -v p="$p95" -v s="$SLO_P95_LATENCY_MS" 'BEGIN{if (p+0 <= s+0) print "PASS"; else print "FAIL"}')

# Output (Markdown-friendly)
echo "AI SLA Report ($SINCE to $UNTIL, UTC)"
echo ""
echo "- Requests total: $total  |  Success (2xx): $succ"
echo "- 4xx (excl. 429): $fourxx_no429  |  429s: $rate429  |  5xx: $fivexx"
echo "- Experience availability (2xx/total): ${avail_all}%"
echo "- Provider availability (2xx/(total-4xx_except_429)): ${avail_no4xx}%  => SLO ${SLO_AVAILABILITY}%: ${status_slo_avail}"
echo "- Latency ms (success only): P50=${p50}  P95=${p95}  P99=${p99}  => SLO P95 ≤ ${SLO_P95_LATENCY_MS}ms: ${status_slo_p95}"
echo "- Tokens: in=${tin}  out=${tout}  total=${ttok}"
echo "- Policy violations (if logged): ${pviol}"
echo "- Avg throughput: ${rps} req/s over ${duration_sec}s"

Usage examples:

  • Run for yesterday (default):

    ./ai-sla-report.sh

  • Run for a custom window:

    SINCE="2026-07-08T00:00:00Z" UNTIL="2026-07-08T23:59:59Z" ./ai-sla-report.sh

  • Tighten the latency SLO to 800 ms:

    SLO_P95_LATENCY_MS=800 ./ai-sla-report.sh

Security tip: Logs may include user content. Scrub PII upstream or store only hashes/metadata.

Step 4: Automate reports (cron or systemd) and email

  • Cron (user-level, runs at 01:05 UTC daily):

    crontab -e

    Add:

    5 1 * * * LOG_FILE=/var/log/ai_gateway/requests.jsonl /path/to/ai-sla-report.sh | tee -a ~/ai-sla/daily.log

  • Optional email delivery

    • Ensure mailx is installed (see package manager commands above).
    • Send the report:
    /path/to/ai-sla-report.sh | mail -s "AI SLA Report $(date -u +\%F)" ops@example.com

  • systemd timer (root or user units)

    • /etc/systemd/system/ai-sla-report.service
    [Unit]
Description=AI SLA daily report

[Service]
Type=oneshot
Environment=LOG_FILE=/var/log/ai_gateway/requests.jsonl
Environment=SLO_AVAILABILITY=99.9
Environment=SLO_P95_LATENCY_MS=1000
ExecStart=/path/to/ai-sla-report.sh
    • /etc/systemd/system/ai-sla-report.timer
    [Unit]
Description=Run AI SLA report daily at 01:05 UTC

[Timer]
OnCalendar=*-*-* 01:05:00 UTC
Persistent=true

[Install]
WantedBy=timers.target
    • Enable and start:
    sudo systemctl daemon-reload
sudo systemctl enable --now ai-sla-report.timer

Step 5: Real-world usage examples

  • New model rollout check

    • Slice by model in the script (add select(.model=="gpt-4o-mini")) to compare P95 latencies and 429 rates pre/post rollout.
    • Abort rollout if P95 jumps 30% or provider availability dips below 99.9%.

  • Rate-limit tuning

    • Monitor 429% daily. If > 0.2%, increase concurrency backoff or request batching. Re-run report to verify improvement.

  • Capacity and cost planning

    • Track tokens_in/out month-over-month to anticipate spend and negotiate provider quotas before traffic spikes.

  • Incident review

    • During a 5xx spike, the report gives you exact percentages and windows, separated from client-side 4xx noise.

Troubleshooting and extensions

  • No records found: verify timestamps are in UTC ISO-8601 with Z and match the SINCE/UNTIL window.

  • Performance: for very large logs, consider splitting per day or pre-aggregating with a streaming job.

  • Add dimensions: group by provider, region, or customer tier using jq’s group_by and Bash loops.

  • Export metrics: emit Prometheus text format and scrape locally, or push to your TSDB after each run.

  • Charts: you can optionally install gnuplot for quick PNG histograms. Install with:

    • apt:
    sudo apt update && sudo apt install -y gnuplot
    • dnf:
    sudo dnf install -y gnuplot
    • zypper:
    sudo zypper install -y gnuplot

Conclusion and next steps

SLA reporting for AI doesn’t have to start with a SaaS subscription. With Bash, jq, and disciplined logging, you can establish rigorous, auditable SLAs today:

  • Define clear AI SLIs/SLOs

  • Log minimal, structured metadata per request

  • Automate a daily report and wire it into your ops loops

Call to action: 1) Drop the script into your repo and point it at your AI gateway logs.
2) Set initial SLOs, schedule the job, and share the first report with your team.
3) Iterate: add grouping, alerts, or export to your metrics stack.

If you’d like, share your modified version or questions—happy to help you tailor this to your stack.

Related posts