Posted on
• Artificial Intelligence

RAG Case Studies

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

RAG Case Studies for Linux Users: Practical, Bash-Friendly Blueprints

Tired of LLMs “sounding confident” while getting your infrastructure details wrong? Retrieval-Augmented Generation (RAG) fixes this by letting models look up your real docs, configs, and logs before they answer. In other words: fewer hallucinations, more grounded, auditable output—while your data stays on your machines.

This post walks through why RAG is worth your time, real case studies from Linux-heavy environments, and a minimal, reproducible stack you can run from Bash. You’ll get copy-paste snippets for apt, dnf, and zypper along the way.

Why RAG matters on Linux

  • Your infra is unique. LLMs trained on the internet haven’t seen your runbooks, service topology, or patched kernels. RAG injects your ground truth into the answer.

  • Privacy and control. Keep data local; no need to ship logs and configs to third-party APIs.

  • Reproducible interfaces. Bash, systemd, cron, and plain files provide stable, automatable plumbing.

  • Auditable answers. RAG surfaces the specific passages used, so you can verify source and improve your docs.

Case studies: RAG in the trenches

1) On-call runbook copilot

  • Problem: New SREs needed hours to find the right wiki page or script under pressure.

  • RAG: Index Markdown runbooks, postmortems, and common scripts; query in natural language; show exact snippets used for answers.

  • Outcome: Faster time-to-answer, fewer “tribal knowledge” bottlenecks, and better doc hygiene (holes surface quickly).

2) Log triage for incidents

  • Problem: Sifting through journald logs and rotated files during incidents is slow.

  • RAG: Chunk and embed logs, retrieve similar incidents and likely root causes; show diffs between today’s failure and prior resolved ones.

  • Outcome: Focused search with contextual memory; easier to explain incident patterns.

3) Codebase Q&A for infra repos

  • Problem: “Where is TLS enforced?” or “Which script restarts service X?” is tedious across large repos.

  • RAG: Index code comments, READMEs, and scripts; retrieval points to function names and files; LLM explains flow with citations.

  • Outcome: Fewer Slack pings to “code historians,” better onboarding.

4) Security advisories and package notes

  • Problem: Keeping up with CVEs for your specific stack is noisy.

  • RAG: Ingest curated advisories and release notes; query “Is our OpenSSL version affected?” with immediate source links.

  • Outcome: Focused, actionable risk summaries for your actual environment.

Minimal, Bash-first RAG blueprint

Below is a compact, reproducible setup using:

  • FAISS for vector search (local)

  • Sentence-Transformers for embeddings

  • Ollama for a local LLM (e.g., Llama 3)

  • Simple Python scripts invoked from Bash

You can swap pieces later (e.g., Chroma, Milvus, llama.cpp) without changing the workflow much.

1) Install prerequisites

  • Debian/Ubuntu (apt):
sudo apt update
sudo apt install -y python3 python3-venv python3-pip git curl ripgrep jq build-essential
  • Fedora/RHEL/CentOS (dnf):
sudo dnf install -y python3 python3-pip git curl ripgrep jq gcc make
  • openSUSE/SLES (zypper):
sudo zypper refresh
sudo zypper install -y python3 python3-pip python3-venv git curl ripgrep jq gcc make

Optional (CPU-only PyTorch wheel to speed up installs):

python3 -m pip install --upgrade pip
python3 -m pip install --index-url https://download.pytorch.org/whl/cpu torch

Install Ollama (local LLM runtime):

curl -fsSL https://ollama.com/install.sh | sh
# In a separate terminal or tmux pane:
ollama serve
# Pull a model (adjust to your hardware/model preference):
ollama pull llama3

Tip: If you prefer a systemd user service for Ollama:

systemctl --user enable ollama
systemctl --user start ollama

2) Create a workspace and Python env

mkdir -p ~/rag-lab && cd ~/rag-lab
python3 -m venv .venv
. .venv/bin/activate
pip install --upgrade pip
pip install sentence-transformers faiss-cpu tqdm requests

3) Prepare your corpus

Start small and focused:

  • Runbooks and wikis exported to .md or .html

  • Logs from /var/log or journald exports

  • README.md and scripts from your infra repos

Example:

mkdir -p data/runbooks
# copy or export your docs here
cp -r ~/wiki/runbooks/*.md data/runbooks/ 2>/dev/null || true

# collect some logs (mind PII/secrets; sanitize as needed)
sudo journalctl -u nginx --since "2 days ago" > data/nginx.journal.log
sudo cp -a /var/log/nginx/*.log data/ 2>/dev/null || true

4) Indexing script (chunk + embed + FAISS)

Save as ingest.py:

#!/usr/bin/env python3
import os, json, argparse, pathlib, re
from tqdm import tqdm
import faiss
import numpy as np
from sentence_transformers import SentenceTransformer

def read_text(path):
    try:
        with open(path, "r", encoding="utf-8", errors="ignore") as f:
            return f.read()
    except Exception:
        return ""

def chunk_text(text, size=1200, overlap=200):
    # character-based chunks; simple and robust for mixed content
    chunks = []
    i = 0
    while i < len(text):
        chunks.append(text[i:i+size])
        i += max(1, size - overlap)
    return [c.strip() for c in chunks if c.strip()]

def collect_files(root):
    exts = {".md", ".txt", ".log", ".conf", ".ini", ".yaml", ".yml", ".py", ".sh"}
    for p in pathlib.Path(root).rglob("*"):
        if p.is_file() and (p.suffix.lower() in exts or p.suffix == ""):
            yield str(p)

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--src", required=True, help="Folder with docs/logs/code")
    ap.add_argument("--out", default="index", help="Output index dir")
    ap.add_argument("--model", default="sentence-transformers/all-MiniLM-L6-v2")
    ap.add_argument("--chunk", type=int, default=1200)
    ap.add_argument("--overlap", type=int, default=200)
    args = ap.parse_args()

    os.makedirs(args.out, exist_ok=True)
    meta_path = os.path.join(args.out, "meta.jsonl")
    index_path = os.path.join(args.out, "faiss.index")
    map_path = os.path.join(args.out, "idmap.json")

    model = SentenceTransformer(args.model)
    vecs = []
    metas = []
    id2meta = {}

    idx = 0
    files = list(collect_files(args.src))
    for fp in tqdm(files, desc="Indexing"):
        text = read_text(fp)
        if not text.strip():
            continue
        # light cleanup for logs
        text = re.sub(r"\x1b\[[0-9;]*m", "", text)  # strip ANSI
        for chunk in chunk_text(text, args.chunk, args.overlap):
            metas.append({"path": fp, "chunk_id": idx, "text": chunk[:5000]})
            idx += 1

    if not metas:
        print("No content found. Check your --src path.")
        return

    texts = [m["text"] for m in metas]
    emb = model.encode(texts, batch_size=64, show_progress_bar=True, normalize_embeddings=True)
    emb = np.asarray(emb, dtype="float32")

    index = faiss.IndexFlatIP(emb.shape[1])  # cosine via normalized vectors
    index.add(emb)
    faiss.write_index(index, index_path)

    with open(meta_path, "w", encoding="utf-8") as f:
        for i, m in enumerate(metas):
            m_ = dict(m)
            m_["id"] = i
            f.write(json.dumps(m_) + "\n")

    id2meta = {i: {"path": m["path"]} for i, m in enumerate(metas)}
    with open(map_path, "w", encoding="utf-8") as f:
        json.dump(id2meta, f)

    print(f"Wrote {len(metas)} chunks")
    print(f"Index: {index_path}")
    print(f"Meta:  {meta_path}")

if __name__ == "__main__":
    main()

Usage:

chmod +x ingest.py
./ingest.py --src data --out index

5) Query + generate script (retrieval + Ollama)

Save as query.py:

#!/usr/bin/env python3
import os, json, argparse, requests, faiss
import numpy as np
from sentence_transformers import SentenceTransformer

def load_meta(meta_path):
    metas = []
    with open(meta_path, "r", encoding="utf-8") as f:
        for line in f:
            metas.append(json.loads(line))
    return metas

def build_prompt(question, contexts):
    header = "You are a helpful assistant. Use ONLY the context below. If unknown, say you don't know.\n"
    ctx = "\n\n".join([f"[{i+1}] ({c['path']})\n{c['text']}" for i, c in enumerate(contexts)])
    return f"{header}\nQuestion:\n{question}\n\nContext:\n{ctx}\n\nAnswer:"

def ollama_generate(model, prompt):
    url = "http://localhost:11434/api/generate"
    resp = requests.post(url, json={"model": model, "prompt": prompt, "stream": False}, timeout=600)
    resp.raise_for_status()
    return resp.json().get("response", "").strip()

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--index", default="index/faiss.index")
    ap.add_argument("--meta", default="index/meta.jsonl")
    ap.add_argument("--model", default="sentence-transformers/all-MiniLM-L6-v2")
    ap.add_argument("--ollama", default="llama3", help="Ollama model name")
    ap.add_argument("--k", type=int, default=5)
    ap.add_argument("question", nargs="+")
    args = ap.parse_args()

    question = " ".join(args.question)
    metas = load_meta(args.meta)
    em = np.array([m["text"] for m in metas])  # just to keep length aligned

    enc = SentenceTransformer(args.model)
    q = enc.encode([question], normalize_embeddings=True).astype("float32")
    index = faiss.read_index(args.index)
    D, I = index.search(q, args.k)
    hits = [metas[i] for i in I[0]]

    prompt = build_prompt(question, hits)
    try:
        answer = ollama_generate(args.ollama, prompt)
        print(answer)
        print("\n---\nSources:")
        for i, h in enumerate(hits):
            print(f"[{i+1}] {h['path']}")
    except Exception as e:
        print("Ollama not available or request failed. Showing prompt and contexts instead.\n")
        print(prompt)

if __name__ == "__main__":
    main()

Usage:

chmod +x query.py
# Example queries
./query.py "How do I restart nginx gracefully and where is it documented?"
./query.py --k 8 "What changed in yesterday's error logs for nginx?"

Tip: If you prefer cURL over Python requests, you can print the prompt and pipe it to ollama run:

PROMPT="$(./query.py --k 5 'Where do we configure TLS ciphers?' 2>/dev/null)"
echo "$PROMPT" | ollama run llama3

3–5 actionable practices that make RAG work in prod

  • Start with a narrow, high-value corpus
    Index the 50–200 most used runbooks, top logs from one critical service, or one repo. Measure usefulness before scaling.

  • Keep chunks semantic and source-rich
    Include file paths, timestamps, and service names in the metadata that appears with every retrieved chunk. This drives trust and debuggability.

  • Evaluate retrieval before generation
    Sanity-check top-k retrieval with simple grep/baselines. If the right docs don’t show up, no LLM can rescue the answer. Keep a tiny gold set of Q/A with expected sources.

  • Sanitize and scope logs
    Strip ANSI, redact secrets/tokens, and avoid indexing high-churn noise. Maintain exclusion patterns (node_modules, .git, tmp).

  • Automate refresh + prune
    Re-index on a schedule (e.g., nightly), and prune stale docs. Automate with cron or systemd user timers.

Example systemd timer:

# ~/.config/systemd/user/rag-index.service
[Unit]
Description=Rebuild RAG index

[Service]
Type=oneshot
WorkingDirectory=%h/rag-lab
ExecStart=%h/rag-lab/.venv/bin/python %h/rag-lab/ingest.py --src %h/rag-lab/data --out %h/rag-lab/index

# ~/.config/systemd/user/rag-index.timer
[Unit]
Description=Nightly RAG index rebuild

[Timer]
OnCalendar=daily
Persistent=true

[Install]
WantedBy=timers.target

Enable:

systemctl --user daemon-reload
systemctl --user enable --now rag-index.timer

Real-world example walkthroughs

  • On-call copilot

    • Corpus: data/runbooks/*.md, postmortems, team wiki exports.
    • Query: “Our nginx 502 playbook mentions two commands; what are they and when to use each?”
    • Result: Top chunks show the known-good commands with exact file paths; LLM summarizes with steps and citations.

  • Log triage

    • Corpus: journalctl -u myservice, rotated logs for last 7 days.
    • Query: “Why did myservice flap at 02:00 UTC today? Compare to previous flaps.”
    • Result: Retrieved chunks show errors and prior incidents; LLM explains similarity and differences.

  • Code Q&A

    • Corpus: infra repo scripts, configs, README.md.
    • Query: “Where are TLS ciphers set for ingress?”
    • Result: Points to ingress.conf and a helper script; LLM summarizes interplay.

  • Security KB

    • Corpus: curated CVE notes and vendor advisories relevant to your versions.
    • Query: “Is our OpenSSL 1.1.1w affected by CVE-XXXX-YYYY? Mitigation?”
    • Result: Context includes the exact advisory; LLM outputs a short, actionable summary with links and version scope.

Troubleshooting tips

  • “Pip installs torch/cuda I don’t need”

    • Install CPU torch first:
      pip install --index-url https://download.pytorch.org/whl/cpu torch

  • “FAISS not found on import”

    • Ensure you installed faiss-cpu inside the venv; restart the shell; python -c "import faiss; print('ok')"

  • “Ollama connection refused”

    • Run ollama serve in a terminal, or systemctl --user start ollama.
    • Verify: curl -s http://localhost:11434/api/tags | jq.

  • Index too big/slow

    • Reduce chunk size, increase overlap modestly, or restrict file types.
    • Start with k=3..5 for queries.

Conclusion and next steps

RAG helps LLMs answer your infra questions with your own truth, right from Bash. You now have:

  • A minimal local stack (FAISS + Sentence-Transformers + Ollama)

  • End-to-end scripts to index and query

  • Practical patterns from real use cases

Next:

  • Point the indexer at a small but valuable subset of docs/logs.

  • Capture 10–20 real questions your team asks; test retrieval quality first.

  • Automate nightly refresh and iterate on chunking/filters.

When you’re ready to go further, consider:

  • Structured retrievers (OpenSearch/Elasticsearch), better chunking (Markdown/HTML-aware), and evaluation harnesses.

  • Access control per corpus and redaction pipelines.

  • Lightweight APIs around query.py for chat UIs.

If you found this useful, try the blueprint today and tailor it to one concrete workflow—on-call, logs, code, or security. Your future self (and your incident bridge) will thank you.

Related posts