Posted on
• Artificial Intelligence

RAG Project Ideas

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

RAG Project Ideas for the Linux Terminal: Build Practical Retrieval-Augmented Generation Pipelines with Bash

If you’ve ever grepped for the same flag across man pages, trawled logs after an outage, or dug through a repo to recall how a script works, you’ve lived the problem RAG solves. Retrieval-Augmented Generation (RAG) makes large language models (LLMs) grounded and useful by letting them “look up” relevant text (docs, logs, code) before answering.

This post shows how to build small, Unix-y RAG workflows that run from your terminal. You’ll get a minimal, scriptable RAG starter kit, plus four practical project ideas you can ship this week.

  • Why this matters: Out-of-the-box LLMs can hallucinate. RAG adds the missing context—your context—so answers are accurate, auditable, and repeatable.

  • Why Linux/Bash: The Unix philosophy is perfect for RAG. Your system already has text everywhere and excellent tools to slice/dice it. You don’t need a heavyweight stack to get real value.

Prerequisites: Install the tooling

You’ll need some common CLI tools and Python libraries. Use your distro’s package manager.

System packages:

  • Python 3 + pip

  • git, curl, jq

  • ripgrep (rg), fd (optional), pandoc, sqlite3

On Debian/Ubuntu (apt):

sudo apt update && sudo apt install -y \
  python3 python3-venv python3-pip \
  git curl jq ripgrep fd-find pandoc sqlite3
# Note: on Debian/Ubuntu the fd binary is named "fdfind".
# You can alias it if you like:
#   echo 'alias fd=fdfind' >> ~/.bashrc && source ~/.bashrc

On Fedora/RHEL (dnf):

sudo dnf install -y \
  python3 python3-pip \
  git curl jq ripgrep fd-find pandoc sqlite

On openSUSE (zypper):

sudo zypper install -y \
  python3 python3-pip \
  git curl jq ripgrep fd pandoc sqlite3

Python environment:

python3 -m venv .venv
. .venv/bin/activate
pip install --upgrade pip
pip install sentence-transformers numpy tqdm

Optional: run a local LLM via llama.cpp (OpenAI-compatible server). Build tools:

  • apt:
sudo apt update && sudo apt install -y build-essential cmake git
  • dnf:
sudo dnf groupinstall -y "Development Tools"
sudo dnf install -y cmake git
  • zypper:
sudo zypper install -y -t pattern devel_basis
sudo zypper install -y cmake git

Build and run llama.cpp:

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make -j && cd ..
# Start the server (replace with your local GGUF model path)
./llama.cpp/server -m /path/to/model.gguf -c 4096 --port 8080

Alternatively, use a hosted API. Set your key:

export OPENAI_API_KEY="sk-..."

A minimal RAG starter kit you can reuse

This tiny Python script indexes text files into SQLite with embeddings and lets you query the top-k relevant chunks. Glue it to any LLM (local or API) from Bash.

Save as rag_index.py:

#!/usr/bin/env python3
import argparse, os, sys, sqlite3, json, math
from pathlib import Path
import numpy as np
from tqdm import tqdm
from sentence_transformers import SentenceTransformer

DEFAULT_DB = "rag.sqlite"
DEFAULT_MODEL = os.environ.get("EMBED_MODEL", "sentence-transformers/all-MiniLM-L6-v2")

SCHEMA = """
CREATE TABLE IF NOT EXISTS docs(
  id INTEGER PRIMARY KEY,
  path TEXT,
  chunk_idx INTEGER,
  text TEXT,
  vec TEXT
);
CREATE INDEX IF NOT EXISTS idx_docs_path ON docs(path);
"""

def connect(db_path):
    conn = sqlite3.connect(db_path)
    conn.execute("PRAGMA journal_mode=WAL")
    for stmt in SCHEMA.strip().split(";"):
        if stmt.strip():
            conn.execute(stmt)
    return conn

def read_text(path):
    try:
        # Skip files > 2 MiB for demo purposes
        if os.path.getsize(path) > 2 * 1024 * 1024:
            return None
        with open(path, "r", encoding="utf-8", errors="ignore") as f:
            return f.read()
    except Exception:
        return None

def chunk_words(text, size=300, overlap=60):
    words = text.split()
    n = len(words)
    i = 0
    while i < n:
        j = min(i + size, n)
        yield " ".join(words[i:j])
        if j == n:
            break
        i = max(0, j - overlap)

def embed(model, texts):
    # normalize to enable cosine via dot product
    vecs = model.encode(texts, normalize_embeddings=True)
    return [v.tolist() for v in vecs]

def cmd_index(args):
    conn = connect(args.db)
    cur = conn.cursor()
    model = SentenceTransformer(DEFAULT_MODEL)
    files = []
    for p in args.files:
        pp = Path(p)
        if pp.is_file():
            files.append(str(pp))
        elif pp.is_dir():
            for root, _, fns in os.walk(pp):
                for fn in fns:
                    files.append(str(Path(root) / fn))
    if not files:
        print("No files to index", file=sys.stderr)
        return
    batch_texts, batch_meta = [], []
    for path in tqdm(files, desc="Reading"):
        txt = read_text(path)
        if not txt:
            continue
        for i, chunk in enumerate(chunk_words(txt, size=args.chunk, overlap=args.overlap)):
            if not chunk.strip():
                continue
            batch_texts.append(chunk)
            batch_meta.append((path, i, chunk))
            if len(batch_texts) >= args.batch:
                vecs = embed(model, batch_texts)
                cur.executemany(
                    "INSERT INTO docs(path, chunk_idx, text, vec) VALUES(?,?,?,?)",
                    [(m[0], m[1], m[2], json.dumps(v)) for m, v in zip(batch_meta, vecs)]
                )
                conn.commit()
                batch_texts, batch_meta = [], []
    if batch_texts:
        vecs = embed(model, batch_texts)
        cur.executemany(
            "INSERT INTO docs(path, chunk_idx, text, vec) VALUES(?,?,?,?)",
            [(m[0], m[1], m[2], json.dumps(v)) for m, v in zip(batch_meta, vecs)]
        )
        conn.commit()
    print("Indexing complete.")

def cosine_topk(query_vec, mat, k):
    # mat: NxD, query: D
    sims = mat @ query_vec
    idx = np.argpartition(-sims, kth=min(k, len(sims)-1))[:k]
    idx = idx[np.argsort(-sims[idx])]
    return [(int(i), float(sims[i])) for i in idx]

def cmd_search(args):
    conn = connect(args.db)
    cur = conn.cursor()
    rows = cur.execute("SELECT id, path, chunk_idx, text, vec FROM docs").fetchall()
    if not rows:
        print("Empty index", file=sys.stderr); return
    model = SentenceTransformer(DEFAULT_MODEL)
    qv = model.encode([args.query], normalize_embeddings=True)[0]
    vecs = np.array([json.loads(r[4]) for r in rows], dtype=np.float32)
    top = cosine_topk(qv, vecs, args.topk)
    results = []
    for i, score in top:
        rid, path, chunk_idx, text, _ = rows[i]
        results.append({
            "id": rid, "path": path, "chunk_idx": chunk_idx,
            "score": round(score, 4), "text": text
        })
    if args.json:
        print(json.dumps(results, ensure_ascii=False, indent=2))
    else:
        for r in results:
            print(f"[{r['score']}] {r['path']}:{r['chunk_idx']}\n{r['text']}\n---")

if __name__ == "__main__":
    p = argparse.ArgumentParser(description="Tiny RAG index/search over text files")
    sub = p.add_subparsers(dest="cmd", required=True)

    p_index = sub.add_parser("index")
    p_index.add_argument("--db", default=DEFAULT_DB)
    p_index.add_argument("--chunk", type=int, default=300)
    p_index.add_argument("--overlap", type=int, default=60)
    p_index.add_argument("--batch", type=int, default=64)
    p_index.add_argument("files", nargs="+", help="Files or directories to index")
    p_index.set_defaults(func=cmd_index)

    p_search = sub.add_parser("search")
    p_search.add_argument("--db", default=DEFAULT_DB)
    p_search.add_argument("--topk", type=int, default=5)
    p_search.add_argument("--json", action="store_true")
    p_search.add_argument("query")
    p_search.set_defaults(func=cmd_search)

    args = p.parse_args()
    args.func(args)

Example: query top-5 relevant chunks in your index:

python rag_index.py search --db rag.sqlite --topk 5 --json "How do I reload systemd units?"

Glue retrieval to an LLM (API):

Q="How do I reload systemd units?"
python rag_index.py search --db rag.sqlite --topk 5 --json "$Q" \
  | jq -r '.[].text' > /tmp/context.txt

curl -sS https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --arg ctx "$(cat /tmp/context.txt)" --arg q "$Q" \
        '{model:"gpt-4o-mini",
          messages:[
            {role:"system",content:"Answer using only the provided Linux context."},
            {role:"user",content:("Context:\n"+$ctx+"\n\nQuestion: "+$q)}
          ],
          temperature:0.2}')" \
  | jq -r '.choices[0].message.content'

Glue retrieval to a local llama.cpp server:

Q="Why does grep -r skip binary files?"
python rag_index.py search --db rag.sqlite --topk 5 --json "$Q" \
  | jq -r '.[].text' > /tmp/context.txt

curl -sS http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d "$(jq -n --arg ctx "$(cat /tmp/context.txt)" --arg q "$Q" \
        '{model:"local",
          messages:[
            {role:"system",content:"Answer using only the provided Linux context."},
            {role:"user",content:("Context:\n"+$ctx+"\n\nQuestion: "+$q)}
          ],
          temperature:0.2}')" \
  | jq -r '.choices[0].message.content'

4 Practical RAG Project Ideas (with Bash-friendly steps)

These blueprints use the same starter kit. Swap in your sources, reindex, ask better questions.

1) Man Pages + /etc Config Whisperer

Problem: You remember a flag exists but not where. You tweak a .conf and forget the syntax.

Ingest:

mkdir -p data
# A few commonly read man pages (add your own)
man --pager=cat bash     > data/man-bash.txt
man --pager=cat systemd  > data/man-systemd.txt
man --pager=cat grep     > data/man-grep.txt
man --pager=cat journalctl > data/man-journalctl.txt

# Configs under /etc (limit depth to keep it fast)
find /etc -maxdepth 2 -type f \( -name "*.conf" -o -name "*.ini" -o -name "*.cfg" \) -readable -print0 \
  | xargs -0 -n 100 python rag_index.py index --db rag.sqlite
# Also index the manpage dumps:
python rag_index.py index --db rag.sqlite data/*.txt

Ask:

python rag_index.py search --db rag.sqlite --topk 6 --json "Reload vs restart in systemd? When to use daemon-reload?" \
  | jq -r '.[].text' | sed -n '1,80p'

Then pipe the context to your LLM of choice as shown above.

Why it works: Your answers are grounded in the exact versions of man pages and configs on your machine.

2) Log Triage Copilot (journalctl + /var/log)

Problem: After an incident, you need a narrative, not a haystack.

Ingest last 24 hours and classic logs:

mkdir -p logs
sudo journalctl -S "24 hours ago" > logs/journal-24h.txt
sudo cp /var/log/*.log logs/ 2>/dev/null || true
python rag_index.py index --db rag.sqlite logs/*

Ask:

python rag_index.py search --db rag.sqlite --topk 8 --json "Why did nginx reload last night and what errors preceded it?" \
  | jq -r '.[].text' | sed -n '1,120p'

Tip: Reindex on a schedule so your RAG stays fresh. Consider redacting secrets before indexing.

3) Repo and Script Navigator

Problem: You forget how a bespoke script or Make target works.

Ingest source and docs from a repo:

REPO=~/src/myproject
# List texty files; exclude binaries
rg -uu --no-binary --files "$REPO" > /tmp/files.txt
xargs -a /tmp/files.txt -n 100 python rag_index.py index --db rag.sqlite

Ask:

python rag_index.py search --db rag.sqlite --topk 7 --json \
  "How do I run the integration tests and what env vars do they require?" \
  | jq -r '.[].text' | sed -n '1,120p'

Nice add-ons:

  • Include CHANGELOG* and git log --oneline output for release context.

  • Use pandoc to convert README.md and docs/*.md to plain text before indexing if you want cleaner chunks.

4) Ops Handbook: Offline Docs and Wikis

Problem: You need operational knowledge even when the VPN or site is down.

If you have HTML exports of your wiki or vendor docs:

mkdir -p txt
for f in docs_export/*.html; do
  out="txt/$(basename "$f" .html).txt"
  pandoc -f html -t plain "$f" -o "$out"
done
python rag_index.py index --db rag.sqlite txt/*.txt

Ask:

python rag_index.py search --db rag.sqlite --topk 6 --json \
  "Postgres vacuum best practices and what thresholds we use?" \
  | jq -r '.[].text' | sed -n '1,120p'

Tip: Keep per-team or per-service indexes (rag-postgres.sqlite, rag-kafka.sqlite) to stay targeted and fast.

Why these projects deliver value

  • They keep answers grounded in your actual environment: your configs, your logs, your code.

  • They’re auditable. Every answer can point back to source chunks.

  • They’re small and Unix-y. No heavyweight vector DB required; SQLite and a few scripts go a long way.

  • They’re extensible. Swap sources, change chunking, plug in any LLM backend (API or local).

Performance notes:

  • For very large corpora, switch vector storage to a library like FAISS or SQLite extensions with vector search. For most team-scale uses, the in-memory cosine approach above is fine.

  • Tune chunk sizes: configs like 200–400 words with 10–20% overlap tend to work well.

Call to Action

  • Pick one idea (man pages + /etc or logs) and stand up the starter kit in 20 minutes.

  • Wire retrieval to your preferred LLM (API or llama.cpp) and ask three “real” questions you struggled with last week.

  • Iterate: add sources, refine chunking, script nightly reindexing via cron/systemd timers.

If you want a follow-up post with FAISS-based retrieval, multi-index routing, or a TUI wrapper, say the word—and share what you build!

Related posts