Building Artificial Intelligence Knowledge Bases from the Linux Command Line

If your team’s know-how lives in scattered READMEs, PDFs, tickets, and wikis, you’re burning time hunting for answers. An AI-powered knowledge base turns that sprawl into a fast, searchable system that can even draft answers. The best part? You can build it locally with Bash and a few open-source tools—no vendor lock-in, and your data stays on your machines.

This guide shows you how to assemble a private, scriptable Retrieval-Augmented Generation (RAG) pipeline on Linux. You’ll normalize documents to plain text, embed them as vectors, index for fast search, and query them via CLI. We’ll keep it minimal, reproducible, and distro-friendly.

Why this matters

• Faster answers, less context switching: Ask a question and pull relevant snippets in seconds.

• Privacy by default: Keep sensitive docs on-prem with local embeddings and indices.

• Repeatable and automatable: Cron, systemd, Git, and shell scripts keep it updated and auditable.

• Extensible: Bolt on an LLM (local or remote) later to generate summarized answers.

What you’ll build

• A text normalizer for PDFs, Office docs, images (OCR), HTML, Markdown

• A Python-based embedder using Sentence Transformers

• A FAISS vector index for approximate similarity search

• A CLI to query the knowledge base, optionally piping context into an LLM like Ollama

---

0) Install prerequisites

Tools we’ll use:

• Core: Python 3, pip/venv, Git

• Converters: pandoc, poppler utilities (pdftotext), tesseract (OCR)

• Helpers: ripgrep, jq

On Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y \
  python3 python3-venv python3-pip git \
  pandoc poppler-utils tesseract-ocr \
  ripgrep jq

On Fedora/RHEL/CentOS (dnf):

# You may need EPEL for some packages on RHEL/CentOS:
# sudo dnf install -y epel-release

sudo dnf install -y \
  python3 python3-pip git \
  pandoc poppler-utils tesseract \
  ripgrep jq

On openSUSE (zypper):

sudo zypper refresh
sudo zypper install -y \
  python3 python3-pip git \
  pandoc poppler-tools tesseract \
  ripgrep jq

Create a workspace and Python environment:

mkdir -p ~/kb/{data/raw,data/text,index,tools}
cd ~/kb
python3 -m venv .venv
. .venv/bin/activate
pip install --upgrade pip
pip install "sentence-transformers>=2.6" "faiss-cpu>=1.8" "tqdm>=4.66" "numpy>=1.26"

---

1) Ingest and normalize your documents

Start by collecting raw files under data/raw. Examples:

# Example: pull internal wiki export or docs
# cp -r /mnt/share/runbooks/* data/raw/

# Example: clone public docs (replace with your repos)
git clone --depth=1 https://github.com/curl/curl.git data/raw/curl

Convert everything to plain text with a single Bash script. Save as tools/totext.sh:

#!/usr/bin/env bash
set -euo pipefail

in="${1:-data/raw}"
out="${2:-data/text}"
mkdir -p "$out"

shopt -s globstar nullglob
for f in "$in"/**/*; do
  [ -f "$f" ] || continue
  rel="${f#"$in"/}"
  base="${rel%.*}"
  ext="${f##*.}"
  mkdir -p "$out/$(dirname "$rel")"

  case "${ext,,}" in
    txt|md|rst)
      # Preserve structure
      cp -f "$f" "$out/$rel"
      ;;
    pdf)
      pdftotext -layout "$f" "$out/$base.txt" || true
      ;;
    docx|odt|pptx|xlsx|html|htm)
      pandoc -s "$f" -t plain -o "$out/$base.txt" || true
      ;;
    png|jpg|jpeg|tif|tiff)
      # OCR images
      tesseract "$f" "$out/$base" >/dev/null 2>&1 || true
      ;;
    *)
      # Unknown format: skip silently
      :
      ;;
  esac
done

# Quick sanity: list converted files
echo "Converted files:"
rg -nH --glob '!*.git/*' '' "$out" | head -n 10 || true

Make it executable and run:

chmod +x tools/totext.sh
bash tools/totext.sh data/raw data/text

Tip: Keep only what matters. Fewer, higher-quality docs beat noisy dumps.

---

2) Chunk and embed text (vectorization)

We’ll split text into overlapping chunks (good for retrieval) and embed with a compact, fast model.

Save as tools/index.py:

#!/usr/bin/env python3
import os, json, pathlib, glob
import numpy as np
from tqdm import tqdm
from sentence_transformers import SentenceTransformer
import faiss

TEXT_DIR = "data/text"
INDEX_DIR = "index"
META_PATH = os.path.join(INDEX_DIR, "meta.jsonl")
INDEX_PATH = os.path.join(INDEX_DIR, "faiss.index")
MODEL_NAME = "sentence-transformers/all-MiniLM-L6-v2"  # 384-dim

CHUNK_SIZE = 800     # characters
CHUNK_OVERLAP = 120  # characters

def chunk_text(s, size=CHUNK_SIZE, overlap=CHUNK_OVERLAP):
    s = s.strip()
    if not s:
        return []
    chunks = []
    start = 0
    while start < len(s):
        end = min(len(s), start + size)
        chunks.append((start, s[start:end]))
        start += size - overlap
        if start <= 0:  # avoid infinite loop when overlap >= size
            break
    return chunks

def load_docs():
    patterns = ["**/*.txt", "**/*.md", "**/*.rst"]
    files = []
    for p in patterns:
        files.extend(glob.glob(os.path.join(TEXT_DIR, p), recursive=True))
    files = sorted(set(files))
    docs = []
    for f in files:
        try:
            text = pathlib.Path(f).read_text(encoding="utf-8", errors="ignore")
            if text.strip():
                docs.append((f, text))
        except Exception:
            pass
    return docs

def main():
    os.makedirs(INDEX_DIR, exist_ok=True)
    model = SentenceTransformer(MODEL_NAME)

    meta = []
    vectors = []
    for path, text in tqdm(load_docs(), desc="Chunking/encoding"):
        for start, chunk in chunk_text(text):
            end = start + len(chunk)
            meta.append({"path": path, "start": start, "end": end, "preview": chunk[:240].replace("\n", " ")})
            vectors.append(model.encode(chunk, normalize_embeddings=True))  # cosine-ready

    if not vectors:
        print("No text found to index. Did you run totext.sh?")
        return

    X = np.vstack(vectors).astype("float32")
    dim = X.shape[1]
    index = faiss.IndexFlatIP(dim)  # inner product + normalized vectors = cosine
    index.add(X)
    faiss.write_index(index, INDEX_PATH)

    with open(META_PATH, "w", encoding="utf-8") as f:
        for m in meta:
            f.write(json.dumps(m, ensure_ascii=False) + "\n")

    print(f"Indexed {len(meta)} chunks into {INDEX_PATH}")
    print(f"Metadata written to {META_PATH}")

if __name__ == "__main__":
    main()

Run it:

python3 tools/index.py

Result:

• index/faiss.index: your vector index

• index/meta.jsonl: mapping of vectors to source files and locations

---

3) Query the knowledge base from Bash

A simple CLI that searches top-k chunks and prints sources. Optionally, send context to a local LLM (if you run one).

Save as tools/query.py:

#!/usr/bin/env python3
import os, json, sys, subprocess
import numpy as np
from sentence_transformers import SentenceTransformer
import faiss

INDEX_PATH = "index/faiss.index"
META_PATH = "index/meta.jsonl"
MODEL_NAME = "sentence-transformers/all-MiniLM-L6-v2"

def load_meta():
    meta = []
    with open(META_PATH, "r", encoding="utf-8") as f:
        for line in f:
            meta.append(json.loads(line))
    return meta

def answer_with_ollama(context, question, model):
    prompt = f"""You are a helpful assistant. Use ONLY the context to answer.

Question: {question}

Context:
{context}

Answer:"""
    try:
        out = subprocess.run(
            ["ollama", "run", model],
            input=prompt,
            text=True,
            check=True,
            stdout=subprocess.PIPE
        ).stdout
        return out.strip()
    except Exception as e:
        return f"(Ollama failed: {e})"

def main():
    if len(sys.argv) < 2:
        print("Usage: query.py 'your question here' [k]", file=sys.stderr)
        sys.exit(1)

    question = sys.argv[1]
    k = int(sys.argv[2]) if len(sys.argv) > 2 else 5

    index = faiss.read_index(INDEX_PATH)
    meta = load_meta()
    model = SentenceTransformer(MODEL_NAME)

    q = model.encode(question, normalize_embeddings=True).astype("float32")
    D, I = index.search(np.expand_dims(q, 0), k)
    hits = [meta[i] for i in I[0] if i < len(meta)]

    print("Top matches:")
    for rank, h in enumerate(hits, 1):
        print(f"{rank}. {h['path']} [{h['start']}:{h['end']}]")
        print(f"   {h['preview'][:200]}...")
        print()

    ollama_model = os.environ.get("OLLAMA_MODEL")
    if ollama_model:
        context = "\n\n".join([f"{h['path']}:\n{h['preview']}" for h in hits])
        print("LLM answer (via Ollama):")
        print(answer_with_ollama(context, question, ollama_model))

if __name__ == "__main__":
    main()

Usage:

python3 tools/query.py "How do I rotate nginx logs?" 5

Optional: If you have a local LLM via Ollama, you can ask it to generate an answer from the retrieved context:

# Example: use a small local model
export OLLAMA_MODEL=llama3
python3 tools/query.py "How do I rotate nginx logs?"

Note: Ollama provides a Linux install script on their site. This guide doesn’t require it to run.

---

4) Real-world patterns that work

• SRE/runbook KB: Point data/raw at your incident guides, postmortems, on-call docs, and service READMEs. Query “restart playbook for service X” or “what to check before failover?”

• Compliance/legal: Use contract PDFs and policy docs. Ask for “termination clause summary” or “password policy specifics.”

• Customer support: Index FAQ, troubleshooting trees, and release notes. Ask “why does feature Y fail after upgrade?”

• Developer enablement: Index architecture docs, ADRs, code guidelines. Ask “how do we add a new microservice?” or “logging standards?”

Quality tips:

• Chunk size 500–1,000 characters with 10–20% overlap works well.

• Keep sources deduplicated; avoid machine-generated noise.

• Make provenance visible by always showing file paths in answers.

---

5) Keep it fresh (automation)

Minimal cron job to re-run normalization and reindex nightly:

crontab -e

Add:

0 2 * * * cd /home/youruser/kb && . .venv/bin/activate && bash tools/totext.sh && python3 tools/index.py >> /var/log/kb-reindex.log 2>&1

You can also wire this into CI after docs change, or use a systemd timer for finer control.

---

Troubleshooting

• No matches found:

  • Check data/text has .txt output: ls -R data/text | head
  • Ensure converters are installed (pandoc, poppler, tesseract).

• Index small or empty:

  • Did totext.sh run before index.py?
  • Are your docs binary or images without OCR?

• Slow or memory heavy on huge corpora:

  • Consider FAISS IVF/HNSW indexes for scalability.
  • Shard by topic (one index per department) to keep queries snappy.

---

Wrap-up and next steps

You now have a private, scriptable AI knowledge base:

• Normalize docs to plain text

• Create embeddings and a vector index

• Query with provenance

• Optionally summarize with a local LLM

Next steps:

• Add a tiny web UI that shells out to query.py

• Enrich metadata (tags, owners) and filter by them

• Add a reranker (e.g., cross-encoder) for higher answer quality

• Gate PII and secrets before indexing

If this helped, try pointing it at your team’s most painful docs silo today. Ten minutes of setup can save hours of searching this week.