Posted on
• Artificial Intelligence

Knowledge Bases with Artificial Intelligence

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Build a Searchable Knowledge Base with AI on Linux (Bash-Friendly)

Ever spent 20 minutes digging through wikis, tickets, Slack threads, and repos for “that one answer”? Multiply that across a team and it’s hours of burn. The fix is simple and surprisingly light: turn your docs into a local, searchable knowledge base powered by AI embeddings. In this guide, you’ll build a private, on-disk KB you can query from the terminal, with optional LLM answers. No vendor lock-in, no data exfiltration—just Linux, Bash, and open tools.

What you’ll get:

  • A fast local KB using embeddings + vector search (RAG-friendly)

  • Bash commands to ingest and query your docs

  • Optional answer generation via a local or hosted LLM

  • Clear install instructions for apt, dnf, and zypper

Why AI-powered knowledge bases are worth it

  • Traditional search is string-matching; it fails on paraphrases and synonyms. Embeddings understand meaning.

  • RAG (Retrieval-Augmented Generation) gives LLMs the right context at answer time, reducing hallucinations and keeping knowledge fresh.

  • You control your data. Keep the whole pipeline local, audit what’s indexed, and version it like code.

  • It scales from a single dev’s laptop to team servers with the same tooling.

What we’ll build

  • Storage: ChromaDB (simple, file-based vector DB)

  • Embeddings: sentence-transformers (all-MiniLM-L6-v2, CPU-friendly)

  • Ingestion: a Python script that chunks and indexes text, Markdown, PDFs, configs, and code

  • Query: a Python script that returns the most relevant passages, plus optional LLM answer generation

  • Bash wrappers so you can run kb-ingest and kb-ask from anywhere

Prerequisites (install via your package manager)

We’ll use Python, Git, jq, Lynx (for HTML to text), and pdftotext.

  • Debian/Ubuntu (apt):
sudo apt update
sudo apt install -y python3 python3-venv python3-pip git jq lynx poppler-utils
  • Fedora/RHEL/CentOS (dnf):
sudo dnf install -y python3 python3-pip git jq lynx poppler-utils
  • openSUSE (zypper):
sudo zypper refresh
sudo zypper install -y python3 python3-pip git jq lynx poppler-tools

Note: poppler-utils (apt/dnf) and poppler-tools (zypper) provide pdftotext.

Step 1 — Set up a workspace

mkdir -p ~/kb/{kb_src,kb_store}
cd ~/kb
python3 -m venv .venv
. .venv/bin/activate
pip install --upgrade pip

Install Python packages:

pip install chromadb sentence-transformers pypdf tqdm

Optional (for OpenAI-compatible answer generation):

pip install openai

Step 2 — Prepare some sources to ingest

Drop files in ~/kb/kb_src. You can pull in docs from your repos, wiki exports, HTML, and PDFs.

  • Convert HTML to text (example):
lynx -dump -nolist https://example.com/docs > ~/kb/kb_src/example_docs.txt
  • Extract text from PDFs (batch):
find /path/to/pdfs -type f -name '*.pdf' -print0 | while IFS= read -r -d '' f; do
  out=~/kb/kb_src/$(basename "${f%.pdf}").txt
  pdftotext -layout "$f" "$out"
done
  • Capture man pages for quick CLI knowledge:
for m in ls grep awk sed find tar; do
  man -P cat "$m" | col -b > ~/kb/kb_src/${m}.txt
done

Or just copy your project’s README.md, docs/, design/, migrations/, and config files into kb_src.

Step 3 — Ingest: chunk, embed, and index

Create ingest.py in ~/kb:

#!/usr/bin/env python3
import os, sys, time, hashlib
from typing import List, Tuple
from tqdm import tqdm
from pypdf import PdfReader
import chromadb
from chromadb.utils import embedding_functions

PERSIST_DIR = os.environ.get("KB_STORE", "kb_store")
SRC_DIR = sys.argv[1] if len(sys.argv) > 1 else os.environ.get("KB_SRC", "kb_src")

ALLOWED_EXT = {".txt",".md",".rst",".log",".conf",".ini",".yaml",".yml",".json",".py",".sh",".cfg",".toml",".htm",".html",".pdf"}

def read_file(path: str) -> str:
    ext = os.path.splitext(path)[1].lower()
    try:
        if ext == ".pdf":
            reader = PdfReader(path)
            return "\n".join((p.extract_text() or "") for p in reader.pages)
        else:
            with open(path, "r", encoding="utf-8", errors="ignore") as f:
                return f.read()
    except Exception as e:
        return ""

def chunk_text(text: str, size: int = 800, overlap: int = 120) -> List[str]:
    text = " ".join(text.split())
    if not text:
        return []
    chunks = []
    i = 0
    while i < len(text):
        chunk = text[i:i+size]
        chunks.append(chunk)
        i += size - overlap
    return chunks

def doc_id(path: str, chunk_idx: int) -> str:
    stat = os.stat(path)
    payload = f"{os.path.abspath(path)}::{int(stat.st_mtime)}::{chunk_idx}".encode()
    return hashlib.sha1(payload).hexdigest()

def collect_files(root: str) -> List[str]:
    files = []
    for dirpath, _, filenames in os.walk(root):
        for fn in filenames:
            ext = os.path.splitext(fn)[1].lower()
            if ext in ALLOWED_EXT:
                files.append(os.path.join(dirpath, fn))
    return files

def main():
    os.makedirs(PERSIST_DIR, exist_ok=True)
    client = chromadb.PersistentClient(path=PERSIST_DIR)
    embed_fn = embedding_functions.SentenceTransformerEmbeddingFunction(model_name="all-MiniLM-L6-v2")
    col = client.get_or_create_collection(name="kb", embedding_function=embed_fn, metadata={"hnsw:space": "cosine"})

    files = collect_files(SRC_DIR)
    if not files:
        print(f"No files found under {SRC_DIR}")
        return

    to_upsert = {"ids": [], "documents": [], "metadatas": []}

    for f in tqdm(files, desc="Indexing"):
        txt = read_file(f)
        chunks = chunk_text(txt)
        for i, ch in enumerate(chunks):
            to_upsert["ids"].append(doc_id(f, i))
            to_upsert["documents"].append(ch)
            to_upsert["metadatas"].append({"source": os.path.abspath(f), "chunk": i})

        # Batch upserts to avoid huge payloads
        if len(to_upsert["ids"]) >= 1000:
            col.upsert(**to_upsert)
            to_upsert = {"ids": [], "documents": [], "metadatas": []}

    if to_upsert["ids"]:
        col.upsert(**to_upsert)

    print(f"Done. Store at: {os.path.abspath(PERSIST_DIR)}")

if __name__ == "__main__":
    main()

Run it:

cd ~/kb
. .venv/bin/activate
python ingest.py  # or: KB_SRC=/some/path python ingest.py

Re-run after you update docs; upserts are idempotent because we hash file path + mtime + chunk.

Step 4 — Query the KB (and optionally generate answers)

Create query.py:

#!/usr/bin/env python3
import os, sys, textwrap
import chromadb
from chromadb.utils import embedding_functions

PERSIST_DIR = os.environ.get("KB_STORE", "kb_store")

def main():
    if len(sys.argv) < 2:
        print("Usage: python query.py \"your question here\" [N_RESULTS]")
        sys.exit(1)

    question = sys.argv[1]
    n_results = int(sys.argv[2]) if len(sys.argv) > 2 else 5

    client = chromadb.PersistentClient(path=PERSIST_DIR)
    embed_fn = embedding_functions.SentenceTransformerEmbeddingFunction(model_name="all-MiniLM-L6-v2")
    col = client.get_or_create_collection(name="kb", embedding_function=embed_fn)

    res = col.query(query_texts=[question], n_results=n_results, include=["documents","metadatas","distances"])
    docs = res["documents"][0]
    metas = res["metadatas"][0]
    dists = res["distances"][0]

    print(f"\nQ: {question}\n")
    for i, (doc, meta, dist) in enumerate(zip(docs, metas, dists), 1):
        snippet = (doc[:400] + "…") if len(doc) > 400 else doc
        print(f"[{i}] {meta.get('source','?')} (score={1-dist:.3f})")
        print(textwrap.fill(snippet, width=100, subsequent_indent="    "))
        print()

    # Hint to pipe into an LLM if you want generation:
    print("Tip: To generate an answer with an LLM, pass these snippets to your model (e.g., Ollama or OpenAI).")

if __name__ == "__main__":
    main()

Query it:

python query.py "How do I deploy the service to staging?" 5

Optional: generate an answer with a local LLM via Ollama (if installed and running):

  • Install Ollama (their installer detects your distro and uses apt/dnf under the hood):
curl -fsSL https://ollama.com/install.sh | sh

Pull a model and ask with retrieved context:

CTX="$(python query.py 'How do I deploy the service to staging?' 4 | sed -n '/^\[/,$p')"

curl -s http://localhost:11434/api/generate -d "{
  \"model\": \"llama3\",
  \"prompt\": \"You are a helpful assistant. Using only the context below, answer the question.\\n\\nContext:\\n$CTX\\n\\nQuestion: How do I deploy the service to staging?\\nAnswer:\"
}" | jq -r '.response' | sed '/^$/q'

Or use an OpenAI-compatible endpoint (set OPENAI_API_KEY and optionally OPENAI_BASE_URL):

python - <<'PY'
import os, sys, textwrap, subprocess, json
from openai import OpenAI
q = "How do I deploy the service to staging?"
ctx = subprocess.check_output(["python","query.py",q,"4"], text=True)
prompt = f"Using only the context below, answer the question.\n\nContext:\n{ctx}\n\nQuestion: {q}\nAnswer:"
client = OpenAI()
resp = client.chat.completions.create(model=os.getenv("MODEL","gpt-4o-mini"),
    messages=[{"role":"user","content":prompt}],
    temperature=0.2)
print(resp.choices[0].message.content)
PY

Step 5 — Make it Bash-friendly

Add handy wrappers to your shell:

cat >> ~/.bashrc <<'SH'
kb-venv() { . ~/kb/.venv/bin/activate; }
kb-ingest() { (cd ~/kb && kb-venv && KB_SRC="${1:-$HOME/kb/kb_src}" python ingest.py "$KB_SRC"); }
kb-ask() { (cd ~/kb && kb-venv && python query.py "$@"); }
SH
. ~/.bashrc

Usage:

kb-ingest ~/projects/my-repo/docs
kb-ask "Where is the migration guide?"

Real-world examples and tips

  • Project bootstrap: Ingest README.md, docs/, docs/architecture/, k8s/, and terraform/ to answer on-call questions fast.

  • Platform notes: Ingest runbooks, incident postmortems, and dashboards notes. Add your team’s Slack exports after converting to text.

  • CLI helpdesk: Store common man pages and internal scripts; query usage with natural language.

  • Keep it fresh: Add a cron to reindex nightly:

crontab -e
# Reindex at 2:30AM
30 2 * * * bash -lc 'kb-ingest ~/projects/my-repo/docs >/tmp/kb_ingest.log 2>&1'
  • Quality check: Maintain a small “golden questions” file and verify the top-3 passages contain the answer. Adjust chunk size/overlap if needed.

Alternatives and extensions

  • Vector stores: Chroma is simple. For multi-user or distributed setups, look at Qdrant or Weaviate (both offer Docker images).

  • Pipelines: Haystack 2 (pip install haystack-ai) or LangChain can orchestrate complete RAG flows.

  • Models: Swap embedding models in SentenceTransformerEmbeddingFunction (e.g., all-MiniLM-L12-v2) for better recall vs. speed.

Conclusion and next steps (CTA)

You now have a private, fast, Linux-native knowledge base that actually understands your content. Start by indexing one repo or wiki export and ask three painful questions the old search never answered. If the passages look right, wire in an LLM for full RAG answers and invite your team to try it.

Next steps:

  • Turn your KB into a service: add a FastAPI endpoint and a tiny TUI/CLI.

  • Evaluate regularly: keep a “golden set” of queries and measure retrieval quality as docs grow.

  • Scale out: move to a server, schedule ingests, and add role-based access controls if needed.

If you want a follow-up post with a FastAPI wrapper, Dockerfile, and a systemd unit to run this as a service, let me know.

Related posts