Posted on
• Artificial Intelligence

Artificial Intelligence Knowledge Bases on Linux

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence Knowledge Bases on Linux: From Zero to Queryable

Ever grepped a directory of PDFs at 2 a.m. and still couldn’t find that one snippet from the runbook? Traditional search breaks down when your docs are large, scattered, or poorly tagged. AI-powered knowledge bases solve this by letting you ask questions in natural language and retrieve semantically relevant passages—fast, locally, and privately on Linux.

This guide shows you why AI knowledge bases fit perfectly on Linux and walks you through a practical, bash-friendly setup from scratch. You’ll build a lightweight knowledge base you can query from your terminal, then see how to scale it with a vector database. All commands include apt, dnf, and zypper variants.

Why AI Knowledge Bases on Linux?

  • Privacy and control: Keep sensitive documents on your machine. No vendor lock-in, no cloud dependencies.

  • Scriptable and automatable: Cron, systemd, pipelines—Linux makes it easy to keep indexes fresh.

  • Performance and reliability: Resource-efficient servers and CLIs are Linux’s home turf.

  • Real outcomes: Faster on-call responses, quicker onboarding, and less time hunting through wikis and logs.

What you’ll build

  • A local, persistent knowledge base using Python, sentence-transformers (for embeddings), and Chroma (a simple vector store).

  • Optional: The same knowledge base backed by a standalone vector DB (Qdrant) running in a container (Podman or Docker).

  • Real-world ingestion examples: PDFs, Markdown, man pages, and a cron-able workflow.

1) Prerequisites: Install the essentials

You’ll need Python, a virtual environment, build tools, and utilities to extract text from PDFs.

  • Debian/Ubuntu (apt):
sudo apt update
sudo apt install -y python3 python3-venv python3-pip git build-essential pkg-config poppler-utils
  • Fedora/RHEL (dnf):
sudo dnf install -y python3 python3-virtualenv python3-pip git gcc gcc-c++ make pkgconfig poppler-utils
  • openSUSE Leap/Tumbleweed (zypper):
sudo zypper refresh
sudo zypper install -y python3 python3-venv python3-pip git gcc gcc-c++ make pkg-config poppler-tools

Create and activate a virtual environment:

python3 -m venv ~/.venvs/kb
source ~/.venvs/kb/bin/activate
python -m pip install --upgrade pip

Install Python packages for the local knowledge base:

pip install "sentence-transformers" "chromadb" "tqdm"

Optional but recommended if you plan to use a vector database:

pip install "qdrant-client"

2) Build a tiny, fast, private knowledge base (Chroma + sentence-transformers)

Below is a single-file script that:

  • Walks a directory of documents (txt, md, pdf),

  • Extracts and chunks text,

  • Generates embeddings with a small, fast model,

  • Stores them locally in a Chroma index,

  • Lets you run semantic queries.

Save as kb.py:

#!/usr/bin/env python3
import argparse, os, sys, subprocess, uuid, json
from pathlib import Path
from typing import List, Iterable
from tqdm import tqdm

import chromadb
from sentence_transformers import SentenceTransformer

def extract_text(path: Path) -> str:
    ext = path.suffix.lower()
    if ext in [".txt", ".md"]:
        return path.read_text(errors="ignore")
    if ext == ".pdf":
        # Uses poppler-utils (pdftotext)
        try:
            out = subprocess.check_output(
                ["pdftotext", "-layout", "-nopgbrk", str(path), "-"],
                stderr=subprocess.DEVNULL
            )
            return out.decode("utf-8", errors="ignore")
        except Exception as e:
            sys.stderr.write(f"Failed to extract PDF {path}: {e}\n")
            return ""
    return ""

def chunk(text: str, size: int = 700, overlap: int = 150) -> List[str]:
    words = text.split()
    chunks = []
    i = 0
    while i < len(words):
        window = words[i:i+size]
        chunks.append(" ".join(window))
        i += max(1, size - overlap)
    return [c for c in chunks if c.strip()]

def discover_files(root: Path) -> Iterable[Path]:
    for p in root.rglob("*"):
        if p.suffix.lower() in [".txt", ".md", ".pdf"]:
            yield p

def ensure_collection(client, name: str):
    # We’ll pass embeddings manually to keep control over the model
    try:
        return client.get_collection(name=name)
    except:
        return client.create_collection(name=name)

def ingest(docs_dir: Path, db_path: Path, model_name: str, collection_name: str):
    client = chromadb.PersistentClient(path=str(db_path))
    col = ensure_collection(client, collection_name)
    model = SentenceTransformer(model_name)
    files = list(discover_files(docs_dir))
    if not files:
        print(f"No documents found under {docs_dir}")
        return

    all_texts, all_metas, all_ids = [], [], []
    for f in tqdm(files, desc="Reading"):
        txt = extract_text(f)
        if not txt.strip():
            continue
        for idx, ch in enumerate(chunk(txt)):
            all_texts.append(ch)
            all_metas.append({"source": str(f), "chunk": idx})
            all_ids.append(str(uuid.uuid4()))

    # Batch to control memory usage
    batch = 512
    for i in tqdm(range(0, len(all_texts), batch), desc="Indexing"):
        batch_texts = all_texts[i:i+batch]
        batch_metas = all_metas[i:i+batch]
        batch_ids   = all_ids[i:i+batch]
        embs = model.encode(batch_texts, normalize_embeddings=True, show_progress_bar=False).tolist()
        col.add(documents=batch_texts, metadatas=batch_metas, embeddings=embs, ids=batch_ids)

    print(f"Ingested {len(all_texts)} chunks into collection '{collection_name}' at {db_path}")

def query(db_path: Path, model_name: str, collection_name: str, q: str, k: int):
    client = chromadb.PersistentClient(path=str(db_path))
    col = client.get_collection(collection_name)
    model = SentenceTransformer(model_name)
    q_emb = model.encode([q], normalize_embeddings=True).tolist()
    res = col.query(query_embeddings=q_emb, n_results=k)
    hits = []
    for doc, meta in zip(res.get("documents", [[]])[0], res.get("metadatas", [[]])[0]):
        hits.append({"source": meta.get("source"), "chunk": meta.get("chunk"), "text": doc})
    print(json.dumps(hits, indent=2, ensure_ascii=False))

def main():
    ap = argparse.ArgumentParser(description="Local AI Knowledge Base (Chroma)")
    ap.add_argument("--db", default="kb_chroma", help="Path for Chroma persistence")
    ap.add_argument("--collection", default="docs", help="Collection name")
    ap.add_argument("--model", default="sentence-transformers/all-MiniLM-L6-v2", help="Embedding model")
    sub = ap.add_subparsers(dest="cmd", required=True)

    ing = sub.add_parser("ingest", help="Ingest a directory of documents")
    ing.add_argument("path", help="Directory with txt/md/pdf files")

    qu = sub.add_parser("query", help="Run a semantic query")
    qu.add_argument("q", help="Your question or search text")
    qu.add_argument("-k", type=int, default=5, help="Top-K results")

    args = ap.parse_args()
    if args.cmd == "ingest":
        ingest(Path(args.path).expanduser().resolve(), Path(args.db), args.model, args.collection)
    else:
        query(Path(args.db), args.model, args.collection, args.q, args.k)

if __name__ == "__main__":
    main()

Make it executable and ingest your docs:

chmod +x kb.py
./kb.py ingest ~/docs

Query it:

./kb.py query "How do I restart the service safely?" -k 3

You’ll get a JSON list of top matches containing the most relevant passages and their source files.

Tip: Persist kb_chroma somewhere backed up (e.g., under ~/.local/share/kb_chroma) to keep indexes across runs.

3) Real-world ingestion recipes

  • Index all man pages:
mkdir -p ~/kb/man
for m in $(compgen -A command); do man -P cat "$m" 2>/dev/null | col -b > ~/kb/man/"$m".txt || true; done
./kb.py ingest ~/kb/man
  • Index PDFs from a shared drive:
./kb.py ingest /mnt/share/runbooks
  • Keep it fresh with cron (runs at 2:30 a.m. daily):
( crontab -l 2>/dev/null; echo "30 2 * * * source ~/.venvs/kb/bin/activate && ~/kb/kb.py ingest ~/docs >> ~/kb/reindex.log 2>&1" ) | crontab -

4) Scale or centralize: Run Qdrant as a vector database

When your corpus grows or you want multiple clients querying the same index, switch to a vector DB. Qdrant is fast, open-source, and easy to run in a container.

Install a container engine (Podman or Docker):

  • Podman

    • apt:
    sudo apt update
sudo apt install -y podman
    • dnf:
    sudo dnf install -y podman
    • zypper:
    sudo zypper install -y podman

  • Docker

    • apt:
    sudo apt update
sudo apt install -y docker.io
sudo systemctl enable --now docker
    • dnf (Fedora’s upstream engine is moby):
    sudo dnf install -y moby-engine
sudo systemctl enable --now docker
    • zypper:
    sudo zypper install -y docker
sudo systemctl enable --now docker

Run Qdrant:

  • Podman:
podman run -d --name qdrant -p 6333:6333 qdrant/qdrant:latest
  • Docker:
sudo docker run -d --name qdrant -p 6333:6333 qdrant/qdrant:latest

Ingest and query with Python (qdrant-client):

#!/usr/bin/env python3
import os, uuid, subprocess
from pathlib import Path
from sentence_transformers import SentenceTransformer
from qdrant_client import QdrantClient
from qdrant_client.http.models import VectorParams, Distance, PointStruct

QDRANT_URL = os.environ.get("QDRANT_URL", "http://localhost:6333")
COLLECTION  = os.environ.get("QDRANT_COLLECTION", "docs")
MODEL_NAME  = os.environ.get("EMB_MODEL", "sentence-transformers/all-MiniLM-L6-v2")

def extract_text(path: Path) -> str:
    if path.suffix.lower() in [".txt", ".md"]:
        return path.read_text(errors="ignore")
    if path.suffix.lower() == ".pdf":
        return subprocess.check_output(["pdftotext", "-layout", "-nopgbrk", str(path), "-"]).decode("utf-8", errors="ignore")
    return ""

def chunk(text, size=700, overlap=150):
    words, i, chunks = text.split(), 0, []
    while i < len(words):
        chunks.append(" ".join(words[i:i+size]))
        i += max(1, size-overlap)
    return [c for c in chunks if c.strip()]

def ensure_collection(qc: QdrantClient, dim: int):
    cols = [c.name for c in qc.get_collections().collections]
    if COLLECTION not in cols:
        qc.create_collection(
            collection_name=COLLECTION,
            vectors_config=VectorParams(size=dim, distance=Distance.COSINE)
        )

def ingest_dir(d):
    model = SentenceTransformer(MODEL_NAME)
    qc = QdrantClient(url=QDRANT_URL)
    ensure_collection(qc, model.get_sentence_embedding_dimension())

    points = []
    for p in Path(d).rglob("*"):
        if p.suffix.lower() not in [".txt", ".md", ".pdf"]:
            continue
        txt = extract_text(p)
        for idx, ch in enumerate(chunk(txt)):
            vec = model.encode(ch, normalize_embeddings=True).tolist()
            points.append(PointStruct(id=str(uuid.uuid4()), vector=vec, payload={"source": str(p), "chunk": idx, "text": ch}))
            if len(points) >= 256:
                qc.upsert(collection_name=COLLECTION, points=points); points = []
    if points:
        qc.upsert(collection_name=COLLECTION, points=points)

def query(q, k=5):
    model = SentenceTransformer(MODEL_NAME)
    qc = QdrantClient(url=QDRANT_URL)
    vec = model.encode(q, normalize_embeddings=True).tolist()
    hits = qc.search(collection_name=COLLECTION, query_vector=vec, limit=k, with_payload=True)
    for h in hits:
        print(f"[{h.score:.3f}] {h.payload.get('source')}#{h.payload.get('chunk')}\n{h.payload.get('text')}\n---")

if __name__ == "__main__":
    import sys
    if len(sys.argv) >= 3 and sys.argv[1] == "ingest":
        ingest_dir(sys.argv[2])
    elif len(sys.argv) >= 2 and sys.argv[1] == "query":
        q = " ".join(sys.argv[2:]) if len(sys.argv) > 2 else input("Query: ")
        query(q)
    else:
        print("Usage: qdrant_kb.py ingest <DIR> | query <TEXT>")

Usage:

python qdrant_kb.py ingest ~/docs
python qdrant_kb.py query "rollback steps for database migration"

This centralizes your KB and opens the door to multi-user access, high recall at scale, and easier horizontal growth.

5) Actionable best practices

  • Normalize early: Strip boilerplate (menus, footers) before indexing to improve relevance.

  • Chunk wisely: 500–800 words with 20–30% overlap often balances context and precision.

  • Persist everything: Keep the raw docs, chunked artifacts, and the vector store in known locations. Version them if possible.

  • Automate refresh: Cron or systemd timers to reindex nightly. Hash files to skip unchanged content.

  • Add a thin CLI: Wrap queries in a bash function that pretty-prints top hits so you and your team actually use it.

Example bash function:

kbq () {
  source ~/.venvs/kb/bin/activate >/dev/null 2>&1 || return 1
  ./kb.py query "$*" -k 5 | jq -r '.[] | "\(.source):\n" + (.text | gsub("\n"; " ") | .[0:300]) + "…\n"'
}

Conclusion and next steps

You now have a local, private AI knowledge base on Linux—and a path to scale it. Start by indexing a single directory that slows you down today, then iterate:

  • Add your wiki export and runbooks.

  • Wrap queries in a CLI and put it on your PATH.

  • If your corpus grows, switch the backend to Qdrant.

  • Want conversational answers? Add a local LLM and wire up Retrieval-Augmented Generation (RAG) on top of your KB.

Your docs shouldn’t be a graveyard. Turn them into a system you can actually ask. Try it this week and reclaim your search time.

Related posts