Posted on
• Artificial Intelligence

Artificial Intelligence Knowledge Bases

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Build an AI-Powered Knowledge Base on Linux (From Bash to Answers)

Ever lost 30 minutes grepping across wikis, READMEs, tickets, and logs just to answer, “How did we deploy staging last quarter?” Traditional search (and even good grep-fu) can’t always match the way humans ask. An AI knowledge base changes that: you ask in plain language, and it finds the relevant passages—fast.

In this guide, you’ll build a local, private AI-powered knowledge base on Linux using Bash, Python, and an open-source vector database. You’ll organize documents, embed them into vectors, index them, and query them from your terminal. No SaaS, no data leaving your machine.

You’ll get:

  • What an AI knowledge base is and why it matters for Linux users

  • A minimal, repeatable stack you can run locally or on a server

  • 3–5 actionable steps with real commands and scripts

  • A working example you can adapt today

What is an AI Knowledge Base?

An AI knowledge base uses embeddings (numerical representations of text) and a vector database to find content semantically—by meaning, not just keywords. This is the foundation of Retrieval-Augmented Generation (RAG): retrieve the right context, then optionally pass it to an LLM to generate a final answer.

Even without an LLM, semantic search beats traditional grep when:

  • The question’s phrasing doesn’t match exact keywords in your docs

  • You want best matches across many doc types (markdown, logs, configs)

  • You value speed and privacy over SaaS convenience

Why This Matters on Linux

  • Private-by-default: Your code and docs stay on your box or server

  • Scriptable: Glue everything together with Bash, cron, systemd, and CLI tools

  • Portable: Works headless on laptops, workstations, or VPS

  • Cost-effective: Open source end to end

What We’ll Build

  • A document directory you control (markdowns, text files, logs)

  • A local vector database (Qdrant) running in Docker or Podman

  • A Python indexer to embed and upsert docs into Qdrant

  • A Python CLI to ask questions and get relevant passages

  • Optional: a filesystem watcher to re-index on changes

1) Install prerequisites

You need Python, a container engine (Docker or Podman), and some CLI helpers. Pick your package manager.

Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y \
  python3 python3-venv python3-pip \
  git curl jq ripgrep inotify-tools \
  docker.io docker-compose-plugin podman \
  poppler-utils
# Optional: enable Docker for non-root use
sudo systemctl enable --now docker
sudo usermod -aG docker "$USER"
newgrp docker

Fedora/RHEL/CentOS Stream (dnf):

sudo dnf install -y \
  python3 python3-virtualenv python3-pip \
  git curl jq ripgrep inotify-tools \
  moby-engine docker-compose podman \
  poppler-utils
# Start and enable Docker (moby-engine)
sudo systemctl enable --now docker
sudo usermod -aG docker "$USER"
newgrp docker

openSUSE (zypper):

sudo zypper refresh
sudo zypper install -y \
  python3 python3-pip python3-virtualenv \
  git curl jq ripgrep inotify-tools \
  docker docker-compose podman \
  poppler-tools
sudo systemctl enable --now docker
sudo usermod -aG docker "$USER"
newgrp docker

Notes:

  • Docker and Podman are interchangeable for this guide; use whichever you prefer.

  • poppler-utils/tools provides pdftotext if you later want to index PDFs.

2) Start a local vector database (Qdrant)

Run Qdrant in a container. Use either Docker or Podman.

Using Docker:

mkdir -p ~/kb/qdrant_data
docker run -d --name qdrant \
  -p 6333:6333 \
  -v ~/kb/qdrant_data:/qdrant/storage \
  qdrant/qdrant:latest

Using Podman:

mkdir -p ~/kb/qdrant_data
podman run -d --name qdrant \
  -p 6333:6333 \
  -v ~/kb/qdrant_data:/qdrant/storage:Z \
  qdrant/qdrant:latest

Verify:

curl -s http://localhost:6333/ | jq

You should see a JSON response with the Qdrant version.

3) Prepare your documents

Pick a directory to store or mirror your knowledge sources.

export DOCS=~/kb/docs
mkdir -p "$DOCS"

Examples to populate:

  • Drop your Markdown docs, READMEs, meeting notes, and runbooks into $DOCS

  • Convert PDFs to text (optional):

    pdftotext -layout design-doc.pdf "$DOCS/design-doc.txt"

  • Collect logs or snippets:

    cp /var/log/myapp/*.log "$DOCS"/

Quick sanity check:

rg -n "" "$DOCS" | head

4) Create a Python environment and install libraries

We’ll use sentence-transformers for embeddings and qdrant-client for indexing.

cd ~/kb
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install qdrant-client sentence-transformers tqdm

Note: The first time you run the model, it’ll download weights (~80–100 MB).

5) Index your documents into Qdrant

Save this as index.py in ~/kb:

#!/usr/bin/env python3
import os
import argparse
import pathlib
import uuid
from tqdm import tqdm
from typing import List

from qdrant_client import QdrantClient
from qdrant_client.models import Distance, VectorParams, PointStruct
from sentence_transformers import SentenceTransformer

SUPPORTED_EXT = {".txt", ".md", ".rst", ".log", ".cfg", ".ini", ".conf", ".py", ".sh"}

def iter_files(root: str):
    for p in pathlib.Path(root).rglob("*"):
        if p.is_file() and p.suffix.lower() in SUPPORTED_EXT:
            yield p

def read_text(path: pathlib.Path) -> str:
    try:
        with open(path, "r", encoding="utf-8", errors="ignore") as f:
            return f.read()
    except Exception:
        return ""

def chunk(text: str, size: int = 500, overlap: int = 50) -> List[str]:
    chunks = []
    start = 0
    n = len(text)
    while start < n:
        end = min(n, start + size)
        chunks.append(text[start:end])
        start = end - overlap
        if start < 0:
            start = 0
        if start >= n:
            break
    return [c.strip() for c in chunks if c.strip()]

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--root", default=os.environ.get("DOCS", "./docs"), help="Docs root directory")
    ap.add_argument("--host", default="localhost")
    ap.add_argument("--port", type=int, default=6333)
    ap.add_argument("--collection", default="kb")
    args = ap.parse_args()

    # Model: small, fast, 384-dim
    model_name = "sentence-transformers/all-MiniLM-L6-v2"
    model = SentenceTransformer(model_name)

    client = QdrantClient(host=args.host, port=args.port)
    vector_size = 384  # all-MiniLM-L6-v2 dimension

    # Ensure collection exists
    collections = [c.name for c in client.get_collections().collections]
    if args.collection not in collections:
        client.recreate_collection(
            collection_name=args.collection,
            vectors_config=VectorParams(size=vector_size, distance=Distance.COSINE),
        )

    points = []
    files = list(iter_files(args.root))
    for path in tqdm(files, desc="Indexing files"):
        text = read_text(path)
        if not text.strip():
            continue
        # Simple heuristic: chunk by characters
        pieces = chunk(text)
        if not pieces:
            continue
        embs = model.encode(pieces, convert_to_numpy=True, show_progress_bar=False)
        for piece, vec in zip(pieces, embs):
            points.append(
                PointStruct(
                    id=str(uuid.uuid4()),
                    vector=vec.tolist(),
                    payload={"path": str(path), "text": piece[:10000]},
                )
            )
        # Upsert in batches to avoid huge payloads
        if len(points) >= 512:
            client.upsert(collection_name=args.collection, points=points)
            points = []

    if points:
        client.upsert(collection_name=args.collection, points=points)

    print(f"Indexed {len(files)} files into collection '{args.collection}'.")

if __name__ == "__main__":
    main()

Run it:

source ~/kb/.venv/bin/activate
python ~/kb/index.py --root "$DOCS"

6) Query your knowledge base from the CLI

Create ask.py in ~/kb:

#!/usr/bin/env python3
import argparse
from qdrant_client import QdrantClient
from sentence_transformers import SentenceTransformer

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("question", nargs="+", help="Your question")
    ap.add_argument("--host", default="localhost")
    ap.add_argument("--port", type=int, default=6333)
    ap.add_argument("--collection", default="kb")
    ap.add_argument("--top-k", type=int, default=5)
    args = ap.parse_args()

    question = " ".join(args.question)

    model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
    qvec = model.encode([question])[0].tolist()

    client = QdrantClient(host=args.host, port=args.port)
    results = client.search(
        collection_name=args.collection,
        query_vector=qvec,
        limit=args["top_k"] if isinstance(args, dict) and "top_k" in args else args.top_k,
        with_payload=True,
    )

    print(f"Top {args.top_k} matches for: {question}\n")
    for i, r in enumerate(results, 1):
        path = r.payload.get("path", "unknown")
        text = r.payload.get("text", "")[:400].replace("\n", " ")
        score = r.score
        print(f"[{i}] {path}  (score: {score:.4f})")
        print(f"    {text}")
        print()

if __name__ == "__main__":
    main()

Make it executable and ask something:

chmod +x ~/kb/ask.py
~/kb/ask.py "How do we rotate logs in production?"

Tip: Wrap it in a Bash function in your shell rc so you can just run askkb:

askkb() { ~/kb/ask.py "$@"; }
export -f askkb

7) Keep the index fresh (optional)

Automatically re-index when files change using inotify-tools:

while true; do
  inotifywait -r -e close_write,move,create,delete "$DOCS" && \
  source ~/kb/.venv/bin/activate && \
  python ~/kb/index.py --root "$DOCS"
done

Run this in a tmux pane or a systemd user service.

Real-World Example

  • Ops runbooks + logs: Ask “How do we roll back a failed deploy?” and get the exact passage from your runbook, plus recent log snippets.

  • Onboarding: “Where is the SSO config documented?” returns the team wiki MD file and the relevant section.

  • Developer memory: “What’s the difference between v2 and v3 of the API?” surfaces the changelog and PR notes.

Troubleshooting

  • No results or poor matches:

    • Ensure index ran without errors and files were detected
    • Increase chunk size to 800–1000 if your docs are very terse

  • Model download too slow:

    • Preload the model in an environment with internet, or configure a local model cache

  • Docker permission denied:

    • Add your user to the docker group and re-login: sudo usermod -aG docker "$USER"; newgrp docker
    • Or use Podman instead of Docker

Where to Go Next

  • Add an LLM for full answers: feed top 3–5 retrieved chunks into an LLM (local via llama.cpp or remote) to synthesize responses

  • Add more parsers: convert HTML, DOCX, and PDFs with dedicated tools before indexing

  • Reranking: use a smaller cross-encoder to reorder top hits for even better accuracy

  • Schedule indexing with cron or systemd timers

You now have a private, bash-friendly AI knowledge base, running locally, that understands meaning—not just keywords. Point it at your real docs and start asking better questions today.

Related posts