Posted on
• Artificial Intelligence

Artificial Intelligence Search on Linux

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence Search on Linux: Find Meaning, Not Just Keywords

Ever spent 10 minutes grepping through notes or logs only to miss the exact thing you needed because you didn’t use the “right” word? Traditional search is literal—great at exact matches, terrible at meaning. AI-powered search changes that: it retrieves content by concept and intent. On Linux, you can run it locally, privately, and from Bash.

This article shows you how to:

  • Build a tiny local semantic search tool that runs entirely on your Linux box

  • Query it from Bash as easily as grep

  • Scale to a vector database if your corpus grows

  • Install everything via apt, dnf, or zypper

No cloud, no API keys—just your shell and open-source tools.

Why AI search on Linux?

  • Keyword search breaks on synonyms and phrasing. “ssh keys” vs “public key authentication” shouldn’t be two different searches.

  • Semantic search uses embeddings (vector representations of text) to measure conceptual similarity, not string overlap.

  • Privacy and control: running locally keeps your docs, logs, and code on your machine.

  • It’s fast. With a lightweight model and an efficient index, queries are interactive even on laptops.

What you’ll build

  • Step 1: A self-contained Python+FAISS indexer that converts your files into embeddings.

  • Step 2: A CLI query tool that returns the most semantically relevant snippets with file:line references.

  • Step 3: Optional: Scale to a vector DB (Qdrant) for large datasets or multi-user setups.

  • Real-world examples: Search notes, code, or logs by idea, not just words.

Install prerequisites

You’ll need Python 3, a compiler toolchain, and OpenBLAS (for fast math). The commands differ by distro:

Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y python3 python3-venv python3-pip build-essential git curl jq libopenblas-dev

Fedora (dnf):

sudo dnf -y install python3 python3-pip gcc-c++ make git curl jq openblas-devel

openSUSE (zypper):

sudo zypper refresh
sudo zypper install -y python3 python3-pip gcc-c++ make git curl jq openblas-devel

Create and activate a virtual environment (recommended):

python3 -m venv ~/.venvs/aisearch
source ~/.venvs/aisearch/bin/activate
python3 -m pip install --upgrade pip
pip install "sentence-transformers<3" faiss-cpu rich tqdm qdrant-client

Notes:

  • The model we’ll use (all-MiniLM-L6-v2) runs well on CPU and produces 384‑dimensional embeddings.

  • If faiss-cpu wheel isn’t available for your arch, consider alternatives like annoy or use Docker for FAISS.

Step 1: Index your files semantically (local, offline)

Save this as ai_index.py:

#!/usr/bin/env python3
import argparse, os, json, sys, pathlib, math
from typing import List, Tuple
from tqdm import tqdm
import numpy as np
import faiss
from sentence_transformers import SentenceTransformer

DEFAULT_MODEL = "sentence-transformers/all-MiniLM-L6-v2"
EMBED_DIM = 384  # for MiniLM-L6-v2

SKIP_DIRS = {".git", ".hg", ".svn", "node_modules", "__pycache__", "target", "build", "dist", ".venv", "venv"}
ALLOW_EXT = {".md",".txt",".rst",".py",".sh",".c",".cpp",".h",".hpp",".go",".java",".js",".ts",".json",".yml",".yaml",".toml",".ini",".log",".conf",".sql",".rb",".rs",".lua",".php",".pl",".scala",".kt",".ipynb",".tex",".org"}

def iter_files(root: pathlib.Path):
    for dirpath, dirnames, filenames in os.walk(root):
        # prune
        dirnames[:] = [d for d in dirnames if d not in SKIP_DIRS and not d.startswith(".cache")]
        for f in filenames:
            p = pathlib.Path(dirpath) / f
            if p.suffix.lower() in ALLOW_EXT:
                yield p

def chunk_lines(text: str, window: int = 6, step: int = 6):
    lines = text.splitlines()
    for i in range(0, len(lines), step):
        chunk = lines[i:i+window]
        if chunk:
            yield i+1, "\n".join(chunk)

def build_index(root: pathlib.Path, outdir: pathlib.Path, model_name: str):
    outdir.mkdir(parents=True, exist_ok=True)
    model = SentenceTransformer(model_name)
    vecs = []
    metas = []

    files = list(iter_files(root))
    if not files:
        print("No files found to index.", file=sys.stderr)
        return

    for path in tqdm(files, desc="Indexing files"):
        try:
            text = path.read_text(encoding="utf-8", errors="ignore")
        except Exception:
            continue
        for start_line, chunk in chunk_lines(text, window=8, step=6):
            if chunk.strip():
                metas.append({"path": str(path), "line": start_line, "text": chunk[:500]})
                vecs.append(chunk)

    print(f"Encoding {len(vecs)} chunks...")
    embeddings = model.encode(vecs, batch_size=64, show_progress_bar=True, convert_to_numpy=True)
    # Normalize for cosine similarity and use Inner Product
    faiss.normalize_L2(embeddings)
    index = faiss.IndexFlatIP(EMBED_DIM)
    index.add(embeddings)

    faiss.write_index(index, str(outdir / "index.faiss"))
    with open(outdir / "meta.jsonl", "w", encoding="utf-8") as f:
        for m in metas:
            f.write(json.dumps(m, ensure_ascii=False) + "\n")
    with open(outdir / "model.json", "w", encoding="utf-8") as f:
        json.dump({"model": model_name, "dim": EMBED_DIM}, f)

    print(f"Index built: {len(metas)} chunks")
    print(f"Saved to: {outdir}")

def main():
    ap = argparse.ArgumentParser(description="Build a local semantic search index.")
    ap.add_argument("root", help="Directory to index")
    ap.add_argument("--out", default=".ai-index", help="Output directory for index files")
    ap.add_argument("--model", default=DEFAULT_MODEL, help="SentenceTransformer model")
    args = ap.parse_args()

    root = pathlib.Path(args.root).expanduser().resolve()
    outdir = pathlib.Path(args.out).expanduser().resolve()
    build_index(root, outdir, args.model)

if __name__ == "__main__":
    main()

Make it executable and run:

chmod +x ai_index.py
./ai_index.py ~/notes --out ~/.ai-search/notes-index

This walks your files, chunks them, encodes each chunk, and saves:

  • index.faiss: the vector index

  • meta.jsonl: file path, start line, and a snippet

  • model.json: model metadata

Step 2: Query from Bash like a pro

Save as ai_search.py:

#!/usr/bin/env python3
import argparse, json, pathlib, sys, textwrap
import numpy as np
import faiss
from sentence_transformers import SentenceTransformer

def load_meta(meta_path):
    metas = []
    with open(meta_path, "r", encoding="utf-8") as f:
        for line in f:
            metas.append(json.loads(line))
    return metas

def main():
    ap = argparse.ArgumentParser(description="Query a local semantic index")
    ap.add_argument("query", help="Your search query in natural language")
    ap.add_argument("--index", default="~/.ai-search/notes-index", help="Index directory")
    ap.add_argument("-k", type=int, default=8, help="Top K results")
    args = ap.parse_args()

    index_dir = pathlib.Path(args.index).expanduser().resolve()
    model_info = json.loads((index_dir / "model.json").read_text(encoding="utf-8"))
    model = SentenceTransformer(model_info["model"])

    index = faiss.read_index(str(index_dir / "index.faiss"))
    metas = load_meta(index_dir / "meta.jsonl")

    q = model.encode([args.query], convert_to_numpy=True)
    faiss.normalize_L2(q)
    D, I = index.search(q, args.k)  # cosine similarity due to normalization + IP

    for rank, (score, idx) in enumerate(zip(D[0], I[0]), start=1):
        m = metas[int(idx)]
        print(f"[{rank}] {m['path']}:{m['line']}  score={float(score):.3f}")
        print(textwrap.indent(m["text"].strip(), prefix="    "))
        print()

if __name__ == "__main__":
    main()

Try it:

./ai_search.py "how to rotate ssh keys" --index ~/.ai-search/notes-index

Optional: a tiny shell wrapper to jump to the best match in your editor:

ais() {
  local idx="${1:-$HOME/.ai-search/notes-index}"
  local q="${*:2}"
  local first
  first=$(./ai_search.py "$q" --index "$idx" -k 1 | awk 'NR==1 {print $2}')
  if [ -n "$first" ]; then
    local file="${first%%:*}"
    local line="${first##*:}"
    "${EDITOR:-vim}" "+${line}" "$file"
  else
    echo "No result"
  fi
}
# usage: ais ~/.ai-search/notes-index "explain systemd service wantedby"

Step 3: Real-world uses you can drop in today

  • Search logs by cause, not code: “database connection flapping” can surface TLS or DNS errors you didn’t keyword for.

  • Discover code by intent: “function that retries HTTP with backoff” finds relevant utilities across languages.

  • Mine notes and docs: “generate SSH key pair” yields chunks mentioning ssh-keygen, config examples, and key formats.

Tip: Rebuild the index after large changes:

./ai_index.py ~/code --out ~/.ai-search/code-index

Step 4: Scale out with Qdrant (vector database)

If your corpus grows into millions of chunks or you want multi-user search, use a vector DB. Qdrant is fast, open-source, and easy to run on Linux.

Install Qdrant:

Debian/Ubuntu (apt):

# Add Qdrant repo and key
curl -fsSL https://deb.qdrant.tech/qdrant_pub.gpg | sudo gpg --dearmor -o /usr/share/keyrings/qdrant.gpg
echo "deb [signed-by=/usr/share/keyrings/qdrant.gpg] https://deb.qdrant.tech/ qdrant main" | sudo tee /etc/apt/sources.list.d/qdrant.list
sudo apt update
sudo apt install -y qdrant

Fedora (dnf):

sudo rpm --import https://rpm.qdrant.tech/qdrant.gpg
sudo dnf config-manager --add-repo https://rpm.qdrant.tech/qdrant.repo
sudo dnf install -y qdrant

openSUSE (zypper):

sudo rpm --import https://rpm.qdrant.tech/qdrant.gpg
sudo zypper ar https://rpm.qdrant.tech/qdrant.repo qdrant
sudo zypper refresh
sudo zypper install -y qdrant

Start the service:

sudo systemctl enable --now qdrant
# default listens on http://localhost:6333

Create a collection (384 dims for MiniLM, cosine distance):

curl -X PUT "http://localhost:6333/collections/docs" \
  -H "Content-Type: application/json" \
  -d '{"vectors":{"size":384,"distance":"Cosine"}}'

Ingest your documents with the same model you used locally. Save as qdrant_ingest.py:

#!/usr/bin/env python3
import argparse, pathlib, os
from typing import List
from sentence_transformers import SentenceTransformer
from qdrant_client import QdrantClient
from qdrant_client.models import VectorParams, Distance, PointStruct
from tqdm import tqdm
import numpy as np

ALLOW_EXT = {".md",".txt",".py",".sh",".log",".json",".yml",".yaml",".toml",".ini",".rst",".conf"}

def iter_files(root: pathlib.Path):
    for dirpath, _, filenames in os.walk(root):
        if any(s in dirpath for s in [".git","node_modules",".venv","__pycache__"]):
            continue
        for f in filenames:
            p = pathlib.Path(dirpath) / f
            if p.suffix.lower() in ALLOW_EXT:
                yield p

def chunks(lines, window=8, step=6):
    for i in range(0, len(lines), step):
        chunk = lines[i:i+window]
        if chunk:
            yield i+1, "\n".join(chunk)

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("root")
    ap.add_argument("--collection", default="docs")
    ap.add_argument("--host", default="localhost")
    ap.add_argument("--port", type=int, default=6333)
    ap.add_argument("--batch", type=int, default=256)
    ap.add_argument("--model", default="sentence-transformers/all-MiniLM-L6-v2")
    args = ap.parse_args()

    model = SentenceTransformer(args.model)
    client = QdrantClient(host=args.host, port=args.port)

    # Ensure collection exists
    client.recreate_collection(
        collection_name=args.collection,
        vectors_config=VectorParams(size=384, distance=Distance.COSINE)
    )

    points = []
    pid = 0
    for path in tqdm(list(iter_files(pathlib.Path(args.root)))):
        text = path.read_text(encoding="utf-8", errors="ignore")
        ls = text.splitlines()
        for line, chunk in chunks(ls):
            payload = {"path": str(path), "line": line, "text": chunk[:700]}
            points.append((payload, chunk))
            if len(points) >= args.batch:
                payloads = [p[0] for p in points]
                embs = model.encode([p[1] for p in points], convert_to_numpy=True)
                # Qdrant expects raw (it will handle cosine), normalize optional
                client.upsert(
                    collection_name=args.collection,
                    points=[PointStruct(id=pid+i, vector=embs[i].tolist(), payload=payloads[i]) for i in range(len(points))]
                )
                pid += len(points)
                points = []
    # flush remaining
    if points:
        payloads = [p[0] for p in points]
        embs = model.encode([p[1] for p in points], convert_to_numpy=True)
        client.upsert(
            collection_name=args.collection,
            points=[PointStruct(id=pid+i, vector=embs[i].tolist(), payload=payloads[i]) for i in range(len(points))]
        )

if __name__ == "__main__":
    main()

Run it:

python3 qdrant_ingest.py ~/notes --collection docs

Query Qdrant with a natural-language string:

QUERY='how to rotate ssh keys'
# Encode the query to a vector using the same model in a quick Python one-liner:
python3 - <<'PY'
from sentence_transformers import SentenceTransformer
m=SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
import sys, json
import numpy as np
v = m.encode([sys.argv[1]], convert_to_numpy=True)[0].tolist()
print(json.dumps(v))
PY "$QUERY" > /tmp/qvec.json

# Search in Qdrant
curl -s "http://localhost:6333/collections/docs/points/search" \
  -H 'Content-Type: application/json' \
  -d @- <<EOF | jq
{ "vector": $(cat /tmp/qvec.json), "limit": 5 }
EOF

This returns payloads with path:line and snippets—ready for your editor or scripts.

Tips and gotchas

  • Reproducibility: Always encode and search with the same model. Switching models changes vector dimensions and meaning space.

  • Chunk size matters: Too small and you lose context; too big and results are vague. 6–10 lines is a good starting point for code and notes.

  • Performance: FAISS is fast; for very large corpora use HNSW/IVF indexes or offload to Qdrant.

  • Binary/noisy files: Use allowlists or .gitignore-style filters to keep your index clean.

Conclusion and next steps

You now have an AI-powered search that:

  • Runs locally on Linux

  • Finds concepts, not just strings

  • Integrates with your shell and editor

  • Scales to a vector database when you need it

Next steps:

  • Wire ai_search.py into your prompt or fzf workflow.

  • Maintain multiple indexes (notes, code, logs) under ~/.ai-search.

  • Experiment with models (e.g., bge-small, e5-small) for your domain.

  • Add reranking with a cross-encoder for even more precise top results.

Have an idea you want to search better? Point the indexer at that directory and try a “how do I…” query. Your future self will thank you.

Related posts