Posted on
• Artificial Intelligence

Semantic Search on Linux

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Semantic Search on Linux: When grep Knows What You Mean

Ever been sure a snippet existed somewhere in your notes or code, but you couldn’t remember the exact words to grep for? Traditional tools like grep/ripgrep are fast and fantastic—when you know the keywords. But language is messy. We say “sign in,” “login,” “authenticate,” or “SAML.” Semantic search helps your shell find meaning, not just matching strings.

This guide shows how to add private, offline semantic search to your Linux workflow using a tiny Python script and SQLite—no external services required. You’ll learn why it’s useful, how to install it on Debian/Ubuntu, Fedora/RHEL, and openSUSE, and how to index and search your own files from the terminal.

Why semantic search on Linux?

  • Find by concept, not exact phrasing. “passwordless ssh” still finds docs describing SSH keys and config, even if “passwordless” isn’t there.

  • Keep it local and private. Your notes, code, and logs never leave your machine.

  • Plays nice with Bash. Pipe results to fzf, open in $EDITOR, or wire it into cron/systemd for automatic indexing.

What you’ll build

  • A single-file CLI, semsearch.py, that:

    • Indexes a folder of text, Markdown, and code files.
    • Builds vector embeddings for chunks and stores them in SQLite.
    • Answers queries with semantically similar chunks (using cosine similarity).

  • A lightweight embedding backend (fastembed) that runs on CPU and caches models locally.

The first run will download a small model to your cache; subsequent runs are fully offline.

1) Install prerequisites

Pick the commands for your distro.

  • Debian/Ubuntu (apt):

    sudo apt update
sudo apt install -y python3 python3-venv python3-pip sqlite3 curl git fzf ripgrep

  • Fedora/RHEL/CentOS Stream (dnf):

    sudo dnf install -y python3 python3-virtualenv python3-pip sqlite curl git fzf ripgrep

  • openSUSE Leap/Tumbleweed (zypper):

    sudo zypper refresh
sudo zypper install -y python3 python3-virtualenv python3-pip sqlite3 curl git fzf ripgrep

Create and activate a virtual environment, then install Python packages:

python3 -m venv ~/.local/semsearch
~/.local/semsearch/bin/pip install -U pip fastembed numpy tqdm

Tip: Add this venv to your PATH for convenience:

echo 'export PATH="$HOME/.local/semsearch/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc

2) Save the script

Save the file below as semsearch.py and make it executable (chmod +x semsearch.py).

#!/usr/bin/env python3
import argparse
import os
import sys
import sqlite3
import time
from pathlib import Path
from typing import List, Tuple, Iterable

import numpy as np
from tqdm import tqdm

try:
    from fastembed import TextEmbedding
except ImportError:
    print("fastembed not found. Install with: pip install fastembed numpy tqdm", file=sys.stderr)
    sys.exit(1)

DEFAULT_DB = os.path.expanduser("~/.cache/semsearch.sqlite")
DEFAULT_MODEL = "BAAI/bge-small-en-v1.5"
DEFAULT_EXTS = [".txt", ".md", ".rst", ".org", ".log", ".py", ".sh", ".c", ".cpp", ".h", ".go", ".rs", ".java", ".js", ".ts", ".yaml", ".yml", ".toml"]

CHUNK_LINES = 20  # lines per chunk


def connect(db_path: str):
    os.makedirs(os.path.dirname(db_path), exist_ok=True)
    conn = sqlite3.connect(db_path)
    conn.execute("PRAGMA journal_mode=WAL;")
    conn.execute("PRAGMA synchronous=NORMAL;")
    conn.execute("""
    CREATE TABLE IF NOT EXISTS files(
        id INTEGER PRIMARY KEY,
        path TEXT UNIQUE,
        mtime REAL
    );
    """)
    conn.execute("""
    CREATE TABLE IF NOT EXISTS chunks(
        id INTEGER PRIMARY KEY,
        file_id INTEGER,
        chunk_index INTEGER,
        line_start INTEGER,
        line_end INTEGER,
        text TEXT,
        embedding BLOB,
        FOREIGN KEY(file_id) REFERENCES files(id)
    );
    """)
    conn.execute("CREATE INDEX IF NOT EXISTS idx_chunks_file ON chunks(file_id);")
    return conn


def list_files(root: str, exts: List[str]) -> Iterable[Path]:
    rootp = Path(root)
    for p in rootp.rglob("*"):
        if p.is_file() and p.suffix.lower() in exts:
            yield p


def read_text(path: Path) -> str:
    try:
        data = path.read_bytes()
        return data.decode("utf-8", errors="ignore")
    except Exception:
        return ""


def chunk_text(text: str, lines_per_chunk: int = CHUNK_LINES) -> List[Tuple[int, int, str]]:
    lines = text.splitlines()
    chunks = []
    i = 0
    while i < len(lines):
        start = i
        end = min(i + lines_per_chunk, len(lines))
        chunk = "\n".join(lines[start:end]).strip()
        if chunk:
            chunks.append((start + 1, end, chunk))
        i = end
    return chunks


def l2_normalize(mat: np.ndarray, axis: int = 1, eps: float = 1e-9) -> np.ndarray:
    norms = np.linalg.norm(mat, axis=axis, keepdims=True)
    norms = np.maximum(norms, eps)
    return mat / norms


def get_model(model_name: str = DEFAULT_MODEL):
    return TextEmbedding(model_name=model_name)


def upsert_file(conn, path: str, mtime: float) -> int:
    cur = conn.cursor()
    cur.execute("INSERT INTO files(path, mtime) VALUES(?, ?) ON CONFLICT(path) DO UPDATE SET mtime=excluded.mtime", (path, mtime))
    conn.commit()
    cur.execute("SELECT id FROM files WHERE path = ?", (path,))
    return cur.fetchone()[0]


def clear_file_chunks(conn, file_id: int):
    conn.execute("DELETE FROM chunks WHERE file_id = ?", (file_id,))
    conn.commit()


def index_dir(root: str, db_path: str, exts: List[str], model_name: str):
    conn = connect(db_path)
    model = get_model(model_name)
    paths = list(list_files(root, exts))
    print(f"Discovered {len(paths)} files to consider.")

    for p in tqdm(paths, desc="Indexing"):
        mtime = p.stat().st_mtime
        cur = conn.cursor()
        cur.execute("SELECT mtime, id FROM files WHERE path = ?", (str(p),))
        row = cur.fetchone()
        if row and abs(row[0] - mtime) < 1e-6:
            # unchanged file; skip
            continue

        text = read_text(p)
        chunks = chunk_text(text, CHUNK_LINES)
        file_id = upsert_file(conn, str(p), mtime)
        clear_file_chunks(conn, file_id)

        if not chunks:
            continue

        chunk_texts = [c[2] for c in chunks]
        # fastembed returns generator of lists; collect then to numpy
        embs = list(model.embed(chunk_texts, batch_size=256))
        embs = np.array(embs, dtype=np.float32)
        embs = l2_normalize(embs, axis=1)

        with conn:
            for idx, ((line_start, line_end, ctext), emb) in enumerate(zip(chunks, embs)):
                conn.execute(
                    "INSERT INTO chunks(file_id, chunk_index, line_start, line_end, text, embedding) VALUES(?, ?, ?, ?, ?, ?)",
                    (file_id, idx, line_start, line_end, ctext, emb.tobytes())
                )
    conn.close()
    print("Index complete.")


def load_all_chunks(conn) -> List[Tuple[int, str, int, int, str, np.ndarray]]:
    rows = []
    cur = conn.cursor()
    cur.execute("""
    SELECT chunks.id, files.path, chunks.line_start, chunks.line_end, chunks.text, chunks.embedding
    FROM chunks JOIN files ON chunks.file_id = files.id
    """)
    for cid, path, lstart, lend, text, emb in cur.fetchall():
        vec = np.frombuffer(emb, dtype=np.float32)
        rows.append((cid, path, lstart, lend, text, vec))
    return rows


def query(db_path: str, query_text: str, topk: int, model_name: str):
    conn = connect(db_path)
    model = get_model(model_name)

    # embed and normalize query
    qvec = np.array(list(model.embed([query_text]))[0], dtype=np.float32)
    qvec = qvec / max(np.linalg.norm(qvec), 1e-9)

    rows = load_all_chunks(conn)
    if not rows:
        print("No chunks found. Did you run 'index'?", file=sys.stderr)
        return

    mat = np.vstack([r[5] for r in rows])  # (N, d) normalized
    sims = mat @ qvec  # cosine similarity
    top_idx = np.argsort(-sims)[:topk]

    # Output TSV: path<TAB>line_start<TAB>score<TAB>snippet
    for i in top_idx:
        cid, path, lstart, lend, text, _ = rows[i]
        score = float(sims[i])
        snippet = " ".join(text.split())[:220]
        print(f"{path}\t{lstart}\t{score:.3f}\t{snippet}")

    conn.close()


def parse_exts(exts: str) -> List[str]:
    if not exts:
        return DEFAULT_EXTS
    parts = [e.strip() for e in exts.split(",") if e.strip()]
    return [e if e.startswith(".") else "." + e for e in parts]


def main():
    parser = argparse.ArgumentParser(description="Local semantic search for your files (SQLite + fastembed).")
    sub = parser.add_subparsers(dest="cmd")

    p_index = sub.add_parser("index", help="Index a directory")
    p_index.add_argument("root", help="Directory to index")
    p_index.add_argument("--db", default=DEFAULT_DB, help=f"SQLite database path (default: {DEFAULT_DB})")
    p_index.add_argument("--exts", default="", help="Comma-separated file extensions to include (e.g. .md,.txt,.py). Default is a sensible list.")
    p_index.add_argument("--model", default=DEFAULT_MODEL, help=f"Embedding model (default: {DEFAULT_MODEL})")

    p_query = sub.add_parser("query", help="Query the index")
    p_query.add_argument("text", help="Query text (semantic)")
    p_query.add_argument("--db", default=DEFAULT_DB, help=f"SQLite database path (default: {DEFAULT_DB})")
    p_query.add_argument("-k", "--topk", type=int, default=10, help="Number of results")
    p_query.add_argument("--model", default=DEFAULT_MODEL, help=f"Embedding model (default: {DEFAULT_MODEL})")

    args = parser.parse_args()

    if args.cmd == "index":
        exts = parse_exts(args.exts)
        index_dir(args.root, args.db, exts, args.model)
    elif args.cmd == "query":
        query(args.db, args.text, args.topk, args.model)
    else:
        parser.print_help()


if __name__ == "__main__":
    main()

Notes:

  • First run downloads a small model (~20–30 MB) to ~/.cache/fastembed.

  • The database defaults to ~/.cache/semsearch.sqlite.

  • Output is TSV so you can parse it easily in Bash.

3) Index your files

Examples:

  • Index a notes directory:

    ./semsearch.py index ~/notes

  • Index source code with custom extensions:

    ./semsearch.py index ~/src --exts .py,.sh,.c,.h,.go,.rs

  • Store the DB somewhere specific (e.g., a project cache):

    ./semsearch.py index ~/projects/infra --db ~/.cache/infra-sem.sqlite

Re-run the index command anytime; unchanged files are skipped automatically.

4) Query semantically from the shell

  • Basic query:

    ./semsearch.py query "passwordless ssh" -k 8

    Output format (TSV): path<TAB>line_start<TAB>score<TAB>snippet

  • Open a selected result in your editor (fzf + $EDITOR):

    ./semsearch.py query "retry with backoff in bash" -k 50 \
| fzf --with-nth=4.. \
| awk -F'\t' '{print "+"$2" "$1}' \
| xargs -r -I{} sh -c '${EDITOR:-vi} {}'

  • Compare with lexical search (ripgrep) to see the difference:

    rg -n "passwordless|ssh-copy-id|PubkeyAuthentication" ~/notes ~/src

Real-world prompts you can try:

  • “rotate ssh keys across servers”

  • “trap signals in bash and cleanup temp files”

  • “parse json in shell without jq”

  • “idempotent systemd service unit example”

5) Nice-to-haves and production tips

  • Keep the index fresh with cron:

    crontab -e
# Reindex notes daily at 1:15 AM
15 1 * * * /bin/bash -lc '~/.local/semsearch/bin/python ~/bin/semsearch.py index ~/notes >> ~/.cache/semsearch_cron.log 2>&1'

  • Use multiple indexes for different corpora (e.g., notes vs. code) by changing --db.

  • Extend file types by setting --exts per project or globally.

  • Privacy and offline use: after the first model download, everything runs locally.

  • Speed up large corpora by excluding heavy or binary folders (node_modules, target, .git).

Advanced ideas (optional):

  • Swap the embedding model with --model (try multilingual models).

  • Store pre-normalized vectors (already done) for fast cosine similarity.

  • If your dataset grows huge, move to an ANN index (faiss, hnsw) or a vector DB (Qdrant), but keep the same CLI wrapper.

Troubleshooting

  • “ModuleNotFoundError: fastembed”: You’re not in the venv or didn’t install packages. Run:

    ~/.local/semsearch/bin/pip install -U fastembed numpy tqdm

  • Slow first run: The model is downloading. Subsequent runs are fast and offline.

  • Missing packages (fzf/ripgrep/sqlite3):

    • apt:
    sudo apt install -y fzf ripgrep sqlite3
    • dnf:
    sudo dnf install -y fzf ripgrep sqlite
    • zypper:
    sudo zypper install -y fzf ripgrep sqlite3

Conclusion and next step

Semantic search adds “find the idea” to your Linux toolbox. You just built a private, local system that understands meaning well enough to surface the right snippet—even when you forget the words.

Your next step:

  • Index your notes or a codebase today:

    ./semsearch.py index ~/notes && ./semsearch.py query "how to template config files"

  • Wire the query into your day-to-day shell with fzf and $EDITOR.

  • Iterate: add more folders, tune --exts, or try a different --model.

When grep isn’t enough, bring meaning to your terminal.

Related posts