Posted on
• Artificial Intelligence

Artificial Intelligence Desktop Search

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

AI Desktop Search on Linux: Turn your terminal into a semantic search engine

You’ve got 30,000+ files across notes, PDFs, logs, code, and wikis. grep is great—until you can’t remember the exact words. AI Desktop Search bridges that gap: instead of exact matches, you get semantic matches and answers, even when your query is fuzzy.

In this post you’ll:

  • Understand why AI search on your Linux desktop is worth it

  • Set up a quick “LLM-reranked grep” you can use today

  • Build a local, private embeddings index (RAG) for true semantic search

  • Keep it fresh with a background indexer

  • Get package-manager-specific install commands (apt, dnf, zypper)

Why AI desktop search is worth it

  • Natural language beats filenames: “diagram about zero-downtime deploy” should find that blue-green.md, even if you never wrote “zero-downtime”.

  • Summaries over snippets: LLMs can answer, not just list matches.

  • Cross-format recall: PDFs, DOCX, HTML, Markdown—one search.

  • Private and offline: Modern local models and embeddings run on your machine.

Prerequisites (install via your package manager)

You’ll use a mix of familiar CLI tools, plus an LLM runtime (Ollama) and/or Python libs for embeddings.

  • Debian/Ubuntu (apt):

    sudo apt update
sudo apt install -y ripgrep fd-find jq curl python3 python3-venv python3-pip
# On Debian/Ubuntu the fd binary is named "fdfind"; optional convenience alias:
echo 'alias fd=fdfind' >> ~/.bashrc

  • Fedora (dnf):

    sudo dnf install -y ripgrep fd-find jq curl python3 python3-pip python3-virtualenv
# On Fedora the binary is usually "fd"

  • openSUSE (zypper):

    sudo zypper install -y ripgrep fd jq curl python3 python3-pip python3-virtualenv

Install Ollama (local LLM runtime, works offline once the model is downloaded):

curl -fsSL https://ollama.com/install.sh | sh
# Pull a small, capable model (you can swap for your favorite):
ollama pull llama3

Tip: If you’ll build the embeddings index below, you’ll also install Python packages via a virtual environment.

Option A (5–10 minutes): LLM‑reranked grep for smarter results today

This approach uses ripgrep to grab candidate snippets, then asks a local LLM (via Ollama) to re-rank them by meaning and explain why.

1) Save this script as ai-search.sh and make it executable:

#!/usr/bin/env bash
set -euo pipefail

MODEL="${MODEL:-llama3}"

if [ $# -lt 1 ]; then
  echo "Usage: $0 <natural-language query> [search_root=.]" >&2
  exit 1
fi

QUERY="$1"
ROOT="${2:-.}"
TMPDIR="$(mktemp -d)"
SNIPS="$TMPDIR/snippets.txt"

# Turn the query into a reasonable OR-pattern for rg (basic candidate recall)
# Ignore very short words to reduce noise.
PATTERN="$(printf '%s\n' "$QUERY" | tr ' ' '\n' | awk 'length>2' | paste -sd'|' -)"
if [ -z "$PATTERN" ]; then
  PATTERN="$QUERY"
fi

# Collect up to ~200 candidate snippets with some context
rg -n --no-heading --hidden -S -i \
   -g '!.git' -g '!node_modules' -g '!.venv' \
   -C1 --max-filesize 2M --max-count 200 \
   -e "$PATTERN" "$ROOT" | awk 'NF' | nl -ba -w2 -s' ' > "$SNIPS"

if [ ! -s "$SNIPS" ]; then
  echo "No candidate matches found with ripgrep." >&2
  exit 0
fi

PROMPT=$(cat <<'EOF'
You are a local search results re-ranker for code and documents.
Given a user query and a list of snippets in the format:
[N] path:line: context-before
    matched line
    context-after

Return the 10 most relevant entries ranked by semantic relevance to the query.
Output plain text lines exactly in this format (one per line):
rank. [N] path:line — brief reason
EOF
)

echo "Re-ranking with model: $MODEL" >&2
ollama run "$MODEL" <<EOF
$PROMPT

User query:
$QUERY

Snippets:
$(cat "$SNIPS")
EOF

2) Use it:

chmod +x ai-search.sh
./ai-search.sh "how do we rotate database credentials" ~/work/infra

What you get: a human-friendly, meaning-aware ranking and “why” reasoning—using your CPU/GPU only, no cloud required.

Limitations: This still relies on ripgrep for first-pass recall, so it shines when you remember a few terms. For true fuzzy/semantic matches across everything, build the embeddings index below.

Option B (30–45 minutes): Local embeddings index for true semantic search (RAG)

This builds a private vector index of your files. Queries are turned into embeddings and matched by meaning, not exact words—no network required.

1) Create and activate a Python virtual environment:

  • Debian/Ubuntu:

    python3 -m venv ~/.venvs/aisearch
source ~/.venvs/aisearch/bin/activate
pip install --upgrade pip

  • Fedora:

    python3 -m venv ~/.venvs/aisearch  # if missing, install python3-virtualenv
source ~/.venvs/aisearch/bin/activate
pip install --upgrade pip

  • openSUSE:

    python3 -m venv ~/.venvs/aisearch  # if missing, install python3-virtualenv
source ~/.venvs/aisearch/bin/activate
pip install --upgrade pip

2) Install Python packages:

pip install sentence-transformers faiss-cpu pypdf beautifulsoup4 lxml python-docx tqdm

3) Save this indexer as index_dir.py:

#!/usr/bin/env python3
import argparse, os, json, re, sys
from pathlib import Path
from tqdm import tqdm
from sentence_transformers import SentenceTransformer
import faiss

# Lightweight extractors
def read_text(path):
    try:
        return Path(path).read_text(encoding="utf-8", errors="ignore")
    except Exception:
        return ""

def read_pdf(path):
    from pypdf import PdfReader
    try:
        text = []
        r = PdfReader(path)
        for page in r.pages:
            text.append(page.extract_text() or "")
        return "\n".join(text)
    except Exception:
        return ""

def read_docx(path):
    try:
        from docx import Document
        doc = Document(path)
        return "\n".join(p.text for p in doc.paragraphs)
    except Exception:
        return ""

def read_html(path):
    try:
        from bs4 import BeautifulSoup
        t = Path(path).read_text(encoding="utf-8", errors="ignore")
        return BeautifulSoup(t, "lxml").get_text(separator="\n")
    except Exception:
        return ""

def extract_text(path):
    ext = Path(path).suffix.lower()
    if ext in [".txt", ".md", ".rst", ".log", ".cfg", ".ini", ".yaml", ".yml", ".json", ".py", ".sh", ".c", ".cpp", ".js", ".ts", ".java", ".go", ".rb", ".rs", ".toml"]:
        return read_text(path)
    if ext == ".pdf":
        return read_pdf(path)
    if ext in [".htm", ".html"]:
        return read_html(path)
    if ext == ".docx":
        return read_docx(path)
    return ""  # skip others by default

def chunk_text(text, size=800, overlap=200):
    text = re.sub(r"\s+\n", "\n", text)
    text = re.sub(r"\n{3,}", "\n\n", text)
    if len(text) <= size:
        return [text]
    chunks = []
    start = 0
    while start < len(text):
        end = min(len(text), start + size)
        chunks.append(text[start:end])
        if end == len(text):
            break
        start = end - overlap
    return chunks

def main():
    ap = argparse.ArgumentParser(description="Build a local semantic index (FAISS) for a directory")
    ap.add_argument("root", help="Directory to index")
    ap.add_argument("--index", default="index.faiss", help="FAISS index output path")
    ap.add_argument("--meta", default="meta.jsonl", help="Metadata JSONL output")
    ap.add_argument("--model", default="sentence-transformers/all-MiniLM-L6-v2", help="Sentence-Transformers model")
    ap.add_argument("--include", nargs="*", default=None, help="Glob(s) to include, e.g. **/*.md **/*.pdf")
    ap.add_argument("--exclude", nargs="*", default=["**/.git/**", "**/node_modules/**", "**/.venv/**"], help="Glob(s) to exclude")
    args = ap.parse_args()

    root = Path(args.root).expanduser().resolve()
    files = []
    # Walk and match includes/excludes
    if args.include:
        for pat in args.include:
            files.extend(root.glob(pat))
        files = [p for p in files if p.is_file()]
    else:
        for p, _, fns in os.walk(root):
            for fn in fns:
                files.append(Path(p) / fn)

    # Apply excludes
    excl = [root / e for e in []]  # placeholder
    exglobs = [root.as_posix() + "/" + g for g in []]  # not used
    # Simpler: filter with fnmatch on posix paths
    import fnmatch
    cleaned = []
    for f in files:
        posix = f.as_posix()
        skip = False
        for g in args.exclude:
            if fnmatch.fnmatch(posix, (root.as_posix().rstrip("/") + "/" + g.lstrip("/"))):
                skip = True
                break
        if not skip:
            cleaned.append(f)
    files = cleaned

    model = SentenceTransformer(args.model)
    embeddings = []
    meta = []
    print(f"Scanning {len(files)} files...")
    for path in tqdm(files):
        txt = extract_text(path)
        if not txt or len(txt.strip()) < 50:
            continue
        chunks = chunk_text(txt)
        for i, ch in enumerate(chunks):
            meta.append({
                "path": str(path),
                "chunk": i,
                "preview": ch[:200].replace("\n", " "),
            })
            embeddings.append(ch)

    if not embeddings:
        print("No text extracted. Check your include/exclude patterns.", file=sys.stderr)
        sys.exit(1)

    print(f"Encoding {len(embeddings)} chunks with {args.model}...")
    vecs = model.encode(embeddings, normalize_embeddings=True, show_progress_bar=True)
    dim = vecs.shape[1]
    index = faiss.IndexFlatIP(dim)  # cosine via normalized vectors
    index.add(vecs)

    faiss.write_index(index, args.index)
    with open(args.meta, "w", encoding="utf-8") as f:
        for m in meta:
            f.write(json.dumps(m, ensure_ascii=False) + "\n")
    print(f"Wrote {args.index} and {args.meta}")

if __name__ == "__main__":
    main()

4) Save this searcher as search.py:

#!/usr/bin/env python3
import argparse, json
from pathlib import Path
from sentence_transformers import SentenceTransformer
import faiss

def load_meta(path):
    out = []
    with open(path, "r", encoding="utf-8") as f:
        for line in f:
            out.append(json.loads(line))
    return out

def main():
    ap = argparse.ArgumentParser(description="Semantic search over a local FAISS index")
    ap.add_argument("query")
    ap.add_argument("--index", default="index.faiss")
    ap.add_argument("--meta", default="meta.jsonl")
    ap.add_argument("--model", default="sentence-transformers/all-MiniLM-L6-v2")
    ap.add_argument("-k", type=int, default=10)
    args = ap.parse_args()

    index = faiss.read_index(args.index)
    meta = load_meta(args.meta)
    model = SentenceTransformer(args.model)

    q = model.encode([args.query], normalize_embeddings=True)
    D, I = index.search(q, args.k)
    print(f"Top {args.k} results for: {args.query}\n")
    for rank, (score, idx) in enumerate(zip(D[0], I[0]), 1):
        m = meta[int(idx)]
        print(f"{rank}. {m['path']} [chunk {m['chunk']}]  score={score:.3f}")
        print(f"   {m['preview']}\n")

if __name__ == "__main__":
    main()

5) Build the index (examples):

# Index everything under ~/docs and ~/work/code
python3 index_dir.py ~/docs --index docs.faiss --meta docs.jsonl --include '**/*.md' '**/*.pdf' '**/*.txt' '**/*.html' '**/*.docx'
python3 index_dir.py ~/work/code --index code.faiss --meta code.jsonl --include '**/*.py' '**/*.sh' '**/*.js' '**/*.ts' '**/*.go' '**/*.java' '**/*.rs' '**/*.c' '**/*.cpp'

# Or build a single merged index (simpler to start):
python3 index_dir.py ~/ --index desktop.faiss --meta desktop.jsonl --exclude '**/.git/**' '**/node_modules/**' '**/.venv/**'

6) Search it:

python3 search.py "how do we rotate database credentials" --index desktop.faiss --meta desktop.jsonl -k 10
python3 search.py "design doc about zero downtime deployments" --index desktop.faiss --meta desktop.jsonl -k 10

Optional: After retrieving top K chunks, you can feed them to your local model for a proper answer:

# Example: answer from top 5 chunks with Ollama
python3 search.py "what is our staging SSO flow?" --index desktop.faiss --meta desktop.jsonl -k 5 \
| sed -n 's/^   //p' > context.txt

ollama run llama3 <<'EOF'
You are a helpful assistant. Use the following context to answer the user's question.

Context:
<<<
$(cat context.txt)
>>>

Question: what is our staging SSO flow?

Answer concisely and cite file paths when relevant.
EOF

Keep it fresh: automate re-indexing

Use systemd user timers or cron to re-run the indexer nightly.

  • systemd (user):
# ~/.config/systemd/user/aisearch-index.service
[Unit]
Description=Rebuild AI search index

[Service]
Type=oneshot
WorkingDirectory=%h
ExecStart=%h/.venvs/aisearch/bin/python3 %h/index_dir.py %h --index %h/desktop.faiss --meta %h/desktop.jsonl --exclude **/.git/** **/node_modules/** **/.venv/**

# ~/.config/systemd/user/aisearch-index.timer
[Unit]
Description=Run AI indexer nightly

[Timer]
OnCalendar=03:15
Persistent=true

[Install]
WantedBy=timers.target

Enable it:

systemctl --user daemon-reload
systemctl --user enable --now aisearch-index.timer
  • cron (alternative):
15 3 * * * source $HOME/.venvs/aisearch/bin/activate && python3 $HOME/index_dir.py $HOME --index $HOME/desktop.faiss --meta $HOME/desktop.jsonl --exclude '**/.git/**' '**/node_modules/**' '**/.venv/**' >/dev/null 2>&1

Real-world usage ideas

  • Search across runbooks and tickets: “incident response checklist for TLS expiry”

  • Codebase Q&A: “where do we back off on 429?” across multiple languages

  • Compliance and audits: “documents mentioning vendor risk assessment”

  • Personal knowledge base: “notes about qemu bridged networking how-to”

Conclusion and next steps

Start with Option A (LLM‑reranked grep) for immediate wins in your terminal. When you’re ready for true fuzzy recall, build Option B’s embeddings index—private, fast, and offline. From there you can:

  • Wire the top‑K chunks into Ollama for full Q&A answers

  • Add more file types to the extractor

  • Schedule nightly indexing

  • Swap to a larger embedding model for better recall

If you want help tuning chunk sizes, adding OCR for images/scans, or integrating with fzf/tmux, ask and I’ll show you how to extend this setup.

Related posts