Posted on
• Artificial Intelligence

Embedding Models Explained

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Embedding Models Explained: Bring Semantic Search to Your Linux Shell

Ever grepped for something you know exists but couldn’t recall exact words? Embedding models turn that fuzzy intent into precise results. Instead of matching characters, embeddings capture meaning—so “How do I autostart a service?” finds docs about “systemd units,” even if the word “autostart” doesn’t appear.

In this post, you’ll learn what embeddings are, why they matter to Linux users, and how to build a fast, local semantic search for your own docs, logs, and shell history—entirely from the command line with a few small Python scripts.

What are embeddings (and why you should care)

  • An embedding model converts text into a numerical vector (e.g., 384 or 768 floats) that encodes semantic meaning.

  • Similar texts end up with similar vectors. You can measure similarity with cosine similarity or dot product.

  • Why it’s valid and useful on Linux:

    • Search beyond keywords: find “apache crash logs” that talk about “httpd segfaults”.
    • Faster incident triage: group near-duplicate errors or configs.
    • Smarter knowledge lookup: query your scripts, READMEs, and notes by intent.
    • Fuel for RAG (Retrieval-Augmented Generation): retrieve the right snippets to feed your LLM.

Install prerequisites (apt, dnf, zypper)

We’ll use Python, curl, and jq. Choose your distro’s package manager.

Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y python3 python3-venv python3-pip curl jq git

Fedora/RHEL/CentOS Stream (dnf):

sudo dnf install -y python3 python3-pip python3-virtualenv curl jq git

openSUSE (zypper):

sudo zypper refresh
sudo zypper install -y python3 python3-pip python3-virtualenv curl jq git

Create and activate a virtual environment, then install libraries:

python3 -m venv .venv
. .venv/bin/activate
pip install -U pip
pip install sentence-transformers faiss-cpu tqdm

Note: faiss-cpu provides a fast vector index. If your platform lacks wheels, ensure build tools are installed via your package manager.

A real-world example: Semantic search over your docs and history

We’ll index a folder of text-like files (README, .txt, .md, .log, .conf, scripts) and then run semantic queries. You can also add your shell history to the corpus.

Step 1: Gather some text

mkdir -p ~/semantic-corpus
# Example sources
cp -r /usr/share/doc ~/semantic-corpus/doc 2>/dev/null || true
cp ~/.bash_history ~/semantic-corpus/bash_history.txt 2>/dev/null || true
# Add your own notes/projects
cp -r ~/projects/* ~/semantic-corpus/ 2>/dev/null || true

Step 2: Create an index script (build_index.py)

cat > build_index.py << 'PY'
import os, json, argparse, pathlib, re
from pathlib import Path
from tqdm import tqdm
import numpy as np
import faiss
from sentence_transformers import SentenceTransformer

def read_text_file(path):
    try:
        with open(path, "r", encoding="utf-8", errors="ignore") as f:
            return f.read()
    except Exception:
        return ""

def chunk_text(text, size=700, overlap=100):
    text = re.sub(r'\s+', ' ', text).strip()
    chunks = []
    start = 0
    n = len(text)
    while start < n:
        end = min(start + size, n)
        chunk = text[start:end]
        if chunk:
            chunks.append(chunk)
        start = end - overlap
        if start < 0: start = 0
        if start >= n: break
    return chunks

def collect_files(root, exts):
    root = Path(root)
    files = []
    for p in root.rglob("*"):
        if p.is_file() and (p.suffix.lower() in exts or p.suffix == "" and p.name.endswith("_history.txt")):
            files.append(p)
    return files

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--data-dir", required=True, help="Directory to index")
    ap.add_argument("--index", default="index.faiss")
    ap.add_argument("--meta", default="meta.json")
    ap.add_argument("--model", default="sentence-transformers/all-MiniLM-L6-v2")
    ap.add_argument("--batch-size", type=int, default=64)
    ap.add_argument("--chunk-size", type=int, default=700)
    ap.add_argument("--chunk-overlap", type=int, default=100)
    ap.add_argument("--extensions", default=".txt,.md,.log,.conf,.ini,.yaml,.yml,.sh,.py,.rst")
    args = ap.parse_args()

    model = SentenceTransformer(args.model)
    exts = set([e.strip() for e in args.extensions.split(",") if e.strip()])

    files = collect_files(args.data_dir, exts)
    if not files:
        print("No files found to index.")
        return

    dim = model.get_sentence_embedding_dimension()
    index = faiss.IndexFlatIP(dim)  # cosine via normalized dot product
    metadata = []

    texts = []
    meta_buffer = []

    for fp in tqdm(files, desc="Reading & chunking"):
        text = read_text_file(fp)
        if not text:
            continue
        chunks = chunk_text(text, size=args.chunk_size, overlap=args.chunk_overlap)
        for i, ch in enumerate(chunks):
            texts.append(ch)
            meta_buffer.append({"path": str(fp), "chunk_id": i, "text": ch[:300]})

            # Encode in batches to save memory
            if len(texts) >= args.batch_size:
                embs = model.encode(texts, convert_to_numpy=True, normalize_embeddings=True)
                index.add(embs.astype("float32"))
                metadata.extend(meta_buffer)
                texts, meta_buffer = [], []

    # Flush remainder
    if texts:
        embs = model.encode(texts, convert_to_numpy=True, normalize_embeddings=True)
        index.add(embs.astype("float32"))
        metadata.extend(meta_buffer)

    faiss.write_index(index, args.index)
    with open(args.meta, "w", encoding="utf-8") as f:
        json.dump(metadata, f, ensure_ascii=False)
    print(f"Indexed {len(metadata)} chunks from {len(files)} files.")
    print(f"Saved index to {args.index} and metadata to {args.meta}")

if __name__ == "__main__":
    main()
PY

Step 3: Create a query script (search_index.py)

cat > search_index.py << 'PY'
import json, argparse
import numpy as np
import faiss
from sentence_transformers import SentenceTransformer

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--index", default="index.faiss")
    ap.add_argument("--meta", default="meta.json")
    ap.add_argument("--model", default="sentence-transformers/all-MiniLM-L6-v2")
    ap.add_argument("--k", type=int, default=5)
    ap.add_argument("--query", required=True)
    args = ap.parse_args()

    index = faiss.read_index(args.index)
    with open(args.meta, "r", encoding="utf-8") as f:
        meta = json.load(f)
    model = SentenceTransformer(args.model)

    q = model.encode([args.query], convert_to_numpy=True, normalize_embeddings=True).astype("float32")
    D, I = index.search(q, args.k)
    print(f"Top {args.k} results for: {args.query}\n")
    for rank, (idx, score) in enumerate(zip(I[0], D[0]), start=1):
        if idx == -1:
            continue
        m = meta[idx]
        print(f"[{rank}] score={score:.4f}")
        print(f"    file: {m['path']}")
        print(f"    snippet: {m['text']}\n")

if __name__ == "__main__":
    main()
PY

Step 4: Build and query the index

# Build
python3 build_index.py --data-dir ~/semantic-corpus --index ./myindex.faiss --meta ./mymeta.json

# Query examples
python3 search_index.py --index ./myindex.faiss --meta ./mymeta.json --query "how to create a systemd service" --k 5
python3 search_index.py --index ./myindex.faiss --meta ./mymeta.json --query "enable apache access logs" --k 5
python3 search_index.py --index ./myindex.faiss --meta ./mymeta.json --query "tar compress a folder and exclude files" --k 5

Optional: a shell helper for quick queries

semgrep() {
  . .venv/bin/activate >/dev/null 2>&1 || return 1
  python3 search_index.py --index ./myindex.faiss --meta ./mymeta.json --query "$*" --k 8
}
# Usage: semgrep find nginx vhost root

4 actionable tips to get better results

1) Pick the right model

  • General English: sentence-transformers/all-MiniLM-L6-v2 (fast, small).

  • Multilingual: paraphrase-multilingual-MiniLM-L12-v2.

  • Code-heavy repos: consider a code-aware embedding model from sentence-transformers’ model hub.

2) Normalize and use cosine

  • We used normalize_embeddings=True and FAISS IndexFlatIP. That gives cosine-similar behavior with good speed.

3) Chunk wisely

  • Too big: irrelevant text dilutes meaning. Too small: you lose context. Start with 500–800 characters and ~10–15% overlap.

  • For logs, group by N lines; for markdown, split by headings.

4) Keep your index fresh

  • Re-run the indexer in cron/systemd timers.

  • Store file mtime and skip unchanged files for incremental updates (easy enhancement).

Bonus: One-off embeddings via curl

If you prefer an API workflow (e.g., Hugging Face Inference Endpoints), you can obtain vectors with curl. You’ll need curl and jq (installed above) and a token in HF_TOKEN. Example template:

export HF_TOKEN=hf_XXXXXXXXXXXXXXXXXXXXXXXXXXXX

curl -s -H "Authorization: Bearer $HF_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{"inputs": ["how to rotate nginx logs", "create systemd unit file"]}' \
     https://api-inference.huggingface.co/pipeline/feature-extraction/sentence-transformers/all-MiniLM-L6-v2 \
| jq

You can store returned embeddings and feed them into FAISS the same way.

Troubleshooting

  • faiss-cpu install issues: upgrade pip and retry. If wheels are unavailable on your arch, install build tools via your package manager and consider using an alternative vector store (e.g., numpy + sklearn NearestNeighbors) for small datasets.

  • Memory constraints: index fewer files at first, or switch to an IVF/Flat hybrid in FAISS for large corpora.

  • Performance: batch size impacts both memory and speed—tune based on your system.

Conclusion and next steps

Embeddings let you search by meaning, not memorized strings. In minutes, you built a local semantic search for your docs and history using sentence-transformers and FAISS.

Next steps:

  • Add incremental indexing and file change detection.

  • Extend search output to open files directly in your $EDITOR.

  • Try a domain-specific embedding model for your stack.

  • Feed top results into an LLM for on-demand “RAG” answers.

If this helped, share your setup or improvements—Bash-friendly semantic tooling is just getting started.

Related posts