Artificial Intelligence Projects for Home Labs (Bash-Friendly Builds)

Want privacy, speed, and hands-on skills without renting GPUs in the cloud? Bring AI into your home lab. In a few terminal commands, you can stand up a local LLM API, transcribe audio offline, flag weird log activity, and ask natural-language questions over your personal documents—no data leaving your network.

This guide explains why home-lab AI is worth it, then walks through 4 practical projects with step-by-step Bash and Python snippets. You’ll get installation commands for apt, dnf, and zypper where needed, and small, composable scripts you can extend later.

Why run AI in your home lab?

• Privacy and data locality: Keep your chats, docs, and logs off third-party clouds.

• Latency and reliability: Sub-second responses on your LAN, even when the internet is down.

• Cost control: Use the hardware you already own; scale up only if you need to.

• Skill building: Learn the Linux and container skills that power modern AI ops.

---

Project 1: Run a Local LLM API with Ollama (plus llama.cpp option)

What you’ll get: A local HTTP API for chat/completions that you can script with curl. Great for automations, RAG backends, and tinkering.

Install prerequisites (curl)

• Ubuntu/Debian (apt)

sudo apt update
sudo apt install -y curl

• Fedora/RHEL (dnf)

sudo dnf install -y curl

• openSUSE (zypper)

sudo zypper install -y curl

Install Ollama (one-liner)

curl -fsSL https://ollama.com/install.sh | sh
sudo systemctl enable --now ollama

Pull and run a model

ollama pull llama3
ollama run llama3

Tip: Press Ctrl+C to exit the REPL. Keep the daemon running to use the REST API.

Call the REST API from Bash

curl http://localhost:11434/api/generate \
  -d '{"model":"llama3","prompt":"Write a haiku about homelabs."}'

You can also stream responses or use other models like mistral, phi3, qwen2, or small-footprint ones for older CPUs.

Alternative: Build llama.cpp from source (advanced)

Prereqs:

• Ubuntu/Debian (apt)

sudo apt update
sudo apt install -y git cmake build-essential wget

• Fedora/RHEL (dnf)

sudo dnf install -y git cmake gcc-c++ make wget

• openSUSE (zypper)

sudo zypper install -y git cmake gcc-c++ make wget

Build:

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make -j
mkdir -p models

Download a small GGUF model (example: TinyLlama 1.1B Chat). Visit the model page and download a GGUF file into models/. Example:

wget -O models/TinyLlama-1.1B-Chat-v1.0.Q4_K_M.gguf \
  "https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0-GGUF/resolve/main/TinyLlama-1.1B-Chat-v1.0.Q4_K_M.gguf?download=true"

Run:

./main -m models/TinyLlama-1.1B-Chat-v1.0.Q4_K_M.gguf -p "Hello from my homelab!"

---

Project 2: Offline Speech-to-Text (STT) + Text-to-Speech (TTS) Mini Assistant

What you’ll get: Record your voice, transcribe offline with Vosk, and speak back using espeak-ng—no network calls, simple and fast.

Install prerequisites

• Ubuntu/Debian (apt)

sudo apt update
sudo apt install -y python3 python3-venv python3-pip sox alsa-utils espeak-ng wget unzip

• Fedora/RHEL (dnf)

sudo dnf install -y python3 python3-pip sox alsa-utils espeak-ng wget unzip

• openSUSE (zypper)

sudo zypper install -y python3 python3-pip sox alsa-utils espeak-ng wget unzip

Note: On Debian/Ubuntu, python3-venv ensures “python3 -m venv” works. On Fedora/openSUSE, venv is typically included with python3.

Create a project folder and venv (optional but recommended)

mkdir -p ~/stt && cd ~/stt
python3 -m venv .venv
. .venv/bin/activate
python3 -m pip install --upgrade pip
pip install vosk

Download a small English model

wget https://alphacephei.com/vosk/models/vosk-model-small-en-us-0.15.zip
unzip -q vosk-model-small-en-us-0.15.zip
mv vosk-model-small-en-us-0.15 model

Record audio (5 seconds) and transcribe

Record:

arecord -f cd -d 5 -q speech.wav

Transcribe (stt.py):

cat > stt.py << 'PY'
import sys, json, wave
from vosk import Model, KaldiRecognizer

wf = wave.open("speech.wav", "rb")
assert wf.getnchannels() == 1 or wf.getframerate() == 16000, \
    "Use mono, 16kHz WAV for best results (tip: sox can convert)"

model = Model("model")
rec = KaldiRecognizer(model, wf.getframerate())
rec.SetWords(True)

text = []
while True:
    data = wf.readframes(4000)
    if len(data) == 0:
        break
    if rec.AcceptWaveform(data):
        text.append(json.loads(rec.Result()).get("text",""))
text.append(json.loads(rec.FinalResult()).get("text",""))
print(" ".join([t for t in text if t]).strip())
PY

Run:

python3 stt.py

Optional TTS reply:

espeak-ng -s 160 "You said: $(python3 stt.py)"

Tips:

• For better STT quality, use larger models from Vosk’s site.

• To ensure mono/16kHz input: sox speech.wav -r 16000 -c 1 speech_mono.wav.

---

Project 3: Detect Suspicious Log Activity with IsolationForest

What you’ll get: A quick anomaly detector for SSH auth logs that flags unusual IPs or spikes. It’s a lightweight way to add some ML to your homelab security checks.

Install prerequisites

• Ubuntu/Debian (apt)

sudo apt update
sudo apt install -y python3 python3-pip

• Fedora/RHEL (dnf)

sudo dnf install -y python3 python3-pip

• openSUSE (zypper)

sudo zypper install -y python3 python3-pip

Set up environment (optional venv) and install packages:

mkdir -p ~/log-ai && cd ~/log-ai
python3 -m venv .venv 2>/dev/null || true
. .venv/bin/activate 2>/dev/null || true
python3 -m pip install --upgrade pip
pip install scikit-learn numpy

Script: parse logs and flag outliers

This script reads /var/log/auth.log (Debian/Ubuntu) or /var/log/secure (Fedora/openSUSE/RHEL), computes simple features per source IP, and uses IsolationForest to score anomalies.

cat > detect_anomalies.py << 'PY'
import os, re, sys, gzip
from datetime import datetime
import numpy as np
from sklearn.ensemble import IsolationForest

paths = ["/var/log/auth.log", "/var/log/auth.log.1", "/var/log/secure", "/var/log/secure-2024*"]
logs = []
for p in paths:
    for candidate in sorted([p] if "*" not in p else [fp for fp in ["/var/log/secure", "/var/log/secure-2024", "/var/log/secure-2023"] if os.path.exists(fp)]):
        if os.path.exists(candidate):
            logs.append(candidate)

if not logs:
    print("No known auth logs found. Try adjusting paths.")
    sys.exit(1)

ip_re = re.compile(r'(\d{1,3}\.){3}\d{1,3}')
fail_re = re.compile(r'Failed password')
ok_re = re.compile(r'Accepted password|Accepted publickey')

stats = {}  # ip -> [fails, oks, hours_active]
hours_seen = {}

def get_hour(ts):
    # Try syslog-like "MMM DD HH:MM:SS"
    try:
        m = re.match(r'^[A-Z][a-z]{2}\s+\d{1,2}\s+(\d{2}):\d{2}:\d{2}', ts)
        if m:
            return int(m.group(1))
    except:
        pass
    return None

def update(ip, line):
    f = 1 if fail_re.search(line) else 0
    o = 1 if ok_re.search(line) else 0
    h = get_hour(line)
    if ip not in stats:
        stats[ip] = [0,0,0]
        hours_seen[ip] = set()
    stats[ip][0] += f
    stats[ip][1] += o
    if h is not None:
        hours_seen[ip].add(h)

for path in logs:
    opener = gzip.open if path.endswith(".gz") else open
    try:
        with opener(path, "rt", errors="ignore") as fh:
            for line in fh:
                m = ip_re.search(line)
                if not m: continue
                ip = m.group(0)
                update(ip, line)
    except Exception as e:
        continue

for ip in stats:
    stats[ip][2] = len(hours_seen[ip])

# Build feature matrix
ips = list(stats.keys())
X = np.array([stats[ip] for ip in ips], dtype=float)
if len(ips) < 5:
    print("Not enough data to model anomalies.")
    sys.exit(0)

model = IsolationForest(n_estimators=200, contamination='auto', random_state=42)
scores = model.fit_predict(X)
anom_idx = np.where(scores == -1)[0]

print("Potentially anomalous IPs (failures, successes, distinct hours):")
for i in anom_idx:
    print(f"{ips[i]:15s}  {stats[ips[i]]}")
PY

Run (you may need sudo for some logs):

sudo python3 detect_anomalies.py

Automate with cron/systemd and alert via mail or a webhook.

---

Project 4: Private RAG Search Over Your Documents

What you’ll get: Local embeddings with sentence-transformers, a tiny vector store (Chroma), and a CLI to search your notes/wikis. Optional: forward top results to your local LLM (Ollama) for a summarized answer.

Install prerequisites

• Ubuntu/Debian (apt)

sudo apt update
sudo apt install -y python3 python3-pip

• Fedora/RHEL (dnf)

sudo dnf install -y python3 python3-pip

• openSUSE (zypper)

sudo zypper install -y python3 python3-pip

Set up environment and install libs:

mkdir -p ~/rag && cd ~/rag
python3 -m venv .venv 2>/dev/null || true
. .venv/bin/activate 2>/dev/null || true
python3 -m pip install --upgrade pip
pip install "sentence-transformers<3" chromadb

Index a folder of .txt/.md and query

cat > rag.py << 'PY'
import os, glob, sys
import chromadb
from sentence_transformers import SentenceTransformer

docs_dir = sys.argv[1] if len(sys.argv) > 1 else "."
query = sys.argv[2] if len(sys.argv) > 2 else "What is this repository about?"

paths = sorted(glob.glob(os.path.join(docs_dir, "**/*.txt"), recursive=True) +
               glob.glob(os.path.join(docs_dir, "**/*.md"), recursive=True))

if not paths:
    print("No .txt/.md files found.")
    sys.exit(0)

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
client = chromadb.Client()
col = client.get_or_create_collection("local_docs")

# Clear and (re)index quickly for demo
try:
    client.delete_collection("local_docs")
    col = client.create_collection("local_docs")
except:
    pass

docs = []
metas = []
ids = []
for i, p in enumerate(paths):
    with open(p, "r", errors="ignore") as f:
        txt = f.read()
    docs.append(txt)
    metas.append({"path": p})
    ids.append(str(i))

col.add(documents=docs, metadatas=metas, ids=ids)

res = col.query(query_texts=[query], n_results=3)
print("Top matches:")
for doc, meta in zip(res["documents"][0], res["metadatas"][0]):
    print(f"- {meta['path']}\n  {doc[:300].replace('\\n',' ')}...")
PY

Run:

python3 rag.py ~/Documents "How do I deploy my service?"

Optional: Summarize with Ollama. Pull a chat model and then:

read -r CONTEXT << 'EOF'
$(python3 rag.py ~/Documents "How do I deploy my service?")
EOF

curl http://localhost:11434/api/generate \
  -d "{\"model\":\"llama3\",\"prompt\":\"Using the context below, answer succinctly: How do I deploy my service?\n\n$CONTEXT\"}"

---

Operational tips

• Hardware: CPU-only works for small models; GPUs/NPUs accelerate everything. Try smaller “instruct” models first.

• Services: Use systemd user services or containers (e.g., podman) to keep things running after reboot.

• Storage: Model files and embeddings can be large—use a fast SSD.

• Security: Treat local AI endpoints like any service; firewall, auth, and logs matter.

---

Conclusion and next steps

You now have four practical, privacy-friendly AI capabilities running locally:

• A chat/completions API (Ollama or llama.cpp)

• Offline speech transcription and TTS

• Anomaly detection on auth logs

• RAG-style semantic search over your docs

Pick one, ship it this weekend, and then wire them together. For example, feed your RAG results into the LLM, or have your voice assistant trigger queries and read back answers. When you’re ready, wrap them with systemd units, expose them behind a reverse proxy on your homelab network, and share them with your family or team.

If you want a follow-up with systemd units, podman-compose files, or GPU acceleration guides, say the word—happy to help you harden and scale your setup.