Hands-On Artificial Intelligence Projects for Linux

Turn your Linux terminal into an AI lab—no cloud, no massive GPUs, no mystery boxes. With a handful of open-source tools and a Bash-friendly workflow, you can build real, local AI utilities that respect your privacy and slot neatly into scripts, cron jobs, and pipelines.

In this guide, you’ll:

• Understand why on-device AI is worth it on Linux.

• Set up a clean, reproducible environment.

• Build 4 practical AI projects you can run today from your terminal.

• Get distro-specific install commands for apt, dnf, and zypper.

Why AI on Linux (and why now)?

• Privacy and control: Keep data local—no third-party services.

• Reproducibility: Pin versions, script everything, ship a requirements.txt.

• Ecosystem maturity: PyTorch, Transformers, scikit-learn, and Vosk all run well on CPUs.

• Composability: Bash pipelines glue everything together: journalctl | python script.py | jq ...

Prerequisites: System packages and Python environment

Install system dependencies (replace with your distro’s package manager):

• Debian/Ubuntu (apt):

  sudo apt update
  sudo apt install -y python3 python3-venv python3-pip git build-essential gcc g++ make \
    wget unzip sox portaudio19-dev libsndfile1
  

• Fedora/RHEL (dnf):

  sudo dnf install -y python3 python3-pip python3-virtualenv git gcc-c++ make \
    wget unzip sox portaudio-devel libsndfile
  

• openSUSE (zypper):

  sudo zypper refresh
  sudo zypper install -y python3 python3-pip python3-venv git gcc-c++ make \
    wget unzip sox portaudio-devel libsndfile1
  

Create and activate a Python virtual environment:

python3 -m venv ~/.venvs/ai
source ~/.venvs/ai/bin/activate
python -m pip install -U pip setuptools wheel

Install core Python libraries:

# PyTorch (CPU wheels)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu

# NLP + local models
pip install transformers accelerate sentencepiece safetensors

# Classic ML + data tools
pip install scikit-learn numpy pandas matplotlib

# Speech recognition stack (offline)
pip install vosk soundfile sounddevice

Note: The sounddevice package uses PortAudio; we installed system libraries above.

---

Project 1: A local terminal LLM (no API keys)

Goal: Generate text locally (summaries, boilerplate, ideas) from the command line.

Why this is useful:

• Quick drafts and prompts without shipping your text to a cloud service.

• Works well for small tasks on CPU using compact models.

Install (already covered in prerequisites) and create a minimal script:

llm_tty.py

#!/usr/bin/env python3
import sys
from transformers import pipeline

def main():
    prompt = " ".join(sys.argv[1:]) or "Write a one-paragraph tip for Linux beginners."
    # A compact model that runs on CPU. For better quality, try 'gpt2' or 'distilgpt2'.
    gen = pipeline("text-generation", model="distilgpt2", device_map="auto")
    out = gen(prompt, max_new_tokens=120, do_sample=True, temperature=0.8)[0]["generated_text"]
    print(out)

if __name__ == "__main__":
    main()

Run it:

python llm_tty.py "Suggest 5 bash aliases for faster Git workflows."

Tips:

• For multilingual or instruction-tuned behavior, try small instruct models (be mindful of RAM).

• Cache lives in ~/.cache/huggingface/; your second run will be much faster.

---

Project 2: Image classification from the CLI (ResNet-18)

Goal: Classify images locally with a pretrained CNN and print top-5 labels.

Why this is useful:

• Batch-tag photos, build quick content moderation filters, or pre-label datasets.

• Works offline; moderate CPU usage.

Install prerequisites (already covered) and create:

classify_image.py

#!/usr/bin/env python3
import sys
import torch
from PIL import Image
from torchvision import models

def main():
    if len(sys.argv) < 2:
        print("Usage: classify_image.py <image>")
        sys.exit(1)
    path = sys.argv[1]
    weights = models.ResNet18_Weights.DEFAULT
    model = models.resnet18(weights=weights)
    model.eval()
    preprocess = weights.transforms()

    img = Image.open(path).convert("RGB")
    x = preprocess(img).unsqueeze(0)
    with torch.no_grad():
        probs = torch.softmax(model(x), dim=1)[0]
    topv, topi = probs.topk(5)
    labels = [weights.meta["categories"][i] for i in topi.tolist()]
    for p, l in zip(topv.tolist(), labels):
        print(f"{p:.4f}  {l}")

if __name__ == "__main__":
    main()

Run it:

pip install pillow
python classify_image.py ./sample.jpg

Batch example:

find ./photos -type f -iname "*.jpg" -print0 | xargs -0 -I{} python classify_image.py "{}" | sed 's/^/[{}] /'

---

Project 3: Offline speech-to-text with Vosk

Goal: Transcribe audio files completely offline.

Why this is useful:

• Meeting notes, voice memos, and CLI documentation without sending audio to cloud services.

• Easy to script: convert > transcribe > grep > summarize.

Download a small English model:

wget https://alphacephei.com/vosk/models/vosk-model-small-en-us-0.15.zip
unzip vosk-model-small-en-us-0.15.zip
mv vosk-model-small-en-us-0.15 models-en

Create a transcriber for WAV/PCM audio:

stt_file.py

#!/usr/bin/env python3
import sys, json, wave
from vosk import Model, KaldiRecognizer

def main():
    if len(sys.argv) < 3:
        print("Usage: stt_file.py <model_dir> <audio.wav>")
        sys.exit(1)
    model_dir, wav_path = sys.argv[1], sys.argv[2]
    model = Model(model_dir)
    wf = wave.open(wav_path, "rb")
    if wf.getnchannels() != 1 or wf.getsampwidth() != 2 or wf.getframerate() not in (8000, 16000, 32000, 44100, 48000):
        print("Please provide mono 16-bit WAV. Convert with: sox input.* -r 16000 -c 1 -b 16 output.wav")
        sys.exit(2)
    rec = KaldiRecognizer(model, wf.getframerate())
    rec.SetWords(True)
    text = []
    while True:
        data = wf.readframes(4000)
        if len(data) == 0:
            break
        if rec.AcceptWaveform(data):
            text.append(json.loads(rec.Result())["text"])
    text.append(json.loads(rec.FinalResult())["text"])
    print(" ".join(t for t in text if t))

if __name__ == "__main__":
    main()

Convert and transcribe:

# Convert any audio to mono 16 kHz WAV
sox input.mp3 -r 16000 -c 1 -b 16 input.wav

# Transcribe
python stt_file.py ./models-en ./input.wav

Optional (live mic):

• Mic access needs working PortAudio. If you installed the packages above, try sounddevice-based scripts from the Vosk examples repo. For clean automation, prefer file-based transcription in CI/cron.

---

Project 4: Log anomaly detector (Isolation Forest)

Goal: Flag “weird” log lines from system logs using classic ML.

Why this is useful:

• Surface unusual events quickly without crafting complex rules.

• Works on any text log: SSH, systemd, app logs.

Collect logs:

# systemd-based systems (most modern distros):
journalctl --since "24 hours ago" -o short-iso > logs.txt

# If journalctl isn't available or you prefer file logs:
# Debian/Ubuntu:
sudo tail -n 20000 /var/log/syslog > logs.txt
# RHEL/Fedora/openSUSE:
sudo tail -n 20000 /var/log/messages > logs.txt

Create the detector:

log_anomaly.py

#!/usr/bin/env python3
import sys
import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.ensemble import IsolationForest

def main():
    if len(sys.argv) < 2:
        print("Usage: log_anomaly.py <logs.txt> [contamination]")
        sys.exit(1)
    path = sys.argv[1]
    contamination = float(sys.argv[2]) if len(sys.argv) > 2 else 0.02

    lines = [l.strip() for l in open(path, "r", errors="ignore") if l.strip()]
    if len(lines) < 100:
        print("Not enough log lines; collect more data.")
        sys.exit(2)

    # Tokenize by whitespace; include bigrams for more context.
    vec = TfidfVectorizer(ngram_range=(1,2), token_pattern=r"[^ ]+")
    X = vec.fit_transform(lines)

    iso = IsolationForest(n_estimators=200, contamination=contamination, random_state=0)
    iso.fit(X)

    scores = -iso.decision_function(X)  # higher = more anomalous
    idx = np.argsort(scores)[-50:][::-1]  # top 50 anomalies

    for i in idx:
        print(f"[score={scores[i]:.4f}] {lines[i]}")

if __name__ == "__main__":
    main()

Run it:

pip install scikit-learn numpy
python log_anomaly.py logs.txt 0.02 | tee anomalies.txt

Pipeline-friendly usage:

journalctl --since "6 hours ago" -o short-iso | tee /tmp/j.log \
  && python log_anomaly.py /tmp/j.log 0.01 | grep -i "ssh\|sudo\|failed"

---

Bash-first tips to tie it all together

• Glue components:

  • Summarize anomalies: python log_anomaly.py logs.txt | head -n 10 | python llm_tty.py "Summarize these anomalies:"
  • Batch transcribe then search: for f in *.wav; do python stt_file.py models-en "$f" > "$f.txt"; done && grep -i "error" *.txt

• Reproducibility: freeze Python deps:

  pip freeze > requirements.txt
  

• Schedule with cron/systemd timers for regular scans or batch transcriptions.

---

Conclusion and Next Steps

You now have four fully local AI tools you can run from Bash:

• A terminal LLM for quick drafts.

• An image classifier you can batch over folders.

• Offline speech-to-text.

• Anomaly detection for logs.

Next steps:

• Wrap each script in a Makefile or simple CLI.

• Containerize with Podman/Docker for isolated runs.

• Try larger or domain-specific models as your hardware allows.

• Share your scripts, open issues, and iterate—Linux AI thrives on community.

Have another use case in mind? Reply with your goal and distro, and I’ll help you sketch the pipeline and pick the right models.