Artificial Intelligence Chatbot Projects on Linux (Bash First)

If you’ve ever wished your terminal could talk back—help you triage logs, summarize docs, or answer teammate questions—now’s the moment. Modern open-source LLMs run locally, play nicely with Bash, and scale from solo hacks to real services. This guide shows you why building chatbots on Linux is a great idea and walks you through three practical, bash-friendly projects you can ship today.

• Problem: Cloud AI can be costly, slow, and risky for private data.

• Value: Local chatbots are fast, private, automatable, and easy to wire into your Linux workflows with shell scripts.

Why chatbots on Linux make sense

• Control and privacy: Keep data on your box or VPS. No mystery APIs, no vendor lock-in.

• Cost and performance: CPU-only models are usable; GPU acceleration is a bonus. Your infra, your budget.

• DevOps-native: Systemd services, cron jobs, pipes, and logs—Bash glues everything together.

• Reproducibility: Package installs and scripts are explicit; simple to debug and CI/CD-friendly.

Prerequisites (install via your package manager)

We’ll use curl, git, jq (for JSON), Python 3 (with venv/pip), and basic build tools (useful for various AI libs).

For Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y git curl jq python3 python3-venv python3-pip build-essential cmake

For Fedora/RHEL (dnf):

sudo dnf groupinstall -y "Development Tools"
sudo dnf install -y git curl jq python3 python3-pip cmake

For openSUSE (zypper):

sudo zypper refresh
sudo zypper install -y git curl jq python3 python3-venv python3-pip gcc gcc-c++ make cmake

Tip: If python3’s venv module is missing on your distro, install the venv/virtualenv package for it (already included above for apt/zypper; Fedora’s python3 includes venv).

---

Project 1: A 10-minute Terminal Chatbot with Ollama

Ollama runs LLMs locally with a simple CLI and an HTTP API, perfect for Bash scripts.

1) Install Ollama:

curl -fsSL https://ollama.com/install.sh | sh

This sets up the daemon (systemd) and the ollama CLI.

2) Pull a model (start small; you can upgrade later):

ollama pull llama3

3) Chat interactively:

ollama run llama3

4) Add a Bash wrapper that keeps lightweight context per “session”:

ai() {
  local session="${1:-default}"
  shift || true
  local store="${XDG_DATA_HOME:-$HOME/.local/share}/ai_chat"
  mkdir -p "$store"
  local log="$store/$session.txt"

  if [ ! -s "$log" ]; then
    printf "System: You are a concise Linux assistant.\n" > "$log"
  fi

  local prompt
  if [ -t 0 ]; then
    prompt="$*"
  else
    prompt="$(cat)"
  fi

  printf "User: %s\n" "$prompt" >> "$log"
  local ctx
  ctx="$(tail -n 120 "$log")"  # keep context short

  # Ask the model; stream is handled by ollama
  local reply
  reply="$(printf "%s\nAssistant:" "$ctx" | ollama run llama3)"

  # Show and log
  echo "$reply"
  {
    echo "Assistant:"
    echo "$reply"
  } >> "$log"
}

Examples:

ai "Explain cgroups in two sentences."
echo "Summarize the top 5 lines of this log:" | ai logs

Why it’s useful:

• Fast iteration for shell-based tasks.

• Scriptable: pipe input from files, system tools, or other commands.

• Easy to extend (e.g., add a --model flag, strip ANSI, or rotate logs).

---

Project 2: Your Docs Q&A Bot (Local RAG) with FAISS + Ollama

Make your chatbot answer questions using your own documents (RAG = Retrieval Augmented Generation). We’ll embed your docs, search for relevant chunks, and ask the model to answer from those.

1) Create a Python virtual environment and install libraries:

python3 -m venv .venv
. .venv/bin/activate
pip install --upgrade pip
pip install sentence-transformers faiss-cpu pypdf requests

2) Ingest your docs into a FAISS index.

Save as ingest.py:

#!/usr/bin/env python3
import sys, os, pickle, faiss
from pathlib import Path
from sentence_transformers import SentenceTransformer
from pypdf import PdfReader

def read_text(path: Path) -> str:
    if path.suffix.lower() in {".txt", ".md"}:
        return path.read_text(errors="ignore")
    if path.suffix.lower() == ".pdf":
        reader = PdfReader(str(path))
        return "\n".join(page.extract_text() or "" for page in reader.pages)
    return ""

def chunk_paragraphs(text, max_chars=800):
    paras = [p.strip() for p in text.split("\n\n") if p.strip()]
    chunks, buf = [], ""
    for p in paras:
        if len(buf) + len(p) + 2 <= max_chars:
            buf = (buf + "\n\n" + p).strip()
        else:
            if buf: chunks.append(buf)
            buf = p
    if buf: chunks.append(buf)
    return chunks

def main():
    if len(sys.argv) < 3:
        print("Usage: ingest.py <docs_dir> <out_dir>")
        sys.exit(1)
    docs_dir = Path(sys.argv[1]).expanduser().resolve()
    out_dir = Path(sys.argv[2]).expanduser().resolve()
    out_dir.mkdir(parents=True, exist_ok=True)

    model_name = "sentence-transformers/all-MiniLM-L6-v2"
    model = SentenceTransformer(model_name)
    dim = model.get_sentence_embedding_dimension()

    texts, metas = [], []
    for path in docs_dir.rglob("*"):
        if path.is_dir(): continue
        if path.suffix.lower() not in {".txt", ".md", ".pdf"}: continue
        try:
            text = read_text(path)
            for chunk in chunk_paragraphs(text):
                texts.append(chunk)
                metas.append({"source": str(path)})
        except Exception as e:
            print(f"Skip {path}: {e}", file=sys.stderr)

    if not texts:
        print("No documents found.")
        sys.exit(1)

    import numpy as np
    embs = model.encode(texts, batch_size=64, show_progress_bar=True, convert_to_numpy=True)
    index = faiss.IndexFlatIP(dim)
    # Normalize for cosine similarity via dot product
    faiss.normalize_L2(embs)
    index.add(embs)

    faiss.write_index(index, str(out_dir / "index.faiss"))
    with open(out_dir / "texts.pkl", "wb") as f:
        pickle.dump(texts, f)
    with open(out_dir / "metas.pkl", "wb") as f:
        pickle.dump(metas, f)
    with open(out_dir / "model.txt", "w") as f:
        f.write(model_name)

    print(f"Ingested {len(texts)} chunks into {out_dir}")

if __name__ == "__main__":
    main()

3) Ask questions with retrieval and generation.

Save as ask.py:

#!/usr/bin/env python3
import sys, pickle, faiss, requests
from pathlib import Path
import numpy as np
from sentence_transformers import SentenceTransformer
from textwrap import shorten

OLLAMA_MODEL = os.environ.get("OLLAMA_MODEL", "llama3")
OLLAMA_URL = os.environ.get("OLLAMA_URL", "http://localhost:11434/api/generate")

def load_index(out_dir: Path):
    index = faiss.read_index(str(out_dir / "index.faiss"))
    texts = pickle.load(open(out_dir / "texts.pkl", "rb"))
    metas = pickle.load(open(out_dir / "metas.pkl", "rb"))
    model_name = (out_dir / "model.txt").read_text().strip()
    model = SentenceTransformer(model_name)
    return index, texts, metas, model

def retrieve(index, model, query, k=5):
    q = model.encode([query], convert_to_numpy=True)
    faiss.normalize_L2(q)
    sims, idxs = index.search(q, k)
    return idxs[0], sims[0]

def ask_ollama(prompt: str) -> str:
    r = requests.post(OLLAMA_URL, json={"model": OLLAMA_MODEL, "prompt": prompt, "stream": False}, timeout=300)
    r.raise_for_status()
    data = r.json()
    return data.get("response", "").strip()

def main():
    if len(sys.argv) < 3:
        print("Usage: ask.py <index_dir> <query...>")
        sys.exit(1)
    out_dir = Path(sys.argv[1])
    query = " ".join(sys.argv[2:])
    index, texts, metas, model = load_index(out_dir)
    idxs, sims = retrieve(index, model, query, k=5)
    context_blocks = []
    for i in idxs:
        if i < 0: continue
        context_blocks.append(f"- {shorten(metas[i]['source'], width=80)}\n{texts[i]}")
    context = "\n\n".join(context_blocks)

    prompt = f"""You are a helpful assistant. Answer the question strictly using the provided context.
If the context is insufficient, say you don't know.

Question:
{query}

Context:
{context}

Answer:"""

    answer = ask_ollama(prompt)
    print(answer)

if __name__ == "__main__":
    import os
    main()

4) Run it:

# 1) Start Ollama if not already running
sudo systemctl enable --now ollama

# 2) Pull a model (done earlier)
ollama pull llama3

# 3) Ingest your docs
python3 ingest.py ~/docs ./rag_index

# 4) Ask questions
python3 ask.py ./rag_index "How do I deploy our service with systemd?"

Why it’s useful:

• Real answers grounded in your own knowledge base.

• Fully local: embeddings + LLM run on your machine.

• Easy to automate (cron, scripts, or webhooks).

---

Project 3: A Telegram Chatbot That Uses Your Local Model

Let your team message a bot; your Linux box answers using a local LLM via Ollama.

1) Create a Telegram Bot (via @BotFather) and grab the token.

2) Install Python deps (in your existing venv is fine):

pip install python-telegram-bot requests

3) Save as telegram_bot.py:

#!/usr/bin/env python3
import os, asyncio, requests
from telegram import Update
from telegram.constants import ChatAction
from telegram.ext import ApplicationBuilder, ContextTypes, MessageHandler, filters

TOKEN = os.environ.get("TELEGRAM_BOT_TOKEN")
OLLAMA_MODEL = os.environ.get("OLLAMA_MODEL", "llama3")
OLLAMA_URL = os.environ.get("OLLAMA_URL", "http://localhost:11434/api/generate")

def ask_ollama(prompt: str) -> str:
    r = requests.post(OLLAMA_URL, json={"model": OLLAMA_MODEL, "prompt": prompt, "stream": False}, timeout=300)
    r.raise_for_status()
    return r.json().get("response", "").strip()

async def handle_message(update: Update, context: ContextTypes.DEFAULT_TYPE):
    text = update.message.text or ""
    await context.bot.send_chat_action(chat_id=update.effective_chat.id, action=ChatAction.TYPING)
    system = "You are a concise assistant. Keep answers short unless asked for details."
    prompt = f"{system}\n\nUser: {text}\nAssistant:"
    try:
        reply = ask_ollama(prompt)
    except Exception as e:
        reply = f"Error talking to local model: {e}"
    await update.message.reply_text(reply)

async def main():
    if not TOKEN:
        raise RuntimeError("Set TELEGRAM_BOT_TOKEN in environment")
    app = ApplicationBuilder().token(TOKEN).build()
    app.add_handler(MessageHandler(filters.TEXT & (~filters.COMMAND), handle_message))
    await app.initialize()
    await app.start()
    await app.updater.start_polling()
    await app.updater.idle()

if __name__ == "__main__":
    asyncio.run(main())

4) Run it:

export TELEGRAM_BOT_TOKEN="123456:ABCDEF..."
# Optional overrides:
# export OLLAMA_MODEL="llama3"
# export OLLAMA_URL="http://localhost:11434/api/generate"

python3 telegram_bot.py

Optional: run as a user service with systemd.

~/.config/systemd/user/telegram-llm.service:

[Unit]
Description=Telegram LLM Bot (local Ollama)

[Service]
Environment=TELEGRAM_BOT_TOKEN=123456:ABCDEF...
Environment=OLLAMA_MODEL=llama3
Environment=OLLAMA_URL=http://localhost:11434/api/generate
ExecStart=%h/path/to/.venv/bin/python %h/path/to/telegram_bot.py
WorkingDirectory=%h/path/to
Restart=on-failure

[Install]
WantedBy=default.target

Enable and start:

systemctl --user daemon-reload
systemctl --user enable --now telegram-llm.service

Why it’s useful:

• Team-friendly interface (no terminals required).

• Private: your data stays on your server.

• Extensible: add auth, slash commands, or RAG integration.

---

Troubleshooting and tips

• RAM/VRAM matters: Smaller models like llama3 or mistral variants work well on 8–16 GB RAM. Prefer quantized models (e.g., Q4_K_M) for CPUs.

• Keep prompts short: Use concise system prompts and trim context to prevent slow or off-topic responses.

• Log everything: Store prompts and outputs for reproducibility and debugging. Your ai() function already keeps per-session logs.

• Upgrade later: Swap models by name only. For Ollama, ollama pull <model> then update your scripts.

---

Conclusion / Call To Action

You don’t need a cloud bill to build useful AI. On Linux, a few Bash-friendly tools let you:

• Chat locally from your terminal.

• Answer questions using your own docs.

• Offer a team-facing Telegram bot—powered by your box.

Pick one project above and ship it today:

• If you live in the terminal: start with the ai() wrapper.

• If you need real answers from your knowledge base: build the RAG index.

• If your team needs easy access: deploy the Telegram bot.

When you’re ready, combine them: hook your Telegram bot to the RAG index and logs, add systemd timers for re-indexing, and iterate. Happy hacking!