Building a Personal Linux Artificial Intelligence Assistant (with Bash)

What if you could ask your terminal for help, in plain English, and get back rock‑solid shell commands, explanations, or even a voice that reads out the answer while you keep typing? In this guide, you’ll build a small, hackable AI assistant for Linux that runs from your shell, optionally speaks, and can use either a cloud model (OpenAI) or a local model (Ollama).

You’ll learn:

• Why building your own assistant is worth it

• How to set up dependencies using apt, dnf, and zypper

• How to write a minimal Bash assistant that keeps context

• How to add voice output (text‑to‑speech) and voice input (speech‑to‑text)

• Practical tips to integrate it into your daily workflow

Why build your own assistant?

• Privacy and control: Choose local models for sensitive work, or cloud models when you need accuracy and tools like speech and vision.

• Speed and flow: Stay in your terminal, keep your hands on the keyboard, and get just‑in‑time help (commands, scripts, explanations) without context switching.

• Extensibility: It’s just Bash and curl. You can pipe data in, cache outputs, add hotkeys, or wire it to dotfiles and task runners.

What you’ll build

• A tiny Bash program that:

  • Maintains a short conversation history
  • Uses either OpenAI (cloud) or Ollama (local) as the backend
  • Optionally speaks answers (TTS) and understands short voice questions (STT)

• No heavy frameworks; just standard Linux tools plus curl and jq.

---

Prerequisites

Install these tools first. Pick the command set for your distro.

• curl: HTTP client to call APIs

• jq: JSON parser (to extract answers)

• mpv or ffmpeg: audio playback for TTS

• alsa-utils: arecord for quick voice capture (optional, for STT)

Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y curl jq mpv ffmpeg alsa-utils

Fedora/RHEL (dnf):

sudo dnf install -y curl jq mpv ffmpeg alsa-utils
# Note: On some Fedora releases you may need ffmpeg-free instead of ffmpeg.

openSUSE (zypper):

sudo zypper refresh
sudo zypper install -y curl jq mpv ffmpeg alsa-utils

Optional (local model backend, Ollama):

curl -fsSL https://ollama.com/install.sh | sh
# Then start ollama if needed:
# ollama serve
# And pull a model, e.g.:
# ollama pull llama3

---

Step 1: Choose a backend and add credentials

Option A: OpenAI (cloud, high quality text/voice)

• Create an API key and export it:

export OPENAI_API_KEY="sk-...your-key..."

Consider adding that line to your shell profile (e.g., ~/.bashrc) and restricting permissions:

chmod 600 ~/.bashrc

Option B: Ollama (local, no internet required)

• After installing, pull a model:

ollama pull llama3

• No key needed. Ensure ollama serve is running (it usually starts automatically).

---

Step 2: A minimal Bash assistant (text-only)

Create a file named assistant.sh and make it executable. This version supports OpenAI or Ollama and keeps a lightweight history.

#!/usr/bin/env bash
set -euo pipefail

# Config
BACKEND="${BACKEND:-openai}"            # openai | ollama
OPENAI_MODEL="${OPENAI_MODEL:-gpt-4o-mini}"
OLLAMA_MODEL="${OLLAMA_MODEL:-llama3}"
HIST_DIR="${XDG_STATE_HOME:-$HOME/.local/state}/bash-ai"
HIST_FILE="$HIST_DIR/history.txt"
SYSTEM_PROMPT="You are a concise, trustworthy Linux assistant.

- Prefer bash one-liners and explain briefly.

- Never execute commands; only suggest them.

- When risky, show a safer dry-run or confirmation pattern."

mkdir -p "$HIST_DIR"
touch "$HIST_FILE"

prompt="${*:-}"
if [[ -z "$prompt" ]]; then
  read -rp "You> " prompt
fi

# Keep the last ~2 turns to control token size
history_tail="$(tail -n 60 "$HIST_FILE" 2>/dev/null || true)"

build_prompt() {
  printf "%s\n\nConversation so far:\n%s\n\nUser: %s\nAssistant:" \
    "$SYSTEM_PROMPT" "$history_tail" "$prompt"
}

ask_openai() {
  : "${OPENAI_API_KEY:?Set OPENAI_API_KEY environment variable}"
  local full_input
  full_input="$(build_prompt)"

  # Build JSON safely with jq
  local payload
  payload="$(jq -n --arg model "$OPENAI_MODEL" --arg input "$full_input" \
              '{model:$model, input:$input}')"

  curl -sS https://api.openai.com/v1/responses \
    -H "Authorization: Bearer '"$OPENAI_API_KEY"'" \
    -H "Content-Type: application/json" \
    -d "$payload" |
    jq -r '.output[0].content[0].text // .output_text // .choices[0].message.content // empty'
}

ask_ollama() {
  local full_input
  full_input="$(build_prompt)"
  local payload
  payload="$(jq -n --arg model "$OLLAMA_MODEL" --arg prompt "$full_input" \
              '{model:$model, prompt:$prompt, stream:false}')"
  curl -sS http://localhost:11434/api/generate \
    -H "Content-Type: application/json" \
    -d "$payload" | jq -r '.response'
}

answer=""
case "$BACKEND" in
  openai) answer="$(ask_openai)" ;;
  ollama) answer="$(ask_ollama)" ;;
  *) echo "Unknown BACKEND=$BACKEND (use openai|ollama)"; exit 1 ;;
esac

# Show and store
printf "AI> %s\n" "$answer"

{
  printf "User: %s\n" "$prompt"
  printf "Assistant: %s\n\n" "$answer"
} >> "$HIST_FILE"

Make it executable:

chmod +x assistant.sh

Try it:

./assistant.sh "Find world-writable files under /var but exclude logs, and explain the flags."

Switch to Ollama backend:

BACKEND=ollama ./assistant.sh "Summarize the key differences between apt, dnf, and zypper."

Tip: Add an alias to your shell:

echo 'alias ai="BACKEND=openai ~/assistant.sh"' >> ~/.bashrc
source ~/.bashrc
ai "Generate a safe rsync command to mirror ~/Pictures to /mnt/backup."

---

Step 3: Add voice output (text-to-speech, TTS)

This uses OpenAI’s TTS API to synthesize audio from the assistant’s reply. You’ll need your OPENAI_API_KEY set, plus mpv or ffplay to play the audio.

Create say.sh:

#!/usr/bin/env bash
set -euo pipefail
: "${OPENAI_API_KEY:?Set OPENAI_API_KEY}"

text="${*:-}"
if [[ -z "$text" ]]; then
  echo "Usage: $0 'text to speak'"
  exit 1
fi

tmp="$(mktemp --suffix=.mp3)"
payload="$(jq -n --arg input "$text" \
  '{model:"gpt-4o-mini-tts", voice:"alloy", input:$input}')"

curl -sS https://api.openai.com/v1/audio/speech \
  -H "Authorization: Bearer '"$OPENAI_API_KEY"'" \
  -H "Content-Type: application/json" \
  -d "$payload" -o "$tmp"

# Play (pick one)
if command -v mpv >/dev/null; then
  mpv --really-quiet --no-video "$tmp" >/dev/null 2>&1
elif command -v ffplay >/dev/null; then
  ffplay -nodisp -autoexit -loglevel quiet "$tmp"
else
  echo "Install mpv or ffmpeg to play audio"
fi

rm -f "$tmp"

Make it executable:

chmod +x say.sh

Use it with your assistant:

reply="$(./assistant.sh "In one paragraph, how does SSH key auth work?")"
echo "$reply"
./say.sh "$reply"

---

Step 4: Add voice input (speech-to-text, STT)

Quickly record audio with arecord, then transcribe it using OpenAI’s Whisper model.

Record a short question (6 seconds):

arecord -d 6 -f cd -t wav ask.wav

Transcribe:

curl -sS -X POST "https://api.openai.com/v1/audio/transcriptions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: multipart/form-data" \
  -F "file=@ask.wav" \
  -F "model=whisper-1" | jq -r '.text'

Wire it together in ask-voice.sh:

#!/usr/bin/env bash
set -euo pipefail
: "${OPENAI_API_KEY:?Set OPENAI_API_KEY}"

tmpwav="$(mktemp --suffix=.wav)"
echo "Recording (press Ctrl+C to cancel)..."
arecord -d 6 -f cd -t wav "$tmpwav" >/dev/null 2>&1 || true

question="$(
  curl -sS -X POST "https://api.openai.com/v1/audio/transcriptions" \
    -H "Authorization: Bearer $OPENAI_API_KEY" \
    -H "Content-Type: multipart/form-data" \
    -F "file=@$tmpwav" \
    -F "model=whisper-1" | jq -r '.text'
)"
rm -f "$tmpwav"

echo "You (transcribed)> $question"
answer="$(./assistant.sh "$question" | sed 's/^AI> //')"
echo "AI> $answer"
./say.sh "$answer"

Make it executable:

chmod +x ask-voice.sh

Run it:

./ask-voice.sh

Tip: If your mic is not the default device, list devices with:

arecord -l

Then choose one using -D hw:card,device:

arecord -D hw:1,0 -d 6 -f cd -t wav ask.wav

---

Real-world examples you can try today

• Generate safe file operations:

  • “Write an rsync command to mirror ~/Projects to /mnt/backup, show progress, and exclude node_modules. Explain each flag.”

• One-liners with explanation:

  • “Give me a one-liner to list the 10 largest files under /var/www and explain how it works.”

• Package troubleshooting:

  • “dnf vs apt vs zypper: how do I search, install, and remove packages in each? Provide commands and short notes.”

• System introspection:

  • “Create a command to show the top 5 processes by memory with PID, command, and RSS in MiB.”

• Shell safety:

  • “Provide a dry-run deletion for files older than 30 days in /tmp and then the real command after confirmation.”

---

3–5 actionable tips to make it yours

1) Create shell shortcuts

• Add an alias for quick calls:

echo 'alias ai="BACKEND=openai ~/assistant.sh"' >> ~/.bashrc
echo 'alias ail="BACKEND=ollama ~/assistant.sh"' >> ~/.bashrc
source ~/.bashrc

2) Keep context small and relevant

• The script already tails the last ~60 lines. Adjust this for cost/speed:

history_tail="$(tail -n 120 "$HIST_FILE")"   # more context

3) Teach it your project

• Pipe docs or code summaries into the prompt:

summary="$(grep -R --include='*.sh' -n '' ./ | head -n 200)"
./assistant.sh "Given this project context:\n$summary\n\nQuestion: How do I add a make target to run tests in parallel?"

4) Confirm before running commands

• Ask the assistant for a command, then copy/paste after reading. Or add a wrapper to require explicit “yes” before execution.

5) Go fully local if needed

• Use BACKEND=ollama with a capable local model (e.g., llama3) for offline, private workflows. Tune prompts for concise, deterministic output.

---

Notes on cost, privacy, and safety

• Cloud usage costs money and sends data to the provider. Keep secrets out of prompts, or go local with Ollama for sensitive material.

• Store your API key in a protected file and limit permissions (chmod 600).

• Never auto-execute commands from the model. Review, understand, then run.

---

Conclusion and next steps (CTA)

You now have a personal Linux AI assistant that lives in your terminal, can keep a short memory, and even supports voice in/out. From here:

• Pick your default backend: openai for best accuracy and TTS/STT, or ollama for private, offline use.

• Add hotkeys: map ./ask-voice.sh to a shortcut in your desktop environment.

• Extend it: add functions to search man pages, read logs, or scaffold scripts on command.

If this boosted your shell superpowers, turn it into a dotfile project and share your tweaks. Your future self (and your team) will thank you.