Posted on
• Artificial Intelligence

Artificial Intelligence Voice Assistants on Linux

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence Voice Assistants on Linux: From Zero to “Hey, Computer”

Voice assistants are everywhere—phones, speakers, cars—but your Linux desktop is still tapping on a keyboard. That’s a missed opportunity. With today’s open-source tools, you can build a private, offline voice assistant on Linux that launches apps, controls audio, and runs shell commands—no telemetry, no subscriptions.

This guide shows you why voice on Linux is worth your time, then walks you through practical, working setups: a 10‑minute offline assistant using Vosk + espeak-ng, and higher-accuracy transcription with Whisper.cpp. You’ll finish with real command examples and a path to keep building.

Why voice on Linux is ready now

  • Privacy by default: Open‑source offline STT/TTS (speech‑to‑text/text‑to‑speech) options like Vosk, Whisper, Piper, and espeak-ng keep your voice local.

  • Excellent CLI integration: “Do it with voice” is a natural extension of shell workflows and dotfiles.

  • Hardware support has matured: PipeWire/PulseAudio and ALSA make mics/speakers reliable and scriptable.

1) Prepare your system: audio + tools

First, install essential audio and Python tooling. Choose your distro commands.

  • Debian/Ubuntu (apt):
sudo apt update
sudo apt install -y \
  alsa-utils sox espeak-ng python3 python3-venv python3-pip git unzip wget \
  portaudio19-dev python3-pyaudio
  • Fedora (dnf):
sudo dnf install -y \
  alsa-utils sox espeak-ng python3 python3-virtualenv python3-pip git unzip wget \
  portaudio-devel python3-pyaudio
  • openSUSE (zypper):
sudo zypper install -y \
  alsa-utils sox espeak-ng python3 python3-virtualenv python3-pip git unzip wget \
  portaudio-devel python3-pyaudio

Quick audio checks:

arecord -l           # list capture devices
arecord -f S16_LE -r 16000 -c 1 -d 3 /tmp/test.wav && aplay /tmp/test.wav

If you use PipeWire, pactl still works:

pactl list short sources   # find your mic
pactl set-source-mute @DEFAULT_SOURCE@ 0
pactl set-source-volume @DEFAULT_SOURCE@ 75%

Tip: If your mic is quiet or clipped, adjust it in your desktop mixer or with pavucontrol.

2) Build a 10‑minute offline assistant (Vosk STT + espeak‑ng TTS)

This minimal setup records your voice, recognizes text offline, runs a command, and speaks back.

  • Create a virtual environment and install Vosk:
python3 -m venv ~/va-venv
source ~/va-venv/bin/activate
pip install --upgrade pip
pip install vosk
  • Download a small English model (adjust if you prefer another language from https://alphacephei.com/vosk/models):
mkdir -p ~/models && cd ~/models
wget https://alphacephei.com/vosk/models/vosk-model-small-en-us-0.15.zip
unzip vosk-model-small-en-us-0.15.zip
  • Record a short command (5 seconds):
arecord -f S16_LE -r 16000 -c 1 -d 5 /tmp/cmd.wav
  • Create a tiny Python runner that turns voice into actions:

Save as ~/voice_do.py:

#!/usr/bin/env python3
import json, sys, subprocess, wave, os
from vosk import Model, KaldiRecognizer

MODEL_DIR = os.environ.get("VOSK_MODEL", os.path.expanduser("~/models/vosk-model-small-en-us-0.15"))

def transcribe(wav_path):
    wf = wave.open(wav_path, "rb")
    if wf.getnchannels() != 1 or wf.getsampwidth() != 2:
        raise RuntimeError("Use 16‑bit mono WAV (arecord example provided).")
    rec = KaldiRecognizer(Model(MODEL_DIR), wf.getframerate())
    text = ""
    while True:
        data = wf.readframes(4000)
        if len(data) == 0:
            break
        rec.AcceptWaveform(data)
    try:
        text = json.loads(rec.FinalResult()).get("text","").strip()
    except json.JSONDecodeError:
        pass
    return text

def speak(msg):
    if not msg:
        msg = "Done."
    subprocess.run(["espeak-ng", msg], check=False)

def run_actions(text):
    text_l = text.lower()
    if "volume up" in text_l:
        subprocess.run(["pactl", "set-sink-volume", "@DEFAULT_SINK@", "+5%"])
        speak("Volume up.")
    elif "volume down" in text_l:
        subprocess.run(["pactl", "set-sink-volume", "@DEFAULT_SINK@", "-5%"])
        speak("Volume down.")
    elif "mute" in text_l and "microphone" in text_l:
        subprocess.run(["pactl", "set-source-mute", "@DEFAULT_SOURCE@", "1"])
        speak("Microphone muted.")
    elif "open browser" in text_l or "start browser" in text_l:
        subprocess.Popen(["xdg-open", "https://example.org"])
        speak("Opening browser.")
    elif "what time" in text_l:
        out = subprocess.check_output(["date", "+%H:%M"]).decode().strip()
        speak(f"The time is {out}.")
    else:
        speak("I heard: " + text)

if __name__ == "__main__":
    if len(sys.argv) != 2:
        print("Usage: voice_do.py /path/to/file.wav", file=sys.stderr)
        sys.exit(2)
    phrase = transcribe(sys.argv[1])
    run_actions(phrase)

Make it executable and run:

chmod +x ~/voice_do.py
~/voice_do.py /tmp/cmd.wav

Automate the whole flow with one Bash alias (record → recognize → act):

alias hey='arecord -f S16_LE -r 16000 -c 1 -d 5 /tmp/cmd.wav && ~/voice_do.py /tmp/cmd.wav'

Now say “volume up”, “open browser”, or “what time is it?” and run:

hey

Notes:

  • Replace actions in run_actions with your own shell tricks (playerctl for media, nmcli for networks, acpi for battery, xdotool for window actions).

  • For other languages, pick a matching Vosk model and set VOSK_MODEL=/path/to/model.

3) Higher accuracy STT with Whisper.cpp (local, fast)

Whisper.cpp provides near state‑of‑the‑art offline transcription. Build it once, then feed it audio captured via ALSA.

Install build tools:

  • Debian/Ubuntu (apt):
sudo apt update
sudo apt install -y build-essential cmake git
  • Fedora (dnf):
sudo dnf groupinstall -y "Development Tools"
sudo dnf install -y cmake git
  • openSUSE (zypper):
sudo zypper install -y gcc-c++ make cmake git

Build and run:

git clone https://github.com/ggerganov/whisper.cpp
cd whisper.cpp
make -j

# Download an English model (base.en is a good start)
bash ./models/download-ggml-model.sh base.en

# Record and transcribe
arecord -f S16_LE -r 16000 -c 1 -d 6 /tmp/ask.wav
./main -m models/ggml-base.en.bin -f /tmp/ask.wav -nt

Tips:

  • Use tiny.en for speed on older CPUs; small.en or medium.en for better accuracy.

  • Pipe the output into your own command router, or adapt voice_do.py to call Whisper.cpp and parse stdout.

4) Add a hotword and go hands-free (optional)

For a “Hey Computer” wake word, use open-source detectors like openWakeWord (Python). You’ve already installed PortAudio and PyAudio above.

Install and try a basic loop:

source ~/va-venv/bin/activate
pip install openwakeword sounddevice numpy

Example snippet (conceptual starter):

import sounddevice as sd, numpy as np
from openwakeword.model import Model
m = Model(wakeword_models=["hey_jarvis.tflite"])  # pick a model you like
samplerate = 16000
def cb(indata, frames, time, status):
    score = m.predict(indata[:,0])
    if score[0] > 0.5:
        print("Wake word!")
with sd.InputStream(channels=1, samplerate=samplerate, callback=cb, blocksize=512):
    sd.sleep(60_000)

Upon detection, you can trigger the arecord → voice_do.py pipeline. This keeps CPU use low and your mic off most of the time.

5) Real‑world voice + CLI examples

Drop these into your run_actions function or a Bash router:

  • Control media with playerctl (install it first):
    • apt: sudo apt install -y playerctl
    • dnf: sudo dnf install -y playerctl
    • zypper: sudo zypper install -y playerctl
playerctl play-pause
playerctl next
playerctl volume 0.05+     # 5% up
  • Network toggles with nmcli:
    • apt: sudo apt install -y network-manager
    • dnf: sudo dnf install -y NetworkManager
    • zypper: sudo zypper install -y NetworkManager
nmcli radio wifi off
nmcli radio wifi on
  • System info:
uptime -p
sensors            # lm-sensors; apt/dnf/zypper: lm-sensors
acpi -b            # battery; apt/dnf/zypper: acpi
  • Launch apps:
xdg-open https://news.ycombinator.com
nohup thunderbird >/dev/null 2>&1 &

Have your assistant speak confirmations with:

espeak-ng "Wi-Fi disabled"

Troubleshooting quick wins

  • Microphone not heard: Select the right source in your mixer, ensure it’s not muted, and test with arecord.

  • Speech garbled: Use 16 kHz mono, 16‑bit PCM. Set mic gain sanely (no clipping).

  • High CPU with Whisper: Use smaller models (tiny/base) or switch to Vosk for command‑and‑control tasks.

Where to go next

  • Expand your command set and bind into your dotfiles.

  • Add a lightweight GUI indicator or a tray icon to show “listening” states.

  • Try better TTS: Piper voices are high‑quality and still local.

  • Integrate with Home Assistant for whole‑home voice control.

Call to action:

  • Install the prerequisites for your distro.

  • Run the 10‑minute Vosk assistant.

  • Swap in Whisper.cpp for accuracy.

  • Share your dotfiles and scripts with the community—voice can be a first‑class Linux interface, and you can help build it.

Happy hacking—and happy talking!

Related posts