Posted on
• Artificial Intelligence

Voice Artificial Intelligence on Linux

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Voice Artificial Intelligence on Linux: Build a Private, Hackable Voice Stack

If you’ve ever wished your voice assistant were faster, more private, and more hackable, Linux has everything you need. Modern open-source models like Whisper (speech-to-text) and Piper (text-to-speech) run entirely on your machine, without shipping your voice to the cloud. In this guide, you’ll learn why Voice AI on Linux is worth your time and how to stand up a working, offline voice pipeline with a few shell commands.

Why Voice AI on Linux is a great idea

  • Privacy by default: Offline speech models keep your audio local, which is crucial for sensitive recordings or regulated environments.

  • Control and composability: Unix tools and pipes make audio and text processing easy to automate, extend, and debug.

  • Performance is finally “good enough”: Whisper variants transcribe well on CPUs, and Piper produces natural-sounding speech with low latency.

  • Broad hardware support: From laptops to Raspberry Pi, there are models and tools that fit your constraints.

Prerequisites and audio sanity check

We’ll install a small toolkit for audio recording, transcription, and speech synthesis. Choose the commands for your distribution.

  • Debian/Ubuntu (apt):
sudo apt update
sudo apt install -y python3 python3-venv python3-pip git ffmpeg alsa-utils espeak-ng jq
  • Fedora (dnf):
sudo dnf install -y python3 python3-virtualenv python3-pip git ffmpeg alsa-utils espeak-ng jq
  • openSUSE (zypper):
sudo zypper install -y python3 python3-virtualenv python3-pip git ffmpeg alsa-utils espeak-ng jq

Quick mic test:

arecord -l              # list capture devices
arecord -f cd -d 5 test.wav   # record 5s at 16-bit, 44.1kHz
aplay test.wav

If you can hear yourself, your audio stack is ready.

1) Offline Speech-to-Text (STT) with Whisper

We’ll demonstrate two CLIs: the original Whisper and a faster CPU-friendly variant.

Create a Python virtual environment for voice tools:

python3 -m venv ~/.venvs/voice
source ~/.venvs/voice/bin/activate

Option A — OpenAI Whisper CLI (simple, accurate; slower on CPU):

pip install -U openai-whisper

Record and transcribe:

arecord -f cd -d 5 -q input.wav
whisper input.wav --model small --language en --fp16 False
# Output text appears in input.txt
cat input.txt

Option B — Whisper via CTranslate2 (often faster on CPU):

pip install -U whisper-ctranslate2
arecord -f cd -d 5 -q input.wav
whisper-ctranslate2 --model small.en --compute_type int8 --device cpu input.wav
# Output defaults to input.txt in the working directory
cat input.txt

Tips:

  • Choose model sizes: tiny, base, small, medium, large. Start with small or base for speed.

  • For non-English audio, pick a multilingual model (e.g., small, not small.en) and set --language appropriately.

2) Offline Text-to-Speech (TTS): Quick and high-quality options

Quickest voice (classic, robotic) with eSpeak NG:

  • Debian/Ubuntu (apt):
sudo apt install -y espeak-ng
  • Fedora (dnf):
sudo dnf install -y espeak-ng
  • openSUSE (zypper):
sudo zypper install -y espeak-ng

Speak:

espeak-ng -v en-us -s 170 "Hello from Linux Voice AI."

Higher-quality, fast TTS with Piper

Piper provides natural-sounding voices and runs offline. Install the binary and a voice model.

1) Install Piper binary (pick the right asset for your CPU/arch from the releases page):

cd /tmp
curl -L -o piper.tar.gz https://github.com/rhasspy/piper/releases/latest/download/piper_linux_x86_64.tar.gz
tar -xzf piper.tar.gz
sudo install -m 0755 piper/piper /usr/local/bin/piper
piper --help

Note: Replace piper_linux_x86_64.tar.gz with the asset matching your architecture (e.g., aarch64 for ARM).

2) Download a voice model (example: en_US Amy high-quality):

curl -LO https://github.com/rhasspy/piper/releases/download/v0.0.2/en_US-amy-high.onnx
curl -LO https://github.com/rhasspy/piper/releases/download/v0.0.2/en_US-amy-high.onnx.json
echo "This is Piper speaking on Linux." | piper --model en_US-amy-high.onnx --output_file out.wav
aplay out.wav

3) Glue it together: A tiny Bash voice assistant

Let’s combine recording, transcription, command handling, and speech.

Create a helper script say using Piper (fallback to eSpeak NG if Piper isn’t installed).

cat > ~/bin/say << 'EOF'
#!/usr/bin/env bash
set -euo pipefail
TEXT="${*:-Hello}"
if command -v piper >/dev/null 2>&1 && [ -f "$HOME/voices/en_US-amy-high.onnx" ]; then
  echo "$TEXT" | piper --model "$HOME/voices/en_US-amy-high.onnx" --output_file /tmp/say.wav
  aplay -q /tmp/say.wav
else
  espeak-ng "$TEXT"
fi
EOF
chmod +x ~/bin/say

Place the Piper voice where the script expects it:

mkdir -p ~/voices
mv en_US-amy-high.onnx en_US-amy-high.onnx.json ~/voices/

Now the assistant script assistant.sh:

cat > ~/bin/assistant.sh << 'EOF'
#!/usr/bin/env bash
set -euo pipefail
TMPDIR="$(mktemp -d)"
WAV="$TMPDIR/input.wav"
TXT="$TMPDIR/input.txt"

# Record your command
arecord -f cd -d 5 -q "$WAV"

# Transcribe (choose one)
if command -v whisper-ctranslate2 >/dev/null 2>&1; then
  whisper-ctranslate2 --model small.en --compute_type int8 --device cpu "$WAV" --output_dir "$TMPDIR" >/dev/null
elif command -v whisper >/dev/null 2>&1; then
  whisper "$WAV" --model small --language en --fp16 False --output_dir "$TMPDIR" >/dev/null
else
  echo "No Whisper CLI found. Activate your venv and install one." >&2
  exit 1
fi

TEXT="$(tr '[:upper:]' '[:lower:]' < "$TXT" | tr -d '\n')"

# Route commands
case "$TEXT" in
  *"open firefox"*)
    nohup firefox >/dev/null 2>&1 &
    say "Opening Firefox."
    ;;
  *"what time is it"*|*"time now"*)
    say "It is $(date +'%H:%M')."
    ;;
  *"system status"*)
    STATUS="$(uptime -p), $(free -h | awk '/Mem:/ {print $3" used of "$2" RAM"}')"
    say "System status: $STATUS"
    ;;
  *"volume up"*)
    pactl set-sink-volume @DEFAULT_SINK@ +5%
    say "Volume up."
    ;;
  *"mute"*)
    pactl set-sink-mute @DEFAULT_SINK@ toggle
    say "Toggled mute."
    ;;
  *)
    say "I heard: $TEXT"
    ;;
esac
rm -rf "$TMPDIR"
EOF
chmod +x ~/bin/assistant.sh

Use it:

assistant.sh

Adjust the case statements to map phrases to your own shell commands (control lights, run scripts, query services, etc.).

If you don’t have pactl, install it:

  • Debian/Ubuntu (apt): sudo apt install -y pulseaudio-utils or on newer systems with PipeWire, sudo apt install -y pipewire-pulse

  • Fedora (dnf): sudo dnf install -y pulseaudio-utils (or ensure PipeWire’s pulse layer is enabled)

  • openSUSE (zypper): sudo zypper install -y pulseaudio-utils

4) Real-world examples you can implement today

  • Meeting notes, locally: Record a meeting, then transcribe with a medium model for higher accuracy:
arecord -f cd -d 1800 -q meeting.wav
whisper-ctranslate2 --model medium --device cpu --compute_type int8 meeting.wav
  • Hands-free terminal checks: “system status,” “free disk,” “is my service running?”
systemctl is-active nginx && say "N G I N X is active" || say "N G I N X is not running"
  • Batch voicemail or podcast processing:
for f in voicemails/*.wav; do
  whisper-ctranslate2 --model small.en --device cpu "$f" --output_dir transcripts/
done
  • Accessibility on the CLI: Pipe any command output to speech:
ls -lh | sed -n '1,20p' | espeak-ng -s 170

Troubleshooting tips

  • Mic not found: Check arecord -l. If empty, ensure your user is in the audio group and that PipeWire/PulseAudio input is selected in your desktop settings.

  • Whisper errors: Ensure ffmpeg is installed and your Python venv is active. For slow CPUs, try small or base models, or whisper-ctranslate2 with --compute_type int8.

  • Piper silence: Verify the .onnx and .onnx.json files are both present, and sample with a short phrase to confirm audio output.

Conclusion and next steps

You now have a private, offline voice pipeline on Linux: record with ALSA, transcribe with Whisper, speak with Piper/eSpeak NG, and orchestrate everything with Bash. From here, you can:

  • Add a hotword listener and run assistant.sh only when triggered.

  • Integrate with Home Assistant or your favorite automation stack.

  • Swap models to balance accuracy and speed for your hardware.

Your call to action: 1) Install the prerequisites for your distro. 2) Try the STT and TTS one-liners to verify your setup. 3) Customize assistant.sh with the commands you use every day.

Voice AI doesn’t have to be a black box. On Linux, it’s yours to build, own, and improve.

Related posts