Installing Artificial Intelligence CLI Tools on Linux

If you love Bash, the terminal is where you think. Good news: modern AI belongs there too. With a few CLI tools, you can summarize logs, generate scripts, refactor code, or run powerful local models—without leaving your shell. This guide shows you exactly how to install and start using AI from the command line on Linux.

We’ll cover:

• Why AI in the terminal is worth it

• Preparing your Linux system

• Installing a cloud-backed AI CLI

• Running local LLMs offline

• Coding with an AI pair-programmer

• Real-world examples you can copy today

Why run AI from the terminal?

• Speed and focus: No context switching—pipe, redirect, filter, repeat.

• Composability: Combine AI with grep, awk, jq, curl, and friends.

• Automation: Drop AI into cron jobs, CI pipelines, or glue scripts.

• Privacy and cost control: Optionally run local models with no network.

• Portability: Works identically on servers, containers, and WSL.

---

1) Prepare your Linux box (once)

Install core tools you’ll use across AI CLIs: Python, pipx (to isolate tools), build utilities, and jq.

Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y curl git python3 python3-pip python3-venv pipx build-essential cmake jq
pipx ensurepath

Fedora (dnf):

sudo dnf groupinstall -y "Development Tools"
sudo dnf install -y curl git python3 python3-pip python3-virtualenv pipx cmake jq
pipx ensurepath

openSUSE (zypper):

sudo zypper refresh
sudo zypper install -y curl git python3 python3-pip python3-venv pipx gcc gcc-c++ make cmake jq
pipx ensurepath

Tip:

• If pipx isn’t available in your repo, bootstrap it:

python3 -m pip install --user pipx
python3 -m pipx ensurepath

• Open a new shell so your PATH picks up pipx.

---

2) A universal terminal interface to AI: llm (cloud and local)

Simon Willison’s llm is a compact CLI that talks to multiple providers and local backends via plugins. It’s a perfect “do-everything” front-end that works well with pipes.

Install:

pipx install llm

Add your API key (example: OpenAI):

llm keys set openai

Try it:

echo "Write a bash one-liner that shows the 5 largest files in the current dir." | llm -m gpt-4o-mini

• Pipe in text, get results out. No ceremony.

• Swap models with -m as needed.

• Store output reproducibly with tee:

journalctl -u ssh | tail -n 1000 | llm -m gpt-4o-mini > ssh-log-summary.txt

Optional: Add the Ollama plugin to talk to local models (see next section):

llm install llm-ollama
llm -m ollama:llama3 "Give me 3 robust bash tips for error handling."

---

3) Run local LLMs with ollama (offline, fast startup)

Ollama downloads and runs models locally with a simple CLI and a background service. Great for privacy, prototyping, and scripting without API costs.

Install (official script covers Debian/Ubuntu, Fedora, openSUSE, etc.):

curl -fsSL https://ollama.com/install.sh | sh
sudo systemctl enable --now ollama

Run a model:

ollama run llama3

• The first run downloads the model, then drops you into an interactive prompt.

• Exit with Ctrl+C. Subsequent runs are instant.

Use it non-interactively in scripts:

printf 'List three ways to harden an SSH server on Linux.' | \
  ollama run llama3

Or hit the local HTTP API (great for Bash + jq):

curl -s http://localhost:11434/api/generate \
  -d '{"model":"llama3","prompt":"Explain set -euo pipefail in bash, briefly."}' | jq -r '.response'

Use with the llm CLI:

pipx inject llm llm-ollama
echo "Generate a safe rm wrapper that moves files to a trash folder." | \
  llm -m ollama:llama3

Notes:

• For NVIDIA GPUs, install the proprietary driver and CUDA as per your distro for acceleration.

• Models: try llama3, mistral, or codellama for coding.

---

4) An AI pair-programmer in your terminal: aider

Aider is a CLI code assistant that integrates with Git and edits your files intelligently. Use it with cloud providers or local models via Ollama.

Install:

pipx install aider-chat

Use with a cloud model (example: OpenAI):

export OPENAI_API_KEY=your_key_here
aider --model gpt-4o-mini

Use with a local model via Ollama:

aider --model ollama/llama3

Typical workflow:

• Start aider in your repo:

cd your-project
git init
aider --model ollama/llama3

• Paste an error or goal; aider proposes diffs; accept or refine; commit.

• Great for refactors, docstrings, tests, and migration helpers.

---

Real-world examples you can copy

• Summarize auth failures from logs:

sudo journalctl -u ssh -S yesterday | \
  awk '/Failed password/ {print $0}' | \
  llm -m gpt-4o-mini -s "Summarize patterns and suggest 3 mitigations."

• Generate a Bash data-cleaning script:

cat sample.csv | llm -m gpt-4o-mini -s "Write a bash script that removes rows with empty 3rd column and outputs to cleaned.csv"

• Explain a gnarly one-liner:

echo "for f in **/*.log; do grep -Hc ERROR \"$f\"; done | sort -t: -k2,2nr" | \
  llm -m ollama:llama3 -s "Explain what this does, step by step."

• Turn requirements into a Makefile target:

cat TODO.md | llm -m gpt-4o-mini -s "Draft a Makefile with targets that implement these tasks. Use portable shell."

Pro tips:

• Save model choice in env:

export LLM_MODEL=ollama:llama3
alias ask='llm -m ${LLM_MODEL:-gpt-4o-mini}'

• Make outputs reproducible:

ask "Summarize release notes at https://example.com/changelog" | tee SUMMARY.txt

---

Troubleshooting and sanity checks

• pipx not on PATH? Run:

pipx ensurepath
exec $SHELL

• Ollama service:

systemctl --user status ollama || sudo systemctl status ollama

• Permissions/firewall when hitting local API? Try:

curl -v http://localhost:11434/api/tags

---

Conclusion and next steps

The terminal is the perfect place for AI: scriptable, composable, and fast. Here’s your two-step plan: 1) Install llm and Ollama:

pipx install llm
llm install llm-ollama
curl -fsSL https://ollama.com/install.sh | sh && sudo systemctl enable --now ollama

2) Pick one daily task—summarizing logs, generating scripts, or refactoring code—and automate it with:

echo "Your prompt here" | llm -m ollama:llama3

Want more? Wire these tools into cron jobs, dotfiles, and CI. Your shell already speaks pipes—now it speaks AI.