Building AI Assistants from the Linux Shell

What if your terminal could answer questions about your codebase, draft commands, summarize logs, and even propose scripts—without leaving Bash? That’s the promise of AI assistants you control from the command line. In this guide, we’ll build a minimal, extensible AI assistant using Bash, curl, and jq. You’ll learn how to connect to a cloud API or a local model, add project-aware context, and handle shell command execution safely.

Why it matters:

• Terminal-first workflows are fast and scriptable.

• Bash + curl + jq lets you integrate AI into anything you can pipe.

• You stay in control: choose your model provider (cloud or local), keep your logs, and make security-conscious choices.

What we’ll cover:

• Installing the essentials (apt, dnf, zypper included)

• A minimal chat assistant in Bash (OpenAI-compatible APIs)

• Running locally with Ollama (no external network needed)

• Adding file/project context to improve answers

• Safely executing suggested commands

Note: All code snippets are Bash and intended for Linux.

Prerequisites

We’ll use these tools:

• curl: to talk to APIs

• jq: to handle JSON

• ripgrep (rg): to fetch relevant snippets from your project (optional but useful)

• Python 3 + pip: optional for future extensions

Install them with your package manager:

• Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y curl jq ripgrep python3 python3-pip

• Fedora/RHEL/CentOS (dnf):

sudo dnf install -y curl jq ripgrep python3 python3-pip

• openSUSE/SLE (zypper):

sudo zypper refresh
sudo zypper install -y curl jq ripgrep python3 python3-pip

Keep your API keys safe:

• If using a cloud provider, export your key before running scripts:

export OPENAI_API_KEY="sk-..."

• Optional: set a custom API base or model via env vars:

export API_BASE="https://api.openai.com/v1"
export MODEL="gpt-4o-mini"

Step 1 — A minimal CLI chat assistant (OpenAI-compatible)

This is a single-file assistant that:

• Maintains a conversation log (JSON)

• Sends messages to a Chat Completions endpoint

• Prints the response

Save as assistant.sh and make it executable.

#!/usr/bin/env bash
set -euo pipefail

# Minimal Bash AI assistant using an OpenAI-compatible Chat Completions API.
# Dependencies: bash, curl, jq
# Env:
#   OPENAI_API_KEY (required for cloud APIs)
#   API_BASE (default: https://api.openai.com/v1)
#   MODEL (default: gpt-4o-mini)

API_BASE="${API_BASE:-https://api.openai.com/v1}"
MODEL="${MODEL:-gpt-4o-mini}"
HISTORY="${HISTORY:-./history.json}"
SYSTEM_PROMPT="${SYSTEM_PROMPT:-You are a concise Linux terminal assistant. Prefer Bash and POSIX tools in answers.}"

usage() {
  echo "Usage: $0 \"your question...\""
  echo "Env: OPENAI_API_KEY (required), API_BASE, MODEL, HISTORY, SYSTEM_PROMPT"
  exit 1
}

if [[ "${1:-}" == "-h" || "${1:-}" == "--help" ]]; then usage; fi
if [[ $# -lt 1 ]]; then usage; fi
if [[ -z "${OPENAI_API_KEY:-}" ]]; then
  echo "Error: OPENAI_API_KEY not set" >&2
  exit 1
fi

init_history() {
  if [[ ! -f "$HISTORY" ]]; then
    printf '[]' > "$HISTORY"
  fi
  # Ensure a system prompt exists as the first message
  if ! jq -e 'map(select(.role=="system")) | length > 0' "$HISTORY" >/dev/null; then
    tmp=$(mktemp)
    jq --arg c "$SYSTEM_PROMPT" '. + [{"role":"system","content":$c}]' "$HISTORY" > "$tmp"
    mv "$tmp" "$HISTORY"
  fi
}

append_message() {
  local role="$1"; shift
  local content="$*"
  local tmp=$(mktemp)
  jq --arg r "$role" --arg c "$content" '. + [{"role":$r,"content":$c}]' "$HISTORY" > "$tmp"
  mv "$tmp" "$HISTORY"
}

ask() {
  local user_content="$*"
  append_message "user" "$user_content"

  local payload
  payload=$(jq -n \
    --arg model "$MODEL" \
    --argjson messages "$(cat "$HISTORY")" '
      {model:$model, messages:$messages, temperature:0.2, stream:false}
    ')

  local resp
  resp=$(curl -fsS "$API_BASE/chat/completions" \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer '"$OPENAI_API_KEY"'" \
    -d "$payload")

  local reply
  reply=$(echo "$resp" | jq -r '.choices[0].message.content // empty')

  if [[ -z "$reply" ]]; then
    echo "API error:" >&2
    echo "$resp" | jq -C . >&2 || echo "$resp" >&2
    exit 1
  fi

  append_message "assistant" "$reply"
  echo "$reply"
}

init_history
ask "$*"

Try it:

./assistant.sh "Give me a one-liner to count unique IPs in an nginx log."

Tips:

• To reset context: rm history.json

• To switch models: export MODEL="gpt-4o-mini" (or your provider’s model)

• To target another OpenAI-compatible service: export API_BASE="https://your-endpoint/v1"

Security note: Keep OPENAI_API_KEY out of your shell history. Consider using a .env file and direnv or pass/age.

Step 2 — Run locally with Ollama (no external network)

Prefer running models on your machine? Ollama serves local LLMs via a simple HTTP API.

Install Ollama:

curl -fsSL https://ollama.com/install.sh | sh

Start the service (if not already active):

# For systemd user service (typical after install)
systemctl --user start ollama
systemctl --user status ollama

Pull a model:

ollama pull llama3.1:8b

Minimal Ollama chat script (no cloud key required). Save as ollama_assistant.sh:

#!/usr/bin/env bash
set -euo pipefail

# Local assistant using Ollama's chat API
# Dependencies: curl, jq, ollama running (http://localhost:11434)
MODEL="${MODEL:-llama3.1:8b}"
HISTORY="${HISTORY:-./ollama_history.json}"
SYSTEM_PROMPT="${SYSTEM_PROMPT:-You are a concise Linux terminal assistant. Prefer Bash and POSIX tools.}"

usage() {
  echo "Usage: $0 \"your question...\""
  exit 1
}
if [[ $# -lt 1 ]]; then usage; fi

init_history() {
  [[ -f "$HISTORY" ]] || printf '[]' > "$HISTORY"
  if ! jq -e 'map(select(.role=="system")) | length > 0' "$HISTORY" >/dev/null; then
    tmp=$(mktemp)
    jq --arg c "$SYSTEM_PROMPT" '. + [{"role":"system","content":$c}]' "$HISTORY" > "$tmp"
    mv "$tmp" "$HISTORY"
  fi
}
append_message() {
  local role="$1"; shift
  local content="$*"
  tmp=$(mktemp)
  jq --arg r "$role" --arg c "$content" '. + [{"role":$r,"content":$c}]' "$HISTORY" > "$tmp"
  mv "$tmp" "$HISTORY"
}

ask() {
  local user_content="$*"
  append_message "user" "$user_content"

  payload=$(jq -n --arg m "$MODEL" --argjson msgs "$(cat "$HISTORY")" \
    '{model:$m, messages:$msgs, stream:false}')

  resp=$(curl -fsS http://localhost:11434/api/chat \
    -H "Content-Type: application/json" -d "$payload")

  reply=$(echo "$resp" | jq -r '.message.content // empty')
  [[ -n "$reply" ]] || { echo "Ollama error"; echo "$resp"; exit 1; }

  append_message "assistant" "$reply"
  echo "$reply"
}

init_history
ask "$*"

Test it:

./ollama_assistant.sh "Summarize what a POSIX-compliant shell script header should include."

Step 3 — Make it project-aware with ripgrep

Large models do better with relevant context. We can cheaply approximate retrieval by grabbing matching lines from your repo and injecting them into the prompt.

Add this helper to create-context.sh:

#!/usr/bin/env bash
set -euo pipefail

# Usage: ./create-context.sh "search terms" [path=.]
QUERY="${1:-}"
ROOT="${2:-.}"
[[ -n "$QUERY" ]] || { echo "Usage: $0 \"search terms\" [path]"; exit 1; }

# Tune ignores to your project (avoid large dirs)
IGNORE_ARGS=(-g '!node_modules' -g '!.git' -g '!dist' -g '!build' -g '!.venv')
rg -n -S -C 2 "${IGNORE_ARGS[@]}" --heading --color=never -- "${QUERY}" "$ROOT" | \
  awk 'NR<=2000' # limit total lines

Use the context with your assistant:

CTX=$(./create-context.sh "nginx log parser" ./)
./assistant.sh "Use this project context to answer: 
===CONTEXT START===
$CTX
===CONTEXT END===
Question: Where does the parser handle malformed lines, and how can we improve it?"

Why this helps:

• Keeps token usage smaller than dumping entire files

• Makes answers specific to your code

• You can replace this later with embeddings/vector search

Step 4 — Safely execute suggested commands

Let the assistant propose commands, but never auto-run them. Here’s a pattern that extracts fenced bash blocks and asks for confirmation.

Save as run_suggestions.sh:

#!/usr/bin/env bash
set -euo pipefail

# Pipe assistant output to this script to selectively run code blocks.
# It looks for ```bash ... ``` or ```sh ... ``` blocks.

content="$(cat)"
blocks=$(printf "%s" "$content" | awk '/^```(bash|sh)/{flag=1;next}/^```/{flag=0}flag')

if [[ -z "$blocks" ]]; then
  echo "No bash/sh fenced code blocks found."
  exit 0
fi

echo "Proposed commands:"
echo "------------------"
echo "$blocks"
echo "------------------"
read -r -p "Execute in current shell? [y/N] " ans
if [[ "$ans" =~ ^[Yy]$ ]]; then
  bash -euo pipefail -c "$blocks"
else
  echo "Aborted."
fi

Use it:

./assistant.sh "Write a one-liner to list top 10 IPs by request count in access.log" | tee reply.txt
cat reply.txt | ./run_suggestions.sh

Security guidelines:

• Require explicit approval before execution

• Run in a restricted environment if possible (e.g., a container or limited user)

• Log everything; never pipe secrets into the model

Real-world examples

• Summarize logs:

tail -n 2000 /var/log/nginx/access.log | \
  ./assistant.sh "Summarize error patterns and peak traffic windows from this log chunk."

• Explain a file:

./assistant.sh "Explain this Dockerfile and suggest security hardening steps:
$(sed -n '1,200p' Dockerfile)"

• Generate a quick script (with confirmation):

./assistant.sh "Write a bash script to monitor disk usage and alert if any mount exceeds 85%." | ./run_suggestions.sh

• Draft a commit message from staged changes:

git diff --staged | ./assistant.sh "Write a concise conventional commit message for these changes."

Troubleshooting

• Missing jq or rg errors

  • Install packages:
  • apt: sudo apt install -y jq ripgrep
  • dnf: sudo dnf install -y jq ripgrep
  • zypper: sudo zypper install -y jq ripgrep

• API errors

  • Check OPENAI_API_KEY is exported
  • Confirm API_BASE and MODEL are valid
  • Print raw responses with: | jq -C . for readability

• Ollama not responding

  • Ensure service is running: systemctl --user status ollama
  • Pull a model: ollama pull llama3.1:8b

Conclusion and Next Steps

You’ve built a terminal-native AI assistant that can:

• Chat via cloud or local models

• Use your project context to give better answers

• Propose commands you can review and run safely

Where to go from here:

• Add a TUI with fzf or gum for history and multi-turn chats

• Cache context chunks and summaries to cut token usage

• Add “tools” (e.g., let the assistant call curl or sqlite) with explicit approval gates

• Swap in embeddings-based retrieval (e.g., a small Python helper with sentence-transformers)

Call to action:

• Wire this into your daily workflow. Start by aliasing the assistant:

alias ai="$PWD/assistant.sh"

• Try it on your current repo with Step 3’s project-aware context.

• Share your improvements and hardening patterns with your team.

Build it small. Keep it safe. Make it yours.