Posted on
• Artificial Intelligence

Building Artificial Intelligence Assistants with Ollama

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Building Artificial Intelligence Assistants with Ollama (on Linux, with Bash)

Tired of copy-pasting secrets into web UIs or burning API credits just to get help on a shell one-liner? You can run fast, modern LLMs locally with Ollama, pipe data in and out with Bash, and build private, scriptable assistants that fit right into your Linux workflow.

In this guide, you’ll install Ollama the right way for your distro, learn why local AI is worth it, and ship a working CLI assistant you can customize and call from any script.

Why Ollama for Linux users?

  • Privacy and control: Your data stays on your machine. No tokens. No third-party logs.

  • Works like Unix: It’s a single binary + REST API. Streams input/output. Plays well with Bash.

  • Models on demand: Pull curated models (e.g., Llama 3, Mistral) with simple commands.

  • Repeatable and customizable: Lock in prompts and defaults with Modelfiles so you can version-control assistants just like code.

Install Ollama on Linux

Recommended: use the official install script. If your environment disallows curl | sh, see the native package notes below.

First, ensure tools:

  • Ubuntu/Debian (apt)
sudo apt update
sudo apt install -y curl ca-certificates jq
  • Fedora/RHEL (dnf)
sudo dnf install -y curl ca-certificates jq
  • openSUSE (zypper)
sudo zypper refresh
sudo zypper install -y curl ca-certificates jq

Now install Ollama:

curl -fsSL https://ollama.com/install.sh | sh

Start (and enable) the service:

sudo systemctl enable --now ollama

Pull a starter model and test:

ollama pull llama3
echo 'Write a Bash script that says hello' | ollama run llama3

Notes:

  • The service listens on localhost:11434.

  • The script will auto-detect your platform and set up the binary and service.

  • GPU support is automatic when compatible drivers/runtimes are installed.

If you prefer native packages:

  • Debian/Ubuntu: Download the latest .deb from the Ollama website, then sudo dpkg -i <file>.deb && sudo apt -f install -y, followed by sudo systemctl enable --now ollama.

  • Fedora/openSUSE/RHEL: Download the .rpm, then sudo rpm -Uvh <file>.rpm (or sudo dnf install <file>.rpm), followed by sudo systemctl enable --now ollama.

5 actionable steps to a working local assistant

1) Talk to a model from the shell

The quickest loop is stdin/stdout. Use this anywhere you can pipe text.

# Ask ad hoc questions
echo 'Summarize the last 100 lines of /var/log/syslog' \
  | tail -n 100 /var/log/syslog | ollama run llama3

# Explain a command
echo 'Explain: find . -type f -mtime -1 -print0 | xargs -0 tar -czf recent.tgz' \
  | ollama run llama3

Prefer the HTTP API? It’s on localhost:11434.

curl -s http://localhost:11434/api/generate -H 'Content-Type: application/json' -d '{
  "model":"llama3",
  "prompt":"Write a Bash script that prints prime numbers up to 50."
}' | jq -r '.response'

2) Create a reusable CLI chat assistant (Bash)

This one-file script preserves chat history and talks to /api/chat.

Save as ~/bin/aichat and chmod +x ~/bin/aichat:

#!/usr/bin/env bash
set -euo pipefail

MODEL="${AI_MODEL:-llama3}"
STATE="${AI_STATE:-$HOME/.cache/ollama-chat.json}"
mkdir -p "$(dirname "$STATE")"

if [[ "${1:-}" == "-n" ]]; then : > "$STATE"; shift; fi

INPUT="$*"
if [ -z "$INPUT" ] && [ -t 0 ]; then
  echo "Usage: $(basename "$0") [-n] your question" >&2
  exit 1
fi
if [ -z "$INPUT" ]; then
  INPUT="$(cat)"
fi

if [ ! -s "$STATE" ]; then
  printf '[]' > "$STATE"
fi

PREV="$(cat "$STATE")"
MESSAGES="$(jq -c --arg u "$INPUT" '. + [{"role":"user","content":$u}]' <<<"$PREV")"

RESP="$(jq -nc --arg m "$MODEL" --argjson msgs "$MESSAGES" \
  '{model:$m, messages:$msgs, stream:false}' \
  | curl -sfS http://localhost:11434/api/chat \
      -H 'Content-Type: application/json' -d @-)"

ASSISTANT="$(echo "$RESP" | jq -r '.message.content')"
echo "$ASSISTANT"

NEWSTATE="$(jq -c --arg a "$ASSISTANT" '. + [{"role":"assistant","content":$a}]' <<<"$MESSAGES")"
echo "$NEWSTATE" > "$STATE"

Examples:

  • Start a fresh chat: aichat -n "Help me write a safe rm wrapper"

  • Pipe context in: git diff | aichat "Summarize changes and suggest a commit message"

Tip: Set a default model via export AI_MODEL=mistral.

3) Package a persona with a Modelfile

Lock in behavior, tone, and defaults so your assistant is consistent and versionable.

Create Modelfile:

FROM llama3
SYSTEM """
You are ShellMate: a concise Linux shell assistant.

- Prefer POSIX sh unless Bash is needed.

- Explain briefly, then show a one-file example.

- When risky, add a safety note.
"""
PARAMETER temperature 0.2

Build and run:

ollama create shellmate -f Modelfile
ollama run shellmate

Use it in your script:

AI_MODEL=shellmate aichat -n "Create a script that backs up ~/work to /mnt/backups with rotation"

4) Wire AI into everyday Unix tasks

  • Summarize logs:
journalctl -u ssh.service -n 300 | aichat "Find suspicious patterns and suggest hardening steps"
  • Turn raw output into reports:
df -h | aichat "Identify partitions near capacity and propose cleanup commands"
  • Explain code quickly:
bat -n src/*.sh | aichat "Explain each function and potential edge cases"
  • Generate docs:
grep -R --line-number TODO src | aichat "Group TODOs, estimate effort, and outline next steps"

5) Manage models and performance

  • Model lifecycle:
ollama list
ollama pull mistral
ollama rm llama3
  • Run the server manually (if needed):
ollama serve   # foreground server on :11434
  • Speed tips:
    • Choose smaller models for quick tasks; larger for reasoning-heavy work.
    • Keep prompts focused. Provide the minimum context necessary.
    • Cache context: reuse aichat histories per project so the model stays “in the loop.”

Real-world example: A log triage helper

  • Collect context:
LOG=$(journalctl -u myapp --since "1 hour ago")
  • Ask for triage:
printf '%s\n\nTask: Identify root causes, rank by severity, and list concrete fixes.' "$LOG" \
  | AI_MODEL=shellmate aichat
  • Iterate: paste the proposed command back into the assistant for validation before you run it.

Troubleshooting quick hits

  • Service not running? systemctl status ollama and check logs via journalctl -u ollama.

  • Port in use? Adjust with OLLAMA_HOST=127.0.0.1:11435 ollama serve and point your client there.

  • Missing JSON tools? Install jq with your package manager (see install section).

Conclusion and next steps

You now have:

  • A private, local AI stack that respects your data.

  • A reusable Bash assistant you can call from any script.

  • A way to package personas with Modelfiles for repeatable behavior.

Call to action: 1) Install Ollama, pull a model, and run the aichat script. 2) Create a Modelfile for your team’s style and publish it in your repo. 3) Pick one daily task (logs, diffs, docs) and wire AI into it today.

Your terminal just became an AI workstation—no tokens, no waiting, no leaks.

Related posts