Posted on
• Artificial Intelligence

Using Ollama from Bash Scripts

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Using Ollama from Bash Scripts: Local AI at Your Fingertips

If you’ve ever wished your shell scripts could summarize logs, write commit messages, or explain cryptic errors without shipping data to the cloud, you’re in luck. Ollama runs large language models (LLMs) locally and exposes a simple CLI and HTTP API—perfect for Bash automation. In this guide, you’ll learn how to install Ollama, why running LLMs from Bash is so useful, and 3–5 actionable patterns to start using it in your scripts today.

Why use Ollama from Bash?

  • Privacy and compliance: Run models locally—no external API keys or outbound data.

  • Repeatability: Your scripts and prompts are version-controlled and reproducible.

  • Speed: Keep everything on-machine; no network round trips.

  • Flexibility: Combine Unix tools (grep, jq, awk, sed) with AI for powerful pipelines.

What you’ll get in this article:

  • Installation and quick-start

  • How to call Ollama from shell with curl and the CLI

  • Real-world Bash patterns for summarizing, generating, and structuring output

  • Tips for reliability, performance, and security

1) Install Ollama and essentials

Ollama provides an official Linux install script. We’ll also install curl and jq since they’re handy for scripting with the HTTP API.

Install prerequisites:

Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y curl jq

Fedora/RHEL (dnf):

sudo dnf install -y curl jq

openSUSE (zypper):

sudo zypper refresh
sudo zypper install -y curl jq

Install Ollama (official script):

curl -fsSL https://ollama.com/install.sh | sh

Start and enable the service (systemd):

sudo systemctl enable --now ollama

Verify it’s running:

systemctl status ollama --no-pager
curl -s http://localhost:11434/api/tags
ollama --version

Pull a starter model (e.g., Llama 3 8B):

ollama pull llama3:8b

Tip:

  • The Ollama server listens on 127.0.0.1:11434 by default.

  • You can override where models are stored with export OLLAMA_MODELS=/path/to/models before starting the service.

  • For GPUs, the installer will try to enable acceleration if supported.

2) Two ways to use Ollama in scripts

  • CLI one-off:
ollama run llama3:8b "Write a one-line bash joke."
  • HTTP API with curl (more control, great for automation):
curl -s http://localhost:11434/api/generate \
  -d '{
        "model": "llama3:8b",
        "prompt": "Write a one-line bash joke.",
        "stream": false
      }' | jq -r .response

The HTTP API returns JSON, which you can pipe to jq for clean extraction. Set "stream": false for a single JSON response, or "stream": true to receive a newline-delimited stream of partial tokens.

3) Actionable Bash patterns

Pattern A: Write a function that “just returns text”

Make a tiny helper for scripts so you can call the model like a normal shell function.

ollama_prompt() {
  local model="${1:-llama3:8b}"
  local prompt="$2"

  curl -fsS http://localhost:11434/api/generate \
    -H 'Content-Type: application/json' \
    -d "$(jq -n --arg m "$model" --arg p "$prompt" \
          '{model:$m, prompt:$p, stream:false}')" \
    | jq -r '.response'
}

Use it anywhere:

msg=$(ollama_prompt llama3:8b "Explain what set -euo pipefail does in bash.")
printf "%s\n" "$msg"

Add basic safeguards:

  • -f makes curl fail on HTTP errors.

  • Wrap with a timeout if needed: curl --max-time 60 ...

Pattern B: Summarize logs into action items

Turn noisy logs into a crisp brief.

last_200=$(journalctl -n 200 -o short-iso 2>/dev/null || tail -n 200 /var/log/syslog)
prompt="Summarize the following logs into 5 bullet points with timestamps and severity, focusing on recurring issues:\n\n$last_200"

ollama_prompt llama3:8b "$prompt"

Tip: For very long input, consider trimming or chunking. You can prompt the model with “part 1/3”, “part 2/3” and then “summarize all parts” for large data.

Pattern C: Auto-generate a Conventional Commit message

Let the model turn your staged diff into a commit title + body.

diff=$(git diff --staged)
prompt="Create a Conventional Commit message (feat/fix/chore/etc.) from this diff.
Return a short title (<= 72 chars) and a concise body with bullet points.\n\n$diff"

message=$(ollama_prompt llama3:8b "$prompt")
printf "%s\n" "$message"

You can even automate the commit:

git commit -m "$(printf "%s" "$message" | head -n 1)" -m "$(printf "%s" "$message" | tail -n +2)"

Pattern D: Ask for structured JSON (and parse it)

When you need machine-readable output, ask the model to return strict JSON and parse it with jq.

readme="$(sed -n '1,200p' README.md)"
payload=$(jq -n \
  --arg m "llama3:8b" \
  --arg p "Extract a JSON object with keys: {title, topics: [strings], summary}. Only output valid JSON. Text:\n\n$readme" \
  '{model:$m, prompt:$p, stream:false}')

resp=$(curl -fsS http://localhost:11434/api/generate \
  -H 'Content-Type: application/json' -d "$payload")

title=$(jq -r '.response | fromjson | .title' <<<"$resp")
topics=$(jq -r '.response | fromjson | .topics[]' <<<"$resp")
summary=$(jq -r '.response | fromjson | .summary' <<<"$resp")

printf "Title: %s\nTopics:\n" "$title"
printf " - %s\n" $topics
printf "Summary:\n%s\n" "$summary"

Notes:

  • The prompt explicitly requests “Only output valid JSON.” Many models follow this reliably, but always validate with jq.

  • For models that support strict JSON mode, you can add "format":"json" to the request to bias toward well-formed JSON.

Pattern E: Stream tokens for a TUI-like feel

Stream partial tokens for a responsive UI in the terminal.

curl -sN http://localhost:11434/api/generate \
  -H 'Content-Type: application/json' \
  -d '{
        "model":"llama3:8b",
        "prompt":"Give me a three-paragraph overview of Linux namespaces.",
        "stream": true
      }' \
| jq -r --unbuffered '.response? // empty' \
| tr -d '\n'; echo

  • -sN (silent + no-buffer) keeps output flowing.

  • Each line is a JSON chunk; jq extracts the “response” field and prints it as it arrives.

4) Tips for reliable, fast scripts

  • Model management:

    • List: ollama list
    • Remove: ollama rm MODEL[:TAG]
    • Inspect: ollama show MODEL[:TAG]
    • Pin exact variants (e.g., llama3:8b) for reproducibility.

  • Control generation:

    • You can pass parameters in the JSON body, for example:
    {
  "model": "llama3:8b",
  "prompt": "Summarize ...",
  "stream": false,
  "options": {
    "temperature": 0.2,
    "top_p": 0.9,
    "num_predict": 512
  }
}
    • Lower temperature for more deterministic output.

  • Timeouts and errors:

    • curl --max-time 60 --retry 2 --retry-connrefused
    • Fail gracefully in scripts and print helpful diagnostics.

  • Security:

    • By default, Ollama binds to localhost. If you expose it (e.g., export OLLAMA_HOST=0.0.0.0:11434), put it behind a firewall or reverse proxy with authentication.
    • Avoid logging sensitive prompts/outputs if not necessary.

  • Performance:

    • Keep prompts concise. Reuse context by including only what’s needed.
    • Prefer smaller models for quick utilities; pull larger ones only when required.

Troubleshooting

  • Service not running:

    systemctl status ollama
journalctl -u ollama -e

  • API not reachable:

    curl -v http://localhost:11434/api/tags

  • GPU not used:

    • Ensure compatible drivers/toolkit are installed; check Ollama docs for your hardware.
    • Try CPU mode first; then add GPU later.

Conclusion and next steps

With Ollama wired into your shell, AI becomes another Unix tool—greppable, composable, and scriptable. Start small: wrap a function, summarize a log, or generate a commit message. Then level up by enforcing structured JSON outputs, streaming tokens for interactivity, and pinning model versions for reproducibility.

Call to action:

  • Install Ollama and a model today.

  • Drop one of the patterns into a script you already use daily.

  • Share a gist of your best Bash+Ollama trick with your team or community.

Happy scripting!

Related posts