Posted on
• Artificial Intelligence

Prompt Engineering for DevOps

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Prompt Engineering for DevOps: Turn Bash into an AI Power Tool

If you’ve ever lost a morning tweaking Kubernetes YAML, summarizing noisy logs during an incident, or translating a vague requirement into a safe Bash script, this post is for you. Prompt engineering lets you turn large language models (LLMs) into repeatable, dependable DevOps helpers—right from your Linux shell.

This isn’t about “magic.” It’s about building small, reliable interfaces so your prompts behave like code: versioned, reproducible, and testable. You’ll get patterns, CLI snippets, and a tiny Bash wrapper you can drop into any repo.

Why prompt engineering matters for DevOps

  • DevOps work is text-heavy: manifests, logs, runbooks, release notes, policy rules, incident timelines. LLMs are naturally good at this.

  • Small wins add up: templating a safe YAML generator, summarizing 500 lines of logs, or normalizing a changelog with a strict JSON schema can save hours each week.

  • Reliability is a must: prompts need structure. Without constraints and context, you’ll get inconsistent results. Treat prompts as code and you’ll get predictable outcomes you can plug into CI, scripts, and runbooks.

Prerequisites and installation

We’ll use only standard Linux tools plus curl and jq.

  • Debian/Ubuntu (apt):
sudo apt update
sudo apt install -y curl jq
  • Fedora/RHEL/CentOS (dnf):
sudo dnf install -y curl jq
  • openSUSE/SLES (zypper):
sudo zypper install -y curl jq

Optional: Local LLMs with Ollama (no apt/dnf/zypper package from vendor; use the install script):

curl -fsSL https://ollama.com/install.sh | sh
# Then pull a model, e.g.:
ollama pull qwen2.5:7b

Cloud LLMs (e.g., OpenAI): set your API key:

export OPENAI_API_KEY="sk-...your-key..."

A tiny Bash wrapper that makes LLMs scriptable

Save this as llm.sh and source it from your shell or CI job. It supports:

  • Provider: OpenAI (default) or local Ollama

  • Low temperature for deterministic outputs

  • Optional strict JSON outputs

#!/usr/bin/env bash
# llm.sh — minimal LLM wrapper for DevOps
set -Eeuo pipefail

: "${LLM_PROVIDER:=openai}"                # openai|ollama
: "${OPENAI_MODEL:=gpt-4o-mini}"           # fast, cost-effective
: "${OPENAI_TEMPERATURE:=0.2}"             # lower = more deterministic
: "${OLLAMA_MODEL:=qwen2.5:7b}"            # local model via ollama
: "${LLM_SYSTEM:="You are a meticulous DevOps assistant. Be brief, accurate, and safe."}"

_llm_openai() {
  if [[ -z "${OPENAI_API_KEY:-}" ]]; then
    echo "ERROR: OPENAI_API_KEY not set" >&2
    exit 1
  fi
  local prompt
  prompt="$(cat)"   # read all stdin as prompt
  local response_format="${1:-}"  # pass 'json' to request JSON-only output

  # Build JSON body safely
  local body
  body="$(
    jq -n \
      --arg model "$OPENAI_MODEL" \
      --arg sys "$LLM_SYSTEM" \
      --arg msg "$prompt" \
      --arg temp "$OPENAI_TEMPERATURE" \
      --argjson rf "$( [[ "$response_format" == "json" ]] && echo '{"type":"json_object"}' || echo 'null' )" '
      {
        model: $model,
        temperature: ($temp | tonumber),
        messages: [
          {role:"system", content:$sys},
          {role:"user",   content:$msg}
        ]
      }
      + ( $rf == null ? {} : {response_format: $rf} )
    '
  )"

  curl -fsS https://api.openai.com/v1/chat/completions \
    -H "Authorization: Bearer ${OPENAI_API_KEY}" \
    -H "Content-Type: application/json" \
    -d "$body" \
  | jq -r '.choices[0].message.content'
}

_llm_ollama() {
  local prompt
  prompt="$(cat)"
  # Ollama can stream; we want final text only. The simplest approach:
  ollama run "$OLLAMA_MODEL" "$prompt"
}

# Public commands:
llm() {
  case "${LLM_PROVIDER}" in
    openai)  _llm_openai "" ;;
    ollama)  _llm_ollama ;;
    *) echo "Unknown LLM_PROVIDER: $LLM_PROVIDER" >&2; exit 1 ;;
  esac
}

llm_json() {
  case "${LLM_PROVIDER}" in
    openai)  _llm_openai "json" ;;
    ollama)  # No enforced JSON; ask the model explicitly to return only JSON.
             _llm_ollama ;;
    *) echo "Unknown LLM_PROVIDER: $LLM_PROVIDER" >&2; exit 1 ;;
  esac
}

Usage:

source ./llm.sh
echo "Give me three safe ways to rotate nginx logs on Linux" | llm

For JSON-only output with OpenAI:

cat << 'EOF' | llm_json | jq
Return only JSON with keys: strategies (array of strings), cautions (array of strings).
Provide brief, distro-agnostic steps to rotate nginx logs safely.
EOF

Tip: Check in llm.sh to your repo and pin your defaults (model, temperature, system message) for reproducibility.

Actionable patterns you can use today

1) Treat prompts as code: templates, variables, and constraints

Keep prompts in version control as .pe (prompt engineering) files. Use clear sections: Role, Context, Constraints, Output.

Example prompts/k8s_deploy.pe:

Role: You are a Kubernetes expert. Prefer simple, portable manifests.
Context:

- App: ${APP_NAME}

- Image: ${IMAGE}

- Port: ${PORT}
Constraints:

- Deployment + Service (ClusterIP).

- No latest tags; pin exact version.

- Add reasonable resource requests.
Output:

- Valid YAML only. Two documents separated by '---'.

Use it:

export APP_NAME="api"
export IMAGE="ghcr.io/example/api:1.4.2"
export PORT=8080

envsubst < prompts/k8s_deploy.pe | llm > deploy.yaml

What you get: a reproducible, reviewable YAML file with clear constraints baked into the template.

2) Make incidents calmer: log triage and summarization

Pipe recent logs into a focused prompt that asks for symptoms, suspects, and next steps.

journalctl -u myservice --since "1 hour ago" --no-pager -n 500 | \
llm << 'EOF'
Summarize the most important errors and their likely root causes.

- Group by error signatures.

- Propose 2-3 targeted next steps.

- Be specific and avoid generic advice.
EOF

For persistent issues, keep a prompt file prompts/log_triage.pe and reuse it across services.

3) Enforce structure with JSON contracts

When you need outputs for scripts or CI, require a strict JSON shape and validate with jq.

Prompt:

cat << 'EOF' | llm_json | tee risk.json >/dev/null
You are a change-risk assessor for infra changes.
Return only JSON with keys:

- risk_level: one of "low","medium","high"

- reasons: array of short strings

- safety_checks: array of shell commands (strings) the on-call can run to verify safety
Analyze the following diff:
<<<DIFF
[Paste or pipe your diff/plan here]
DIFF
EOF

Consume in Bash:

level="$(jq -r '.risk_level' risk.json)"
if [[ "$level" == "high" ]]; then
  echo "High risk change. Escalate for manual review." >&2
  exit 2
fi

This pattern turns fuzzy text into a predictable interface for automation.

4) Generate Bash safely and gate it

Ask for minimal, portable Bash with guardrails, then review before running.

cat << 'EOF' | llm
Write a POSIX-compliant Bash script that:

- Backs up /etc/nginx to /var/backups/nginx-YYYYmmdd.tar.gz

- Verifies the archive after creation

- Prints a one-line success message and exits 0 on success

- Fails safely on any error

Only output the script content. No explanations.
EOF

Always review generated scripts. For extra caution, run a dry run or test in a container. If you use OpenAI, set temperature low (0.0–0.2) for more repeatable code.

5) Compose context packs: examples + policies + acceptance criteria

Few-shot prompting improves reliability. Keep small, high-quality examples near your prompt and reference them inline.

cat examples/ingress_good.yaml examples/ingress_bad.yaml prompts/ingress_rules.pe | llm > ingress.yaml

In your prompt, clearly state acceptance criteria (e.g., “No wildcard hosts; enforce TLS; return only valid YAML”). Over time, curate and refine examples like you would unit tests.

Hardening tips for production use

  • Keep secrets out of prompts. Redact tokens and IDs in logs before sending to a cloud API. Prefer local models (Ollama) for sensitive data.

  • Pin models and settings. Small changes in temperature or model version can affect outcomes.

  • Save inputs and outputs. Log prompts and results to a build artifact for auditability.

  • Fail fast on structure. If llm_json output doesn’t parse or violates your schema, stop the pipeline.

  • Right-size the task. Use LLMs for text-heavy, pattern-based work; keep deterministic logic in code.

Quick start: 5-minute setup

1) Install dependencies:

  • apt:
sudo apt update && sudo apt install -y curl jq
  • dnf:
sudo dnf install -y curl jq
  • zypper:
sudo zypper install -y curl jq

2) Save llm.sh from above and source it:

source ./llm.sh

3) Choose your provider:

  • Cloud (OpenAI):
export OPENAI_API_KEY="sk-..."; export LLM_PROVIDER=openai
  • Local (Ollama):
curl -fsSL https://ollama.com/install.sh | sh
ollama pull qwen2.5:7b
export LLM_PROVIDER=ollama

4) Run an example:

echo "Create a one-paragraph rollout plan for a zero-downtime nginx upgrade" | llm

5) Turn a prompt into code:

mkdir -p prompts
cat > prompts/log_triage.pe <<'EOF'
Summarize key errors, likely root causes, and top 3 next steps.
Group similar stack traces. Be concise and specific.
EOF

journalctl -u sshd --since "30 min ago" --no-pager | cat - prompts/log_triage.pe | llm

Conclusion and next steps

Prompt engineering for DevOps is about discipline: small wrappers, stable templates, strict outputs, and versioned context. With a 30-line Bash helper, you can turn LLMs into reliable building blocks for your day-to-day operations.

Call to action:

  • Drop llm.sh into your dotfiles or repo.

  • Create a prompts/ directory and commit your first two templates (e.g., log triage and YAML generator).

  • Pick one pipeline step this week to structure via llm_json and validate with jq.

Small, safe steps will compound into faster, calmer ops.

Related posts