Posted on
• Artificial Intelligence

Artificial Intelligence Pull Request Reviews

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

AI-Powered Pull Request Reviews from Your Bash Terminal

If your team’s pull requests pile up faster than they get reviewed, you’re not alone. PR backlogs slow shipping, frustrate contributors, and let sneaky bugs slip through. What if you could get fast, consistent, policy-aware feedback on every PR—right from Bash—without replacing human reviewers?

This post shows how to wire up a simple, auditable AI review pipeline in shell. You’ll fetch PR context with the GitHub CLI, send a targeted prompt to a model (cloud or local), parse the result, and plug it into your workflow. You get faster triage, fewer nitpicks, and more time for meaningful human feedback.

Why AI PR reviews are worth it

  • Speed and coverage: AI highlights obvious issues (security footguns, style drift, missing tests) so humans can focus on architecture and intent.

  • Consistency: Encodes your team’s policies and style guides so every PR gets the same baseline checks.

  • Developer ergonomics: Instant feedback from the terminal or CI keeps contributors moving.

  • Works with what you have: Bash + curl + jq + gh = low-friction, reviewable, version-controlled automation.

What you’ll build

  • A Bash script that:

    • Gathers PR diff and metadata with gh and jq
    • Prompts an AI model to analyze changes and return structured findings
    • Prints a clean, actionable summary or posts it back to the PR

  • Integration options:

    • Local runs while you iterate
    • CI runs on every PR

Prerequisites and installation

We’ll use these tools:

  • git: repo operations

  • gh: GitHub CLI for PR context

  • curl: HTTP requests

  • jq: JSON processing

Install them with your distro’s package manager.

Apt (Debian/Ubuntu and derivatives):

sudo apt update
sudo apt install -y git curl jq
# Install GitHub CLI (official repo)
sudo mkdir -p -m 755 /etc/apt/keyrings
curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/etc/apt/keyrings/githubcli-archive-keyring.gpg
sudo chmod go+r /etc/apt/keyrings/githubcli-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" \
  | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null
sudo apt update
sudo apt install -y gh

Dnf (Fedora, RHEL 9+, CentOS Stream):

sudo dnf install -y git curl jq gh

Zypper (openSUSE Leap/Tumbleweed):

sudo zypper refresh
sudo zypper install -y git curl jq gh

Authenticate gh:

gh auth login
# or non-interactive:
# echo "$GH_TOKEN" | gh auth login --with-token

If you’ll use a cloud model, set your API key (example: OpenAI):

export OPENAI_API_KEY="sk-...your-token..."

Optional: local model with Ollama:

curl -fsSL https://ollama.com/install.sh | sh
# Example model:
ollama pull llama3.1

Step 1: Capture PR context with gh and jq

You want only the relevant bits: changed files, diff hunks, PR title/description.

#!/usr/bin/env bash
set -euo pipefail

: "${PR_NUMBER:?Usage: PR_NUMBER env var required}"
: "${REPO:?Usage: REPO=owner/name required}"

# Fetch a concise diff, skipping noisy files
FILTER='( .filename | test("\\.(md|txt|png|jpg|lock|min\\.js)$") | not )'
TMPDIR="$(mktemp -d)"
trap 'rm -rf "$TMPDIR"' EXIT

# PR metadata
gh api repos/:owner/:repo/pulls/"$PR_NUMBER" \
  --repo "$REPO" > "$TMPDIR/pr.json"

# Files and patches
gh api repos/:owner/:repo/pulls/"$PR_NUMBER"/files \
  --repo "$REPO" > "$TMPDIR/files.json"

# Build a trimmed review payload
jq --argjson files "$(jq '[ .[] | {filename, status, patch} ]' < "$TMPDIR/files.json")" '
  {
    number: .number,
    title: .title,
    author: .user.login,
    branch: .head.ref,
    base: .base.ref,
    body: .body,
    files: $files
  }' < "$TMPDIR/pr.json" \
  | jq '.files |= map(select(.patch != null)) | .files |= map(select('"$FILTER"'))' \
  > "$TMPDIR/payload.json"

echo "Prepared PR payload at $TMPDIR/payload.json"
jq '.files | length as $n | "Files to review: \($n)"' "$TMPDIR/payload.json" -r

Run it:

PR_NUMBER=123 REPO=owner/repo ./gather.sh

Step 2: Write a strong prompt (policy + structure)

Good prompts make good reviews. Encode your style rules, security policies, and desired output shape.

Create review_prompt.txt:

You are a senior code reviewer. Task:

- Review ONLY the provided diff patches; avoid speculation beyond context.

- Focus on: security pitfalls, correctness, test coverage, performance hotspots, readability, and policy/style compliance.

- Be concise and actionable.

Return strict JSON with keys:
{
  "severity": "none|low|medium|high|critical",
  "summary": "one-paragraph overview",
  "findings": [
    {
      "title": "short title",
      "severity": "low|medium|high|critical",
      "file": "path",
      "line": number | null,
      "category": "security|bug|style|test|perf|docs|other",
      "details": "what and why",
      "suggestion": "concrete fix or snippet"
    }
  ],
  "metrics": {
    "files_inspected": number,
    "lines_changed": number
  }
}

Policies:

- No secrets/hardcoded creds. Flag .env, API keys, tokens.

- Shell: quote variables; use set -euo pipefail; check return codes.

- Python: prefer pathlib; avoid bare except; add tests for new logic.

- Go: check errors; avoid panics; ensure context usage in I/O.

- JS/TS: sanitize inputs; avoid eval; consistent linting.

If nothing significant, set "severity":"none" and provide 0 findings.

Step 3: Call the model from Bash

Option A: Cloud model (OpenAI-compatible Chat Completions). Save as ai-pr-review.sh:

#!/usr/bin/env bash
set -euo pipefail

: "${PR_NUMBER:?}"
: "${REPO:?}"
: "${OPENAI_API_KEY:?Set OPENAI_API_KEY}"

WORK="$(mktemp -d)"
trap 'rm -rf "$WORK"' EXIT

# 1) Build context (reuse Step 1 inline for brevity)
gh api repos/:owner/:repo/pulls/"$PR_NUMBER" --repo "$REPO" > "$WORK/pr.json"
gh api repos/:owner/:repo/pulls/"$PR_NUMBER"/files --repo "$REPO" > "$WORK/files.json"

jq --argjson files "$(jq '[ .[] | {filename, status, patch} ]' < "$WORK/files.json")" '
  {
    number: .number, title: .title, author: .user.login,
    branch: .head.ref, base: .base.ref, body: .body, files: $files
  }' < "$WORK/pr.json" \
  | jq '.files |= map(select(.patch != null))' > "$WORK/payload.json"

# 2) Build prompt
PROMPT="$(cat review_prompt.txt)"
CONTENT="$(jq -c '.' < "$WORK/payload.json")"

# 3) Call model
MODEL="${MODEL:-gpt-4o-mini}" # adjust as needed
RESPONSE="$(curl -sS https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer ${OPENAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d @- <<EOF
{
  "model": "${MODEL}",
  "temperature": 0,
  "response_format": { "type": "json_object" },
  "messages": [
    { "role": "system", "content": ${PROMPT@Q} },
    { "role": "user", "content": ${CONTENT@Q} }
  ]
}
EOF
)"

# 4) Extract JSON result
RESULT="$(echo "$RESPONSE" | jq -r '.choices[0].message.content')"

# Validate JSON
echo "$RESULT" | jq . > "$WORK/review.json"

# 5) Pretty print summary
echo "=== AI Review Summary ==="
jq -r '.summary' "$WORK/review.json"
echo
echo "=== Findings ==="
jq -r '.findings[]? | "- [\(.severity)] \(.title) (\(.file):\(.line//"n/a"))\n  Category: \(.category)\n  Details: \(.details)\n  Suggestion: \(.suggestion)\n"' "$WORK/review.json"

# Optional: post comment to PR
if [[ "${POST_TO_PR:-0}" == "1" ]]; then
  {
    echo "AI Review:"
    echo
    echo "Severity: $(jq -r '.severity' "$WORK/review.json")"
    echo
    echo "Summary:"
    jq -r '.summary' "$WORK/review.json"
    echo
    echo "Findings:"
    jq -r '.findings[]? | "* [\(.severity)] \(.title) — \(.file):\(.line//"n/a")\n  - \(.details)\n  - Suggestion: \(.suggestion)"' "$WORK/review.json"
  } > "$WORK/comment.md"
  gh pr comment "$PR_NUMBER" --repo "$REPO" --body-file "$WORK/comment.md"
fi

Run it:

PR_NUMBER=123 REPO=owner/repo ./ai-pr-review.sh
# or post to PR:
POST_TO_PR=1 PR_NUMBER=123 REPO=owner/repo ./ai-pr-review.sh

Option B: Local model via Ollama (text response). Quick demo:

PROMPT="$(cat review_prompt.txt)"
CONTEXT="$(cat "$TMPDIR/payload.json")"
printf "%s\n\n%s\n" "$PROMPT" "$CONTEXT" | ollama run llama3.1

Tip: Ask the model to return strict JSON (as in the prompt) and pipe into jq to validate.

Step 4: Put reviewers in CI and local workflows

  • GitHub Actions (runs on every PR):
name: AI PR Review
on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  review:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
      contents: read
    steps:
      - uses: actions/checkout@v4
      - name: Install deps
        run: |
          sudo apt-get update
          sudo apt-get install -y jq curl
      - name: AI Review
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: |
          PR_NUMBER=${{ github.event.pull_request.number }} \
          REPO=${{ github.repository }} \
          POST_TO_PR=1 \
          bash ./ai-pr-review.sh
  • Pre-commit or pre-push (fast local feedback):
# .git/hooks/pre-push
#!/usr/bin/env bash
set -euo pipefail
branch="$(git rev-parse --abbrev-ref HEAD)"
if [[ "$branch" =~ ^(main|master)$ ]]; then
  exit 0
fi
# Only run if diff is not huge
changes="$(git diff --staged | wc -l)"
if (( changes < 2000 )); then
  echo "Running AI quick review on staged changes..."
  # Use a lightweight local model or short cloud call
  # Adapt gather logic to use staged diff instead of PR
fi

Step 5: Make it safe, fast, and useful

  • Privacy and compliance:

    • Don’t send sensitive code to external APIs unless permitted. Prefer local models or enterprise endpoints.
    • Redact secrets before sending. Example:
    sed -E 's/(api[_-]?key|token|passwd|password)\s*=\s*[^[:space:]]+/REDACTED=/gi'

  • Scope the context:

    • Filter out generated/vendor/lock files; cap diff size or chunk by file.
    • Keep the prompt short and specific to reduce cost and latency.

  • Deterministic, parseable output:

    • Ask for strict JSON and validate with jq. On failure, retry once with a “return valid JSON” instruction.

  • Cost and rate limits:

    • Use small, capable models for triage; escalate to bigger models only if severity >= medium.

  • Keep humans in the loop:

    • Post comments as suggestions, not blockers.
    • Auto-apply labels based on severity:
    severity="$(jq -r '.severity' review.json)"
case "$severity" in
  high|critical) gh pr edit "$PR_NUMBER" --add-label "ai:needs-attention" --repo "$REPO" ;;
  none|low) gh pr edit "$PR_NUMBER" --add-label "ai:clean" --repo "$REPO" ;;
esac

Real-world wins

  • Security catch: AI flagged a newly added .env with a probable token. The team rotated credentials and updated .gitignore.

  • Reliability: Found unchecked command failures in a Bash deploy script; adding set -euo pipefail and exit-code checks prevented silent partial deploys.

  • Tests: Identified a new code path without coverage; author added a focused unit test, catching a boundary bug.

Conclusion and next steps

AI PR reviews won’t replace your best reviewers—but they will unblock them. Start small: run the script locally on a couple of PRs, tune the prompt to your style, then wire it into CI. Measure time-to-first-feedback and defect rates; iterate on filters and policies.

Next steps:

  • Drop the scripts into your repo and try a live PR.

  • Add team-specific rules to review_prompt.txt.

  • Enable CI posting and labeling for triage.

  • If your code is sensitive, start with a local model and compare results.

Have questions or want a hardened version for your stack? Try the scripts above, then open an issue or share your experience. Happy reviewing—right from Bash!

Related posts