Artificial Intelligence Bash Testing: Let an LLM Find Your Edge Cases

Ever shipped a Bash script that worked on your machine but broke in production because of a stray space, a weird URL, or an unexpected newline? Manual testing and “happy-path” checks miss the tricky stuff. What if an AI could propose adversarial inputs while your CI enforces style and correctness?

In this article you’ll:

• Set up a portable Bash testing toolchain (ShellCheck, shfmt, Bats, jq).

• Write deterministic unit tests for a real script.

• Use an AI endpoint to generate extra edge cases as executable tests.

• Wire it all into CI so your scripts stay robust.

Why this matters

• Bash glues the modern Linux stack together. A one-liner in a release pipeline can halt deployments if it meets the wrong filename or locale.

• Static analysis is great but can’t anticipate the infinite weirdness of user input.

• LLMs are surprisingly good at generating adversarial test cases—especially for surface-area heavy shell scripts.

• Combine: formatter + linter + deterministic tests + AI-generated edge cases + CI. You get fewer regressions and faster feedback.

---

1) Install the testing toolchain

We’ll use:

• shellcheck: Static analysis to catch common shell bugs.

• shfmt: Opinionated formatting to prevent style drift.

• Bats: TAP-compliant Bash tests.

• jq: Parse AI JSON output for generated tests.

• curl: Call your AI endpoint.

On Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y shellcheck shfmt bats jq curl

On Fedora/RHEL/CentOS (dnf):

sudo dnf install -y ShellCheck shfmt bats jq curl

On openSUSE (zypper):

sudo zypper refresh
sudo zypper install -y ShellCheck shfmt jq curl
# Bats package name varies by repo:
sudo zypper install -y bats || sudo zypper install -y bats-core

If Bats isn’t available in your repo, install from source:

sudo zypper install -y git || sudo dnf install -y git || sudo apt install -y git
git clone https://github.com/bats-core/bats-core.git
sudo ./bats-core/install.sh /usr/local

---

2) Baseline hygiene: format and lint on every change

Create a quick script to format and lint everything:

#!/usr/bin/env bash
set -euo pipefail

# Format in-place and keep diffs minimal.
shfmt -w -s .

# Fail on any ShellCheck finding.
shellcheck -S style -x **/*.sh

Optional: add a simple pre-commit hook to prevent sloppy diffs:

# .git/hooks/pre-commit
#!/usr/bin/env bash
set -euo pipefail
shfmt -w -s .
shellcheck -S style -x **/*.sh
git add -A

Then:

chmod +x .git/hooks/pre-commit

---

3) Write deterministic tests with Bats

Here’s a non-trivial but focused example: a CLI that prints the last path segment of a URL or path, ignoring query and fragment. Trailing slashes collapse to the previous segment.

Script: url_basename.sh

#!/usr/bin/env bash
set -euo pipefail

basename_from_url() {
  local u="${1-}"

  # Trim leading/trailing whitespace
  u="${u#"${u%%[![:space:]]*}"}"
  u="${u%"${u##*[![:space:]]}"}"

  # Strip scheme
  u="${u#*://}"

  # Remove query and fragment
  u="${u%%[\?#]*}"

  # Remove trailing slashes
  while [[ "$u" == */ ]]; do u="${u%/}"; done

  # If looks like domain/path, drop the domain part (up to first slash)
  if [[ "$u" == *"/"* ]]; then
    if [[ "$u" != /* ]]; then
      u="${u#*/}"
    fi
  fi

  # Last segment
  local base="${u##*/}"
  printf '%s\n' "$base"
}

if [[ "${BASH_SOURCE[0]}" == "$0" ]]; then
  if [[ $# -gt 0 ]]; then
    for arg in "$@"; do
      basename_from_url "$arg"
    done
  else
    while IFS= read -r line; do
      basename_from_url "$line"
    done
  fi
fi

Tests: tests/url_basename.bats

#!/usr/bin/env bats

setup() {
  chmod +x ./url_basename.sh
}

@test "simple URL" {
  run ./url_basename.sh "https://example.com/a/b/c.txt"
  [ "$status" -eq 0 ]
  [ "$output" = "c.txt" ]
}

@test "trailing slash yields dir basename" {
  run ./url_basename.sh "https://example.com/a/b/"
  [ "$status" -eq 0 ]
  [ "$output" = "b" ]
}

@test "ignores query and fragment" {
  run ./url_basename.sh "https://ex.com/a/b/file.log?x=1#top"
  [ "$status" -eq 0 ]
  [ "$output" = "file.log" ]
}

@test "stdin mode supports one-per-line" {
  run bash -c 'printf "%s\n" "a/b/c" "/tmp/z" | ./url_basename.sh'
  [ "$status" -eq 0 ]
  [ "$output" = $'c\nz' ]
}

@test "domain-only returns domain" {
  run ./url_basename.sh "https://example.org"
  [ "$status" -eq 0 ]
  [ "$output" = "example.org" ]
}

Run tests:

bats -r tests

---

4) Let an AI draft edge cases as runnable tests

You can ask an LLM to propose adversarial inputs (spaces, Unicode, empty segments, odd slashes) and convert its JSON into Bats tests.

Prerequisites:

• An OpenAI-compatible Chat Completions endpoint (or a compatible gateway).

• Environment variable OPENAI_API_KEY set.

Install jq and curl if you skipped earlier:

• apt: sudo apt install -y jq curl

• dnf: sudo dnf install -y jq curl

• zypper: sudo zypper install -y jq curl

Script: scripts/gen_ai_cases.sh

#!/usr/bin/env bash
set -euo pipefail

: "${AI_ENDPOINT:=https://api.openai.com/v1/chat/completions}"
: "${AI_MODEL:=gpt-4o-mini}"

if [[ -z "${OPENAI_API_KEY:-}" ]]; then
  echo "OPENAI_API_KEY not set. Skipping AI test generation." >&2
  exit 0
fi

mkdir -p tests

prompt=$'Given a CLI ./url_basename.sh that prints the last path segment of a URL or path, ignoring any query (?...) or fragment (#...). Trailing slashes mean take the previous segment (e.g., /a/b/ -> b). If only a domain is present (example.com), echo example.com.\nGenerate 8 concise test cases as strict JSON: [{"input":"...","expected":"..."}] with no commentary.'

payload=$(jq -n --arg model "$AI_MODEL" --arg prompt "$prompt" '{
  model: $model,
  temperature: 0,
  messages: [
    {role:"system", content:"You generate adversarial test cases for a Bash CLI. Respond ONLY with strict JSON: an array of objects with keys input and expected."},
    {role:"user", content:$prompt}
  ]
}')

resp_json=$(curl -sS -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${OPENAI_API_KEY}" \
  -d "$payload" "$AI_ENDPOINT")

# Extract content (JSON string) and ensure it is JSON
cases_json=$(printf '%s' "$resp_json" | jq -r '.choices[0].message.content' || true)

if [[ -z "$cases_json" || "$cases_json" == "null" ]]; then
  echo "AI returned no content. Raw response:" >&2
  printf '%s\n' "$resp_json" >&2
  exit 1
fi

# Validate the content parses as JSON array
if ! printf '%s' "$cases_json" | jq -e 'type=="array"' >/dev/null 2>&1; then
  echo "AI content is not an array. Content was:" >&2
  printf '%s\n' "$cases_json" >&2
  exit 1
fi

out=tests/generated.bats
{
  echo '#!/usr/bin/env bats'
  echo
  echo 'setup() { chmod +x ./url_basename.sh; }'
  idx=1
  printf '%s' "$cases_json" | jq -c '.[]' | while read -r item; do
    input=$(printf '%s' "$item" | jq -r '.input')
    expected=$(printf '%s' "$item" | jq -r '.expected')

    # Shell-quote safely
    printf -v inq "%q" "$input"
    printf -v exq "%q" "$expected"

    cat <<EOF
@test "AI case $idx" {
  run ./url_basename.sh <<< $inq
  [ "\$status" -eq 0 ]
  [ "\$output" = $exq ]
}
EOF
    idx=$((idx+1))
  done
} > "$out"

chmod +x "$out"
echo "Wrote $out"

Generate tests:

bash scripts/gen_ai_cases.sh
bats -r tests

Notes and guardrails:

• Never paste secrets or proprietary inputs into prompts.

• Review generated tests; keep what’s valuable, discard nonsense.

• Lock behavior with your hand-written tests; AI adds breadth, not truth.

---

5) Automate it in CI

Example GitHub Actions workflow (Ubuntu runner):

# .github/workflows/bash-ai-tests.yml
name: Bash + AI tests

on:
  push:
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    env:
      OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} # optional; omit to skip AI gen

    steps:
      - uses: actions/checkout@v4

      - name: Install deps
        run: |
          sudo apt-get update
          sudo apt-get install -y shellcheck shfmt bats jq curl

      - name: Lint and format
        run: |
          shfmt -d .
          shellcheck -S style -x **/*.sh

      - name: Generate AI tests (optional)
        run: |
          bash scripts/gen_ai_cases.sh || true

      - name: Run tests
        run: |
          chmod +x ./url_basename.sh
          bats -r tests

For Fedora/openSUSE CI, use containerized jobs (e.g., with a fedora or opensuse image) and run the corresponding dnf/zypper install commands.

---

Real-world tips

• Lock your interface: decide exactly what your script prints and what exit codes mean, then encode that in Bats.

• Keep tests fast: small, deterministic, filesystem-safe. Put slow or networked cases behind an opt-in flag.

• Normalize the environment in tests: set LC_ALL=C, PATH minimal, and use temporary directories.

• Make AI optional: CI should still pass without it. Treat AI output as proposals, not ground truth.

---

Conclusion and next steps

You now have a practical pattern:

• shfmt + ShellCheck keep code clean and correct.

• Bats pins behavior with deterministic tests.

• An AI endpoint proposes adversarial cases you didn’t think of.

• CI enforces all of the above on every change.

Next steps:

• Drop url_basename.sh into a repo and wire in this setup.

• Point the AI prompt at your real scripts and curate the best generated cases.

• Expand to more scripts and add coverage badges so teams see quality rise.

If you found this useful, share it with your team and start converting your one-off Bash scripts into battle-tested utilities.