Bash Scripts That Diagnose Linux Problems Automatically

Ever been paged at 3 a.m. with “the server is slow” and nothing else? The fastest way to resolve incidents is to collect the right facts—immediately, consistently, and without guesswork. Bash can do that for you, on any distro, with minimal dependencies. In this post, you’ll get plug‑and‑play Bash scripts that automatically diagnose common Linux problems, plus guidance on scheduling and packaging the results.

Why automate diagnosis with Bash?

• It’s everywhere. Bash ships with virtually every Linux system (bare metal, VM, container).

• It reduces mean time to resolution (MTTR). You get a repeatable snapshot of the system, every time, so you can compare “good vs. bad” states.

• It preserves volatile evidence. Kernel and service logs can roll over quickly; network state changes second-by-second. Capture them while they’re still hot.

• It creates a runbook you can share. Your team can collaborate around proven scripts and stop solving the same mystery twice.

Prerequisites (optional but recommended tools)

The scripts below gracefully degrade if a tool isn’t installed. For best results, install these:

• sysstat (iostat, mpstat, pidstat) for I/O and CPU stats

• smartmontools (smartctl) for disk health

• lsof for open files and listening ports

• traceroute for path debugging

• DNS utilities (dig) for name resolution checks

• ethtool for NIC/link details

Install with your package manager:

• Debian/Ubuntu (apt):

  • sudo apt update
  • sudo apt install -y sysstat smartmontools lsof traceroute dnsutils ethtool

• RHEL/CentOS/Fedora (dnf):

  • sudo dnf install -y sysstat smartmontools lsof traceroute bind-utils ethtool

• openSUSE/SLES (zypper):

  • sudo zypper install -y sysstat smartmontools lsof traceroute bind-utils ethtool

Note: Some actions (e.g., reading SMART data, firewall rules) require root. Prefix with sudo or run as root.

---

1) One‑shot system snapshot: collect the facts fast

Drop this script on any host to capture system, service, storage, and network state into a single timestamped tarball.

#!/usr/bin/env bash
# diag-snapshot.sh — collect a structured snapshot for triage
set -Eeuo pipefail
umask 077

TS="$(date +%F-%H%M%S)"
HOST="$(hostname -s 2>/dev/null || echo host)"
BASE_OUT="${OUTDIR:-/var/log/diag}"
OUTDIR="${BASE_OUT}/${HOST}-${TS}"
mkdir -p "$OUTDIR"

log() { echo "[$(date +%T)] $*" | tee -a "$OUTDIR/diag.log"; }
have() { command -v "$1" >/dev/null 2>&1; }
save() { local name="$1"; shift; log "Collect: $name"; { "$@" ; } > "$OUTDIR/$name" 2>&1 || true; }

log "Starting snapshot on $HOST at $TS"

# OS / kernel / uptime / users
save os-release       cat /etc/os-release
save uname            uname -a
save uptime           uptime
save who              who -a

# CPU / memory / processes
save cpuinfo          cat /proc/cpuinfo
save meminfo          cat /proc/meminfo
save top              bash -c 'top -b -n1 | head -n 40'
save ps               bash -c 'ps aux --sort=-%cpu | head -n 30; echo; ps aux --sort=-%mem | head -n 30'
have vmstat    && save vmstat    bash -c 'vmstat 1 5'
have mpstat    && save mpstat    bash -c 'mpstat -P ALL 1 3'
have pidstat   && save pidstat   bash -c 'pidstat 1 3'

# Filesystems / disks
save df-hT           df -hT
save df-inodes       df -i
have lsblk    && save lsblk      lsblk -f
save mounts          mount
have iostat   && save iostat-x   bash -c 'iostat -xz 1 3'

# Disk health (SMART) — requires smartmontools and root
if have smartctl; then
  while read -r dev type; do
    [[ "$type" == "disk" ]] || continue
    d="/dev/$dev"
    save "smart-$(basename "$d")" smartctl -H -A "$d"
  done < <(lsblk -ndo NAME,TYPE)
fi

# Network
have ip       && save ip-a        ip -br a
have ip       && save ip-route    ip r
have ss       && save sockets     bash -c 'ss -tulpn; echo; ss -s'
have ethtool  && save ethtool     bash -c 'for i in /sys/class/net/*; do n=$(basename "$i"); ethtool "$n" || true; done'

# Basic connectivity checks
GW="$(ip route 2>/dev/null | awk "/default/ {print \$3; exit}")" || true
[[ -n "${GW:-}" ]] && save ping-gw   ping -c 4 -W 2 "$GW"
save ping-8.8.8.8  ping -c 4 -W 2 8.8.8.8

# Services / logs (systemd)
if have systemctl; then
  save systemd-failed    systemctl --failed
  save services-running  systemctl list-units --type=service --state=running
fi
if have journalctl; then
  save journal-errors    journalctl -p 3 -xb --no-pager
fi
have dmesg && save dmesg-warn  bash -c 'dmesg -T --level=err,warn || dmesg | tail -n 300'

# Open ports and files
have lsof && save lsof-listen lsof -nP -iTCP -sTCP:LISTEN

# Package versions (quick)
if have rpm; then save packages-rpm rpm -qa --last; fi
if have dpkg; then save packages-deb dpkg -l; fi
if have zypper; then save packages-rpm-zypper zypper se -i; fi

# Bundle results
log "Bundling results"
TARBALL="${OUTDIR}.tgz"
tar -C "$(dirname "$OUTDIR")" -czf "$TARBALL" "$(basename "$OUTDIR")"
log "Done: $TARBALL"
echo "$TARBALL"

How to use:

• Save as diag-snapshot.sh, chmod +x diag-snapshot.sh.

• Run: sudo ./diag-snapshot.sh

• It prints the path to a .tgz you can attach to tickets or share internally.

What you’ll capture: hardware/OS basics, CPU/memory/process top talkers, filesystems and I/O, disk SMART health, network state (addresses/routes/sockets), connectivity to gateway and the internet, failed services, and recent high‑priority logs.

---

2) Network triage in one go

When “the network is slow,” you need to validate DNS, routing, path MTU, and local firewalls. This script does a lightweight, safe check.

#!/usr/bin/env bash
# net-check.sh — quick network triage for a host or IP
set -Eeuo pipefail

TARGET="${1:-8.8.8.8}"
RESOLVER="${RESOLVER:-}"

have() { command -v "$1" >/dev/null 2>&1; }
section() { echo; echo "=== $* ==="; }

section "Target"
echo "Target: $TARGET"

section "Interfaces and routes"
ip -br a || true
ip r || true

section "DNS resolution"
echo "/etc/resolv.conf:"
cat /etc/resolv.conf || true
if have resolvectl; then resolvectl status || true; fi
if have dig; then
  dig +short "$TARGET" || true
  [[ -n "$RESOLVER" ]] && dig @"$RESOLVER" "$TARGET" +short || true
else
  getent ahosts "$TARGET" || true
fi

section "Connectivity (ICMP and PMTU)"
ping -c 4 -W 2 "$TARGET" || true
for s in 1472 1460 1400 1200; do
  echo "PMTU probe size=$s"
  ping -c 1 -W 2 -M do -s "$s" "$TARGET" || true
done

section "Path trace"
if have traceroute; then
  traceroute -n "$TARGET" || true
else
  echo "traceroute not installed"
fi

section "Listening/active sockets"
ss -tuna || true

section "Firewall status (best-effort)"
if command -v firewall-cmd >/dev/null 2>&1; then firewall-cmd --state || true; fi
if command -v ufw >/dev/null 2>&1; then ufw status || true; fi
if command -v iptables >/dev/null 2>&1; then iptables -S || true; fi

section "Links"
if command -v ethtool >/dev/null 2>&1; then
  for i in /sys/class/net/*; do n=$(basename "$i"); echo "--- $n ---"; ethtool "$n" || true; done
fi

Install missing tools if needed:

• apt: sudo apt install -y traceroute dnsutils ethtool

• dnf: sudo dnf install -y traceroute bind-utils ethtool

• zypper: sudo zypper install -y traceroute bind-utils ethtool

Run example:

• ./net-check.sh example.com

• RESOLVER=1.1.1.1 ./net-check.sh example.com

---

3) Service doctor: focus on one systemd unit

Most incidents boil down to “this service misbehaved.” This script pulls the essentials for a single unit: state, recent errors, PID, open ports, and coredumps.

#!/usr/bin/env bash
# service-doctor.sh UNIT — inspect a systemd service
set -Eeuo pipefail
UNIT="${1:-}"
[[ -z "$UNIT" ]] && { echo "Usage: $0 <unit.service>"; exit 1; }

have() { command -v "$1" >/dev/null 2>&1; }
section() { echo; echo "=== $* ==="; }

section "Unit status"
systemctl status --no-pager "$UNIT" || true

section "Restart policy"
systemctl show -p Restart,RestartSec,StartLimitIntervalUSec,StartLimitBurst "$UNIT" || true

section "Recent warnings/errors (last 2h)"
if have journalctl; then
  journalctl -u "$UNIT" --since "2 hours ago" -p warning --no-pager || true
fi

section "Main process and binary"
MAINPID=$(systemctl show -p MainPID "$UNIT" | cut -d= -f2 || true)
echo "MainPID: ${MAINPID:-0}"
if [[ -n "${MAINPID:-}" && "$MAINPID" -gt 0 ]]; then
  readlink -f "/proc/$MAINPID/exe" || true
  echo; echo "Open listening sockets (lsof):"
  if have lsof; then lsof -nP -p "$MAINPID" | grep -E 'LISTEN|ESTABLISHED' || true; fi
  echo; echo "Environment:"
  tr '\0' '\n' < "/proc/$MAINPID/environ" 2>/dev/null | sed 's/^[^=]\+=.*/&/g' || true
fi

section "Coredumps (if any)"
if have coredumpctl; then
  coredumpctl list "$UNIT" || true
fi

section "Dependencies"
systemctl list-dependencies "$UNIT" || true

echo
echo "Tip: To restart (if appropriate), run: sudo systemctl restart $UNIT"

Install helpers if needed:

• apt: sudo apt install -y lsof

• dnf: sudo dnf install -y lsof

• zypper: sudo zypper install -y lsof

Run example:

• ./service-doctor.sh nginx.service

---

4) Disk and filesystem triage

Out-of-space or I/O latency incidents are common. This script finds hot spots quickly.

#!/usr/bin/env bash
# disk-doctor.sh [PATH] — find space/inode pressure and disk errors
set -Eeuo pipefail
TARGET="${1:-/}"

have() { command -v "$1" >/dev/null 2>&1; }
section() { echo; echo "=== $* ==="; }

section "Filesystem usage"
df -hT
df -i

section "Largest directories (top 20) under $TARGET"
du -xhd1 "$TARGET" 2>/dev/null | sort -h | tail -n 20

section "I/O stats (if available)"
if have iostat; then iostat -xz 1 3; else echo "iostat not installed (sysstat)"; fi

section "Kernel disk errors"
dmesg -T | egrep -i 'i/o error|ext[234]-fs error|buffer i/o|resetting link|device offlined' || true

section "SMART health (if available; needs root)"
if have smartctl; then
  while read -r dev type; do
    [[ "$type" == "disk" ]] || continue
    echo "--- /dev/$dev ---"
    sudo smartctl -H -A "/dev/$dev" || true
  done < <(lsblk -ndo NAME,TYPE)
else
  echo "smartctl not installed (smartmontools)"
fi

section "Mount options"
mount | cat

Install helpers if needed:

• apt: sudo apt install -y sysstat smartmontools

• dnf: sudo dnf install -y sysstat smartmontools

• zypper: sudo zypper install -y sysstat smartmontools

Run example:

• ./disk-doctor.sh /var

---

Real‑world tip: capture intermittent issues automatically

If an issue occurs sporadically, schedule snapshots so you don’t miss it. With systemd timers:

Create a unit to run the snapshot:

# /etc/systemd/system/diag-snapshot.service
[Unit]
Description=Collect diagnostic snapshot

[Service]
Type=oneshot
ExecStart=/usr/local/bin/diag-snapshot.sh

Create a timer (every 15 minutes, jittered):

# /etc/systemd/system/diag-snapshot.timer
[Unit]
Description=Run diagnostic snapshot periodically

[Timer]
OnBootSec=5m
OnUnitActiveSec=15m
RandomizedDelaySec=2m
Persistent=true

[Install]
WantedBy=timers.target

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable --now diag-snapshot.timer
systemctl list-timers | grep diag-snapshot

This will populate /var/log/diag/*.tgz with time‑stamped bundles you can compare across incidents.

---

Security and hygiene

• Snapshots may contain hostnames, usernames, IPs, and service details. Treat them as internal artifacts and restrict access (the scripts use umask 077).

• Consider redacting secrets from environment variables or logs before sharing externally.

• Clean up old bundles periodically (e.g., logrotate or a cron/systemd tmpfiles policy).

---

Conclusion and next steps

Bash remains a powerful, universal tool for incident triage. With these scripts you can:

• Capture a consistent, high‑signal snapshot under pressure

• Focus quickly on network, service, or disk trouble

• Automate evidence collection for flaky, intermittent problems

Your next step:

• Install the recommended tools for your distro (apt/dnf/zypper commands above).

• Drop these scripts into /usr/local/bin, make them executable, and test on a non‑production host.

• Wire up a systemd timer for periodic snapshots on critical systems.

If you found this useful, consider building a small internal repo with your team’s additions (container/runtime checks, GPU stats, Kubernetes node health) and standardize your on‑call runbook around it.