Posted on
• Artificial Intelligence

MCP Server Monitoring

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

MCP Server Monitoring: A Bash-first playbook for reliable AI toolchains

If your MCP server goes dark at 2 a.m., your entire AI-powered workflow can grind to a halt. Whether you’re exposing a local database to an LLM or wiring up automation tools, MCP (Model Context Protocol) servers quickly become critical infrastructure. The problem: many teams treat MCP services like dev toys—until uptime matters.

This guide shows you how to monitor MCP servers on Linux using Bash, systemd, and a few small utilities you probably already have. You’ll get practical health checks, auto-restarts, logging hygiene, and optional metrics—without dragging in heavyweight stacks.

What you’ll get:

  • A minimal, robust MCP health check in Bash (with webhooks and cert checks)

  • A systemd timer to run it every minute and auto-recover on failures

  • Logging and rotation patterns that won’t surprise you later

  • Optional Prometheus metrics with node_exporter’s textfile collector

Why monitor MCP servers this way?

  • MCP servers are long-running processes. As soon as an agent or toolchain depends on them, you need predictable liveness checks, restarts, and logs.

  • Bash + systemd is ubiquitous. No extra daemons, fewer moving parts, easy to audit.

  • You can layer in metrics later. Start simple (is it up? can it serve?) and expand (latency, cert expiry, scrapeable metrics) when needed.

1) Install prerequisites

We’ll use curl (HTTP checks), jq (JSON parsing), openssl (TLS sanity), and logrotate (if your MCP writes file logs).

Ubuntu/Debian:

sudo apt update
sudo apt install -y curl jq openssl logrotate

Fedora/RHEL/CentOS:

sudo dnf install -y curl jq openssl logrotate

Note: On RHEL/CentOS you may need EPEL for some optional extras later:

sudo dnf install -y epel-release

openSUSE/SLE:

sudo zypper refresh
sudo zypper install -y curl jq openssl logrotate

2) A robust Bash health check (with optional webhook and cert checks)

Save this as /usr/local/sbin/mcp-health.sh and make it executable.

#!/usr/bin/env bash
# mcp-health.sh — minimal MCP monitor with auto-restart and optional metrics/webhook
set -Eeuo pipefail

# Required (export in EnvironmentFile or environment):
: "${MCP_NAME:=mcp-example}"
: "${MCP_URL:=http://127.0.0.1:8080}"     # Base URL (no trailing slash)
: "${SERVICE_NAME:=mcp-example.service}"  # systemd unit to restart on failure

# Optional knobs:
: "${HEALTH_PATH:=/health}"               # Path returning JSON like {"status":"ok"}
: "${TIMEOUT:=4}"                         # curl timeout (seconds)
: "${RETRIES:=3}"                         # consecutive tries before deciding down
: "${SLEEP_BETWEEN:=2}"                   # seconds between retries
: "${WEBHOOK_URL:=}"                      # e.g. Slack/Discord webhook (optional)
: "${CERT_MIN_DAYS:=14}"                  # warn/restart when TLS < this many days left? (https only)
: "${METRICS_FILE:=/var/lib/mcp-monitor/metrics.prom}"  # Prometheus textfile (optional)
: "${RESTART_ON_DOWN:=true}"              # set to "false" to only alert, not restart
: "${COOLDOWN_SECS:=60}"                  # avoid restart flapping

log() {
  printf '%s %s\n' "$(date -Is)" "$*" | systemd-cat -t "mcp-health[$MCP_NAME]" || true
}

notify_webhook() {
  local msg="$1"
  [[ -z "$WEBHOOK_URL" ]] && return 0
  curl -fsS -X POST -H 'Content-Type: application/json' \
    -d "{\"text\":\"${MCP_NAME}: ${msg}\"}" "$WEBHOOK_URL" >/dev/null || true
}

write_metrics() {
  [[ -z "$METRICS_FILE" ]] && return 0
  local dir; dir="$(dirname "$METRICS_FILE")"
  mkdir -p "$dir"
  umask 022
  {
    echo "# HELP mcp_up MCP health (1=up, 0=down)."
    echo "# TYPE mcp_up gauge"
    echo "mcp_up{name=\"${MCP_NAME}\"} $1"
    if [[ -n "${2-}" ]]; then
      echo "# HELP mcp_cert_days_left Days left on TLS cert."
      echo "# TYPE mcp_cert_days_left gauge"
      echo "mcp_cert_days_left{name=\"${MCP_NAME}\"} $2"
    fi
  } >"$METRICS_FILE.tmp"
  mv "$METRICS_FILE.tmp" "$METRICS_FILE"
}

url_scheme() { sed -E 's#^([a-zA-Z0-9+.-]+)://.*#\1#' <<<"$1"; }
url_host()   { sed -E 's#^[a-zA-Z0-9+.-]+://([^/:]+).*#\1#' <<<"$1"; }
url_port()   { sed -nE 's#^[a-zA-Z0-9+.-]+://[^/:]+:([0-9]+).*#\1#p' <<<"$1"; }

check_cert_days_left() {
  local url="$1"
  local scheme host port end not_after epoch_now epoch_end days
  scheme="$(url_scheme "$url")"
  [[ "$scheme" != "https" ]] && { echo ""; return 0; }
  host="$(url_host "$url")"
  port="$(url_port "$url")"; [[ -z "$port" ]] && port=443
  end="$(echo | openssl s_client -servername "$host" -connect "$host:$port" -brief 2>/dev/null \
        | sed -n 's/^.*-----BEGIN CERTIFICATE-----/-----BEGIN CERTIFICATE-----/,$p' \
        | openssl x509 -noout -enddate 2>/dev/null || true)"
  not_after="${end#notAfter=}"
  [[ -z "$not_after" ]] && { echo ""; return 0; }
  epoch_now="$(date +%s)"
  epoch_end="$(date -d "$not_after" +%s 2>/dev/null || true)"
  [[ -z "$epoch_end" ]] && { echo ""; return 0; }
  days=$(( (epoch_end - epoch_now) / 86400 ))
  echo "$days"
}

health_ok=0
for ((i=1; i<=RETRIES; i++)); do
  if out="$(curl -fsS --max-time "$TIMEOUT" "$MCP_URL$HEALTH_PATH" 2>/dev/null)"; then
    status="$(jq -r '(.status // .state // .ok // "") | ascii_downcase' <<<"${out}" 2>/dev/null || echo "")"
    if [[ "$status" =~ ^(ok|up|healthy|true|1)$ ]]; then
      health_ok=1; break
    fi
    # Fallback: HTTP 200 without JSON "status" still counts as up
    health_ok=1; break
  fi
  sleep "$SLEEP_BETWEEN"
done

cert_days=""
cert_days="$(check_cert_days_left "$MCP_URL" || true)"

# Emit metrics early
if [[ -n "$cert_days" ]]; then
  write_metrics "$health_ok" "$cert_days"
else
  write_metrics "$health_ok"
fi

# If unhealthy, consider restart and alert (with cooldown)
state_dir="/run/mcp-health"
mkdir -p "$state_dir"
cooldown_file="$state_dir/${MCP_NAME}.cooldown"

if [[ "$health_ok" -eq 1 ]]; then
  log "OK: ${MCP_URL}${HEALTH_PATH} responded; cert_days_left=${cert_days:-na}"
  [[ -f "$cooldown_file" ]] && rm -f "$cooldown_file"
  exit 0
fi

msg="DOWN: ${MCP_URL}${HEALTH_PATH} failed after ${RETRIES}x${TIMEOUT}s checks."
log "$msg"

now="$(date +%s)"
last=0
[[ -f "$cooldown_file" ]] && last="$(<"$cooldown_file")" || true

if [[ "$RESTART_ON_DOWN" == "true" ]]; then
  if (( now - last >= COOLDOWN_SECS )); then
    log "Attempting restart: systemctl try-restart ${SERVICE_NAME}"
    systemctl try-restart --no-block "$SERVICE_NAME" || log "WARN: restart failed"
    echo "$now" > "$cooldown_file"
    notify_webhook "$msg Restarted $SERVICE_NAME."
  else
    log "Cooldown active; skipping restart."
    notify_webhook "$msg (restart in cooldown)"
  fi
else
  notify_webhook "$msg"
fi

exit 1

Make it executable:

sudo install -m 0755 /usr/local/sbin/mcp-health.sh /usr/local/sbin/mcp-health.sh

Create an environment file to set variables without editing the script:

sudo mkdir -p /etc/mcp
sudo tee /etc/mcp/mcp-example.env >/dev/null <<'EOF'
MCP_NAME="mcp-example"
MCP_URL="http://127.0.0.1:8080"
SERVICE_NAME="mcp-example.service"
HEALTH_PATH="/health"
TIMEOUT=4
RETRIES=3
SLEEP_BETWEEN=2
RESTART_ON_DOWN=true
COOLDOWN_SECS=60
WEBHOOK_URL=""   # optional: set to your Slack/Discord/Teams incoming webhook
METRICS_FILE="/var/lib/mcp-monitor/metrics.prom"
CERT_MIN_DAYS=14
EOF

3) Wire it into systemd (service + timer)

Example MCP server unit (if you don’t already have one). Replace ExecStart with your real command:

# /etc/systemd/system/mcp-example.service
[Unit]
Description=MCP server: example
After=network-online.target
Wants=network-online.target

[Service]
User=mcp
Group=mcp
EnvironmentFile=/etc/mcp/mcp-example.env
ExecStart=/usr/local/bin/mcp-example --port 8080
Restart=on-failure
RestartSec=2s
# Hardening suggestions:
NoNewPrivileges=yes
PrivateTmp=yes
ProtectSystem=full
ProtectHome=true

[Install]
WantedBy=multi-user.target

Health check as a oneshot service + timer:

# /etc/systemd/system/mcp-health@mcp-example.service
[Unit]
Description=MCP health check (%i)

[Service]
Type=oneshot
EnvironmentFile=/etc/mcp/%i.env
ExecStart=/usr/local/sbin/mcp-health.sh

# /etc/systemd/system/mcp-health@mcp-example.timer
[Unit]
Description=Run MCP health check every minute (%i)

[Timer]
OnUnitActiveSec=60s
AccuracySec=10s
Unit=mcp-health@mcp-example.service

[Install]
WantedBy=timers.target

Enable everything:

sudo systemctl daemon-reload
sudo systemctl enable --now mcp-example.service
sudo systemctl enable --now mcp-health@mcp-example.timer

Check status and logs:

systemctl status mcp-example.service
systemctl list-timers | grep mcp-health
journalctl -u mcp-health@mcp-example.service -n 50 --no-pager

4) Logging that doesn’t bite later

  • If your MCP logs to journald (default for systemd), you’re good:

    journalctl -u mcp-example.service -f

  • If your MCP writes a file, rotate it to avoid disk bloat. Example:

    # /etc/logrotate.d/mcp-example
/var/log/mcp-example/*.log {
  weekly
  rotate 8
  compress
  missingok
  notifempty
  copytruncate
}

    Test it:

    sudo logrotate -d /etc/logrotate.conf

5) Optional: Export simple Prometheus metrics

If you already run Prometheus, you can expose health via node_exporter’s textfile collector. Install node_exporter:

  • Ubuntu/Debian:

    sudo apt update
sudo apt install -y prometheus-node-exporter

  • Fedora/RHEL/CentOS (on RHEL/CentOS ensure EPEL is enabled: sudo dnf install -y epel-release):

    sudo dnf install -y node_exporter

  • openSUSE/SLE:

    sudo zypper refresh
sudo zypper install -y prometheus-node_exporter

Point the textfile collector at the directory our script writes to (/var/lib/mcp-monitor). Create an override for node_exporter:

sudo systemctl edit node_exporter

Paste:

[Service]
Environment="EXTRA_FLAGS=--collector.textfile.directory=/var/lib/mcp-monitor"
ExecStart=
ExecStart=/usr/bin/node_exporter $EXTRA_FLAGS

Then:

sudo systemctl daemon-reload
sudo systemctl restart node_exporter

You should now see metrics like:

# /var/lib/mcp-monitor/metrics.prom
mcp_up{name="mcp-example"} 1
mcp_cert_days_left{name="mcp-example"} 62

Scrape node_exporter as usual; the textfile metrics will appear under that target.

Real-world pattern you can reuse

  • Local-only MCP bound to 127.0.0.1:8080 with a JSON health endpoint at /health

  • Systemd keeps it up with Restart=on-failure

  • The timer runs the Bash health check every minute

  • If 3 checks fail, the script logs, pings a webhook, and try-restarts the unit

  • Optionally, Prometheus scrapes mcp_up and cert days left via textfile collector

This gives you actionable visibility and auto-heal behavior without a full monitoring stack.

Conclusion and next steps

MCP servers may start as experiments, but they quickly become the glue in your AI workflows. Treat them like any other microservice: monitor liveness, keep logs tidy, auto-restart on failure, and export a few essential metrics.

Your next step:

  • Drop the mcp-health.sh script and systemd units into a staging box

  • Tune the RETRIES/TIMEOUT/COOLDOWN for your workload

  • Add a webhook so you’re the first to know, not your users

  • When ready, wire in Prometheus via node_exporter’s textfile collector

Need to harden further? Consider:

  • Restricting network exposure with a firewall/reverse proxy

  • Running under a dedicated unprivileged user

  • Adding SELinux/AppArmor profiles

  • Backing up /etc/mcp/*.env and your unit files

Reliable MCP servers = reliable AI tooling. Start small, ship the basics, and iterate.

Related posts