Posted on
• Artificial Intelligence

Artificial Intelligence Energy Monitoring

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence Energy Monitoring on Linux with Bash: Measure, Optimize, Repeat

AI models are getting faster and larger—but they’re also getting hungrier. If you’ve ever watched your laptop fan take flight during a fine-tune or your server room bill creep up after deploying a new inference service, you already know: without energy visibility, AI is guesswork. The fix is straightforward and very Linux: measure what matters with a few CLI tools and shell scripts, then iterate.

This post shows you how to monitor and manage AI energy use on Linux with Bash. You’ll get:

  • Why AI energy monitoring is worth your time

  • A minimal, reproducible toolset you can install via apt, dnf, or zypper

  • Bash snippets to log CPU and (optionally) GPU power

  • Concrete steps to baseline, optimize, and enforce energy budgets

Why monitor AI energy?

  • Cost control: Power is a significant (and rising) fraction of TCO for GPU nodes and even CPU-heavy inference clusters.

  • Performance per watt: “Faster” models are not always more efficient. Knowing joules per sample beats eyeballing throughput.

  • Sustainability and compliance: Many orgs now track energy and carbon. Accurate, job-level metrics make reporting and tuning possible.

  • Capacity planning: Power caps, core pinning, and scheduling improve density—if you can quantify the effects.

What we’ll build

  • Quick baseline using powertop and turbostat

  • A small Bash wrapper to log CPU energy via RAPL and (optionally) GPU power via NVIDIA tools

  • Practical optimizations you can toggle and validate with hard numbers

  • A simple “energy budget guard” to fail CI if a job exceeds your target

Prerequisites and installation

The examples below use:

  • powertop (baseline and tuning hints)

  • turbostat (CPU package power and temps; Intel/AMD CPUs)

  • nvtop (optional, GPU overview; shows power if drivers expose it)

  • numactl (optional, CPU/NUMA placement)

  • gnuplot (optional, quick graphs)

Install with your package manager.

Apt (Debian/Ubuntu):

sudo apt update
sudo apt install powertop linux-tools-common nvtop numactl gnuplot
# Ensure turbostat matches your running kernel:
sudo apt install "linux-tools-$(uname -r)"

DNF (Fedora/RHEL derivatives):

sudo dnf install powertop kernel-tools nvtop numactl gnuplot

Zypper (openSUSE):

sudo zypper install powertop kernel-tools nvtop numactl gnuplot

Notes:

  • turbostat is part of linux-tools (apt) or kernel-tools (dnf/zypper).

  • GPU power telemetry requires proper vendor drivers. nvtop will display power where NVML/ROCm exposes it.

  • Reading RAPL energy counters may require root; use sudo for the scripts that touch /sys/class/powercap.

Step 1: Establish a baseline

Start with the simplest measurements to understand idle vs. load.

1) Calibrate and inspect with powertop (optional but useful on laptops/workstations):

sudo powertop --calibrate
sudo powertop

  • Use the Overview tab to see estimated power draw at idle and under light load.

  • Calibration can flicker devices; don’t run on production nodes.

2) Watch CPU package power with turbostat:

# 1-second updates, quiet summary, includes RAPL-based package power
sudo turbostat --Summary --quiet --interval 1
  • Keep this running in a side terminal while you kick off AI jobs to correlate spikes with workload phases.

3) Quick GPU sanity check (optional) with nvtop:

nvtop
  • If drivers provide telemetry, you’ll see GPU utilization, memory, and power draw.

Step 2: Log per-job energy with a Bash wrapper

For CPU energy, Linux exposes RAPL energy counters under /sys/class/powercap. We’ll sum top-level package domains and compute joules used during a command. If NVIDIA tools are present, we’ll also sample GPU power at 1 Hz in the background.

Create ai-energy-run.sh:

#!/usr/bin/env bash
set -euo pipefail

# Where to write CSV logs
LOG_FILE="${LOG_FILE:-ai_energy_log.csv}"

# Find top-level RAPL energy files (intel-rapl:0, amd-rapl:0, etc.)
rapl_top_level_files() {
  local p
  for p in /sys/class/powercap/*:*; do
    # Select only one-colon domains (e.g., intel-rapl:0) not subdomains (e.g., intel-rapl:0:0)
    [[ -d "$p" && "$p" != *:*:* ]] && [[ -f "$p/energy_uj" ]] && echo "$p/energy_uj"
  done
}

rapl_sum_uj() {
  local sum=0 val
  for f in $(rapl_top_level_files); do
    if [[ -r "$f" ]]; then
      val=$(<"$f")
      sum=$(( sum + val ))
    fi
  done
  echo "$sum"
}

need_root=false
for f in $(rapl_top_level_files); do
  [[ -r "$f" ]] || need_root=true
done
if $need_root; then
  echo "Note: RAPL energy counters require root on many systems. Re-run with sudo if you see permission errors." >&2
fi

cmd="${*:-}"
if [[ -z "$cmd" ]]; then
  echo "Usage: $0 <command to measure>" >&2
  exit 1
fi

# Optional GPU sampler if nvidia-smi is available and can read power
GPU_TMP=""
GPU_SAMPLER_PID=""
if command -v nvidia-smi >/dev/null 2>&1; then
  if nvidia-smi --query-gpu=power.draw --format=csv,noheader,nounits >/dev/null 2>&1; then
    GPU_TMP="$(mktemp)"
    # Sample total average power across all GPUs each second
    ( while :; do
        ts=$(date +%s)
        # Sum power across GPUs; ignore errors
        watts=$(nvidia-smi --query-gpu=power.draw --format=csv,noheader,nounits 2>/dev/null | awk '{s+=$1} END{print (s==0?0:s)}')
        [[ -n "$watts" ]] && echo "$ts,$watts" >> "$GPU_TMP"
        sleep 1
      done ) &
    GPU_SAMPLER_PID=$!
  fi
fi

start_ts=$(date +%s)
start_uj=$(rapl_sum_uj)
start_ns=$(date +%s%N)

# Run the target command
set +e
eval "$cmd"
exit_code=$?
set -e

end_ns=$(date +%s%N)
end_ts=$(date +%s)
end_uj=$(rapl_sum_uj)

# Cleanup GPU sampler
avg_gpu_watts=""
gpu_joules=""
if [[ -n "${GPU_SAMPLER_PID}" ]]; then
  kill "$GPU_SAMPLER_PID" >/dev/null 2>&1 || true
  wait "$GPU_SAMPLER_PID" 2>/dev/null || true
  if [[ -s "$GPU_TMP" ]]; then
    # Average watts during the run window
    avg_gpu_watts=$(awk -F, -v s="$start_ts" -v e="$end_ts" '($1>=s && $1<=e){n++; sum+=$2} END{if(n>0) printf("%.3f", sum/n)}' "$GPU_TMP")
  fi
  rm -f "$GPU_TMP"
fi

# Compute CPU package joules and wall clock
delta_uj=$(( end_uj - start_uj ))
# Handle wrap-around (rare, but possible)
if (( delta_uj < 0 )); then
  # Assume 64-bit wrap
  delta_uj=$(( (1<<63) + delta_uj ))
fi

wall_s=$(awk -v s="$start_ns" -v e="$end_ns" 'BEGIN{printf("%.6f",(e-s)/1e9)}')
cpu_joules=$(awk -v uj="$delta_uj" 'BEGIN{printf("%.6f", uj/1e6)}')

if [[ -n "$avg_gpu_watts" && -n "$wall_s" ]]; then
  gpu_joules=$(awk -v w="$avg_gpu_watts" -v t="$wall_s" 'BEGIN{printf("%.6f", w*t)}')
fi

# Write CSV header if needed
if [[ ! -f "$LOG_FILE" ]]; then
  echo "timestamp,command,exit_code,wall_s,cpu_joules,avg_gpu_watts,gpu_joules" > "$LOG_FILE"
fi

echo "$(date -Is),\"$cmd\",$exit_code,$wall_s,$cpu_joules,${avg_gpu_watts:-},${gpu_joules:-}" | tee -a "$LOG_FILE"
exit $exit_code

Make it executable:

chmod +x ai-energy-run.sh

Example usage:

# Measure a CPU inference script
sudo ./ai-energy-run.sh python inference.py --model small.onnx --batch 64

# Measure a GPU training step (NVIDIA systems)
sudo ./ai-energy-run.sh python train.py --epochs 1 --lr 3e-4

You’ll get a CSV log like:

timestamp,command,exit_code,wall_s,cpu_joules,avg_gpu_watts,gpu_joules
2026-07-11T10:25:09+00:00,"python train.py --epochs 1",0,42.318742,1450.221,128.3,5425.884

Interpretation:

  • cpu_joules: Energy used by CPU package(s) during the run (J).

  • avg_gpu_watts: Mean GPU power across all GPUs (W).

  • gpu_joules: Approx energy = avg watts × time (J). This is an approximation from 1 Hz sampling.

Step 3: Optimize and verify

Small tweaks can move the needle. Use the wrapper to validate before/after.

1) Right-size CPU threads for math libraries:

# Start with physical cores, not threads
export OMP_NUM_THREADS=$(nproc --ignore=1)
export MKL_NUM_THREADS=$OMP_NUM_THREADS
export OPENBLAS_NUM_THREADS=$OMP_NUM_THREADS
sudo ./ai-energy-run.sh python cpu_job.py
  • Try different values (e.g., half cores) and compare joules/sample.

2) Pin work to specific CPUs/NUMA nodes:

# Pin to NUMA node 0 and first 8 cores
sudo ./ai-energy-run.sh numactl --cpunodebind=0 --physcpubind=0-7 python cpu_job.py
  • Often reduces cross-node traffic and energy.

3) NVIDIA power caps and persistence mode (if supported):

# Enable persistence mode
sudo nvidia-smi -pm 1
# Set a power limit (adjust to your GPU’s allowable range)
sudo nvidia-smi -pl 180
# Re-run your job and compare energy vs. throughput
sudo ./ai-energy-run.sh python train.py --epochs 1
  • Many workloads achieve near-identical throughput at a lower cap, cutting joules.

4) System-wide power tuning (mainly laptops/workstations):

# Apply suggested tunings (verify changes fit your use case)
sudo powertop --auto-tune
  • Reboot resets changes; add to boot scripts if desired.

5) Batch size and precision:

  • Try smaller batch sizes on latency-sensitive inference to reduce peak power.

  • Use lower precision (bf16/fp16/int8) where accuracy permits; measure before/after.

Step 4: Add an “energy budget” guard

Block regressions in CI by failing when a job exceeds a joule target.

Create guard-energy.sh:

#!/usr/bin/env bash
set -euo pipefail

LOG_FILE="${1:-ai_energy_log.csv}"
MAX_JOULES="${2:-2000}"   # default 2 kJ
LAST=$(tail -n 1 "$LOG_FILE")
cpu_joules=$(echo "$LAST" | awk -F, 'NR==1{print $(NF-2)}')  # cpu_joules field
# Fallback if quoting confuses commas
if [[ -z "$cpu_joules" ]]; then
  cpu_joules=$(echo "$LAST" | awk -F, '{print $(NF-2)}')
fi

awk -v j="$cpu_joules" -v m="$MAX_JOULES" 'BEGIN{
  if (j+0 > m+0) { printf("FAIL: %.3f J > %.3f J\n", j, m); exit 1 }
  else { printf("PASS: %.3f J <= %.3f J\n", j, m); exit 0 }
}'

Use it after a run:

./guard-energy.sh ai_energy_log.csv 1500

Real-world example playbook

Try this sequence to see tangible gains: 1) Baseline:

sudo ./ai-energy-run.sh python train.py --epochs 1 --model tiny

2) Cap GPU and set threads:

export OMP_NUM_THREADS=$(nproc)
sudo nvidia-smi -pm 1
sudo nvidia-smi -pl 170
sudo ./ai-energy-run.sh python train.py --epochs 1 --model tiny

3) Pin CPUs and reduce batch size slightly:

sudo ./ai-energy-run.sh numactl --physcpubind=0-15 python train.py --epochs 1 --model tiny --batch 80

4) Compare cpu_joules and gpu_joules across runs; pick the best energy/throughput trade-off.

Optional: plot wall_s vs. total joules with gnuplot:

gnuplot -persist <<'PLOT'
set datafile separator comma
set key autotitle columnhead
set xlabel "Run"
set ylabel "Joules"
plot "<tail -n +2 ai_energy_log.csv | nl -v 1 -s, -w1" using 1:5 with linespoints title "CPU J", \
     "" using 1:7 with linespoints title "GPU J"
PLOT

Troubleshooting

  • Permission denied on energy_uj: run with sudo or adjust udev permissions for /sys/class/powercap.

  • No GPU power shown: ensure vendor drivers are installed and telemetry is exposed. nvtop and nvidia-smi rely on NVML; AMD requires ROCm/AMDGPU telemetry.

  • turbostat not found: install linux-tools (apt) or kernel-tools (dnf/zypper) and ensure versions match your kernel.

Conclusion and next steps

You don’t need heavyweight tooling to get actionable AI energy metrics. With turbostat, powertop, and a ~100-line Bash wrapper, you can:

  • Measure job-level joules on CPU and approximate GPU energy

  • Iterate on threads, placement, batch size, and power caps

  • Enforce energy budgets in CI

Your next step:

  • Install the tools, wrap your most common training/inference command with ai-energy-run.sh, and record three runs with different settings. Keep the configuration that gives the best performance per joule.

  • Then, add guard-energy.sh to your CI to prevent regressions.

Have a trick that saves watts without hurting throughput? Share it—your future self (and your power bill) will thank you.

Related posts