AI Database Capacity Planning on Linux: A Bash-First Playbook

Your vector database is adding millions of embeddings every week. Latency is creeping up, backups take all night, and a new model doubles embedding dimensions without warning. Sound familiar? Capacity planning for AI databases is not optional—it’s the difference between smooth scaling and 3 a.m. incidents.

This guide gives you a practical, Bash-first approach to sizing, measuring, and validating capacity for AI-flavored databases (vector stores, hybrid OLTP + ANN search, etc.). You’ll learn what to measure, how to model growth, what to test, and how to translate all of this into disk, memory, and IOPS you actually need.

Why AI databases are different (and why you should plan now)

• Data shape and growth are non-linear:

  • Embeddings add large, fixed-width vectors (e.g., 768–4096 dims) that scale directly with user/content volume.
  • Model and schema changes (float32 → float16, PQ quantization, new metadata) can swing footprint by 2–5×.

• Performance is tail-sensitive:

  • ANN indexes and range filters can cause heavy, random reads. SLOs care about p95/p99 latency, not just averages.

• Write amplification is real:

  • Ingest + index builds + WAL/logging + compaction can multiply your actual write bandwidth needs.

• Backups, replication, and retention multiply cost:

  • 1 TB of vectors with RF=3 plus snapshots and backups can become 4–6 TB quickly.

The good news: simple Linux tools and a few shell functions can make your planning scientific, repeatable, and fast.

---

Quick-start: install the essential tooling

We’ll use standard Linux tooling to baseline performance and run quick synthetic tests.

Packages: sysstat (iostat, sar), iotop, fio, nvme-cli, smartmontools, zstd, bc

# Debian/Ubuntu
sudo apt update && sudo apt install -y sysstat iotop fio nvme-cli smartmontools zstd bc

# RHEL/CentOS/Fedora
sudo dnf install -y sysstat iotop fio nvme-cli smartmontools zstd bc

# openSUSE/SLES
sudo zypper refresh && sudo zypper install -y sysstat iotop fio nvme-cli smartmontools zstd bc

Enable sysstat (for sar/iostat persistence):

# Debian/Ubuntu
sudo sed -i 's/^ENABLED=.*/ENABLED="true"/' /etc/default/sysstat
sudo systemctl enable --now sysstat

# RHEL/Fedora/openSUSE
sudo systemctl enable --now sysstat

---

Step 1: Define SLOs and translate workload → capacity in Bash

Before touching disks, define:

• Target QPS, p95/p99 latency, ingestion rate (rows/s), and retention window

• Embedding dimension (D), data type (float32=4 bytes, float16=2 bytes), replication factor (RF)

• Index/metadata overhead multiplier (1.2–2.0× typical)

Use these shell helpers to estimate size and write bandwidth:

# Estimated stored size (TB) for embeddings + index/metadata + replication
# N = number of vectors, D = dimension
vec_size_tb() {
  local N="${1:?N_vectors}"
  local D="${2:?dimension}"
  local bytes="${3:-4}"           # 4=float32, 2=float16
  local rf="${4:-2}"              # replication factor
  local overhead="${5:-1.5}"      # index/metadata multiplier
  local meta_bytes="${6:-0}"      # extra metadata bytes
  echo "scale=3; ($N*$D*$bytes*$rf*$overhead + $meta_bytes) / (1024^4)" | bc -l
}

# Estimated sustained ingest bandwidth (MiB/s)
# rows_per_sec = vectors/sec, overhead ~1.2–1.6 for WAL/index write amp
est_ingest_mibps() {
  local rows_per_sec="${1:?rows_per_sec}"
  local D="${2:?dimension}"
  local bytes="${3:-4}"
  local overhead="${4:-1.3}"
  echo "scale=2; ($rows_per_sec*$D*$bytes*$overhead) / (1024*1024)" | bc -l
}

# Example: 100M vectors, 768-dim, float32, RF=3, 1.6× overhead
vec_size_tb 100000000 768 4 3 1.6

# Example: 5k rows/s ingest, 768-dim, float32, 1.3× write amplification
est_ingest_mibps 5000 768 4 1.3

Worked example:

• 100,000,000 vectors × 768 dims × 4 bytes ≈ 307.2 GB raw

• With 1.6× index/metadata ≈ 491.5 GB

• With RF=3 ≈ 1.47 TB on disk

• Add 20–30% slack for growth + 1 full backup on disk, plan ≈ 2–2.5 TB minimum

Rule of thumb:

• Keep 30–50% headroom vs. steady-state needs.

• Size for p99, not average.

---

Step 2: Baseline the system you have

Collect a 10-minute snapshot during typical and peak periods.

# Disks and I/O latency (await), queue depth (avgqu-sz), util (%util)
iostat -xz 1 600

# CPU, run queue, context switches
vmstat 1 600

# Top I/O offenders (requires root)
sudo iotop -oPa -d 1 -n 60

# Historical I/O (after enabling sysstat)
sar -d 1 60

Quick checks:

• If iostat shows %util > 70% with await > 5–10 ms on SSDs during queries, you’re I/O-bound.

• If vmstat shows run queue consistently > CPU cores, you’re CPU-saturated or waiting on I/O.

• Identify the heaviest PIDs with iotop (e.g., compaction, index build, VACUUM-like tasks).

Inspect disks and mounts:

lsblk -o NAME,ROTA,SIZE,TYPE,MOUNTPOINT
findmnt -no TARGET,OPTIONS /
df -hT

What to look for:

• ROTA=0 indicates SSD/NVMe (good for ANN/random read).

• Disable atime on DB volumes (noatime).

• Ensure WAL/transaction logs are on their own fast device when possible.

---

Step 3: Storage layout, media choice, and health

Recommendations:

• Separate concerns: logs/WAL vs. data/indexes vs. backups/snapshots.

• Prefer NVMe for ANN-heavy workloads; use RAID10 when redundancy is needed.

• XFS is a solid default for database volumes.

• Monitor drive health and temperature.

Health checks:

# NVMe inventory and SMART
sudo nvme list
sudo nvme smart-log /dev/nvme0n1

# SATA/SAS SMART
sudo smartctl -a /dev/sda

If you see rising media/errors, high temperature, or throttling, budget for replacements before peak season.

---

Step 4: Validate with synthetic I/O before production says “no”

Use fio to simulate two common profiles:

• Log/WAL: sustained sequential writes, 16–128 KiB blocks

• ANN reads: random 4–32 KiB reads at depth

Create and run a WAL-like write test:

mkdir -p /mnt/db/wal && fallocate -l 2G /mnt/db/wal/test.log >/dev/null 2>&1 || true

cat > log-write.fio <<'EOF'
[log_writes]
ioengine=libaio
direct=1
rw=write
bs=16k
iodepth=32
numjobs=1
size=2G
filename=/mnt/db/wal/test.log
time_based=1
runtime=60
group_reporting=1
EOF

fio log-write.fio

Create and run an ANN-like random read test:

mkdir -p /mnt/db/index && fallocate -l 50G /mnt/db/index/test.idx

cat > ann-read.fio <<'EOF'
[ann_reads]
ioengine=libaio
direct=1
rw=randread
bs=4k
iodepth=64
numjobs=4
size=50G
filename=/mnt/db/index/test.idx
time_based=1
runtime=60
group_reporting=1
EOF

fio ann-read.fio

What to check:

• For WAL: can you sustain your estimated ingest bandwidth (MiB/s) with <5 ms latency?

• For ANN: do p95/p99 clat values meet your SLO when mixed with background writes?

• If not, you likely need faster media, better separation, more devices (shards), or reduced amplification (e.g., batching, compaction tuning).

---

Step 5: Backups, retention, and cost control

• Budget for: primary data × replication + snapshots + backups.

• Test restore time; SLOs mean nothing if your RTO/RPO aren’t feasible.

• Compress where possible (zstd is fast and effective).

Example backup with zstd:

sudo tar --use-compress-program="zstd -19 -T0" -cf /backups/db-$(date +%F).tar.zst /var/lib/yourdb

Ideas to cut footprint:

• Move from float32 to float16 embeddings if recall allows.

• Use product quantization or HNSW/IVF parameters that trade tiny recall for large savings.

• Tier cold partitions to cheaper SSDs; keep hot shards on the fastest NVMe.

---

Real-world mini-scenario

A team planned for 1 TB after loading 100M vectors (768-dim, float32). Their initial math ignored replication, index overhead, and backups:

• Raw: ~307 GB

• With 1.6× index: ~492 GB

• With RF=3: ~1.47 TB

• With 30% headroom + 1 full local backup: ~2.4 TB

After fio tests, their single SSD could not sustain ingest + ANN reads at p99. They moved WAL to a dedicated NVMe, kept indexes on a separate NVMe RAID1, switched to float16 for older partitions, and hit their SLO with 40% headroom.

---

Common pitfalls checklist

• Underestimating index/metadata overhead (assume at least 1.2–2.0×).

• Forgetting replication and backup storage in totals.

• Not separating WAL/logs from random-read-heavy indexes.

• Benchmarks that focus on averages, not p95/p99.

• Skipping restore tests (backups you can’t restore quickly don’t count).

---

Conclusion and next steps (CTA)

Capacity planning for AI databases doesn’t require guesswork. You can: 1) Define SLOs and compute first-order needs with the Bash functions above. 2) Baseline your current stack using sysstat and iotop. 3) Validate disk choices and layout with fio before committing hardware. 4) Include replication, snapshots, and backups in your math. 5) Re-run the plan with each major model or schema change.

Your next step: install the tooling, run a 10-minute baseline during peak, and estimate your 6–12 month footprint with vec_size_tb. If your tests show risk to p99 latency or ingest bandwidth, address storage layout (separate WAL/data), media (NVMe), or overhead (datatype/quantization) now—before the next scale jump.

If you want a sanity check, paste your vec_size_tb and fio results into your team doc and schedule a 30-minute capacity review this week. Your future self will thank you.