Posted on
• Artificial Intelligence

Artificial Intelligence NAS Automation

Author
  • User
    linuxbash
    Posts by this author
    Posts by this author

Artificial Intelligence NAS Automation: Smarter Bash Workflows for Your Home Lab

If your NAS keeps growing faster than your free evenings, you’re not alone. Sorting, tagging, deduplicating, and monitoring terabytes of files by hand is a grind. The good news: you can bolt AI onto your existing Linux + Bash toolkit to automate the repetitive parts—without giving up control or going full vendor lock-in.

This guide shows you how to add practical AI to your NAS using simple, composable shell scripts and CLI tools. You’ll get real-time file labeling, smarter deduplication, predictive drive health alerts, and a safe “ChatOps” wrapper to turn natural language into auditable shell commands.

What you’ll need:

  • Comfort editing Bash scripts and systemd units

  • A Linux NAS or server with Docker/Podman available (local is fine)

  • Willingness to test on a scratch dataset before touching production

Why AI for NAS Ops?

  • Scale: Manual file triage and organization doesn’t scale past a few hundred gigs. AI helps classify and describe new files on arrival.

  • Precision: Traditional deduplication is binary (hash or no hash). AI-powered similarity (like perceptual hashing) can spot near-dupes.

  • Proactivity: SMART data looks like alphabet soup. Basic anomaly detection can warn you about drive drift before failure.

  • Safety and speed: Natural-language interfaces are great, but only if they’re constrained. A safe wrapper gives you speed without foot-guns.

Prerequisites and installation

We’ll use common CLI tools and run a local LLM via container for privacy and offline use. Install the following:

Packages:

  • jq (JSON parsing)

  • inotify-tools (file system events)

  • smartmontools (drive health)

  • fdupes (exact dedup)

  • ExifTool (metadata)

  • Poppler tools (pdftotext)

  • FFmpeg (media metadata)

  • Python 3 + pip

  • rclone (optional sync/backup)

  • Docker or Podman (to run a local LLM, e.g., Ollama)

Python libraries (for perceptual hashing):

  • pillow

  • imagehash

Install commands:

APT (Debian/Ubuntu):

sudo apt update
sudo apt install -y jq inotify-tools smartmontools fdupes libimage-exiftool-perl poppler-utils ffmpeg python3 python3-pip rclone docker.io podman
python3 -m pip install --user pillow imagehash
# If using Docker:
sudo systemctl enable --now docker
sudo usermod -aG docker "$USER"
# Re-login for docker group to take effect

DNF (Fedora/RHEL/CentOS Stream):

sudo dnf install -y jq inotify-tools smartmontools fdupes perl-Image-ExifTool poppler-utils ffmpeg python3 python3-pip rclone moby-engine podman
python3 -m pip install --user pillow imagehash
# If using Docker (moby-engine):
sudo systemctl enable --now docker
sudo usermod -aG docker "$USER"
# Re-login for docker group to take effect

Note: On older Fedora/RHEL where ffmpeg isn’t in the default repos, enable RPM Fusion (see rpmfusion.org) before installing ffmpeg.

ZYPPER (openSUSE Leap/Tumbleweed):

sudo zypper refresh
sudo zypper install -y jq inotify-tools smartmontools fdupes perl-Image-ExifTool poppler-tools ffmpeg python3 python3-pip rclone docker podman
python3 -m pip install --user pillow imagehash
# If using Docker:
sudo systemctl enable --now docker
sudo usermod -aG docker "$USER"
# Re-login for docker group to take effect

Note: If ffmpeg is missing, enable the Packman repository (see the openSUSE wiki) and install ffmpeg from there.

Run a local LLM (Ollama) via container:

# With Docker
docker run -d --name ollama -p 11434:11434 --restart unless-stopped ollama/ollama:latest
# Pull a model (e.g., llama3.1)
docker exec -it ollama ollama pull llama3.1

OR with Podman:

podman run -d --name ollama -p 11434:11434 --restart=always docker.io/ollama/ollama:latest
podman exec -it ollama ollama pull llama3.1

1) Real-time AI file intake: auto-tagging and summaries

Goal: As new files land on your NAS, extract useful text/metadata and ask a local LLM to produce tags and a short summary. Save results in a sidecar JSON (safe and reversible).

Script: watch a directory with inotifywait, extract content, call Ollama, and store metadata.

#!/usr/bin/env bash
# ai-intake.sh: real-time tagging for new files

WATCH_DIR="${1:-/srv/data/incoming}"
META_SUFFIX=".meta.json"
OLLAMA_URL="${OLLAMA_URL:-http://localhost:11434/api/generate}"
MODEL="${MODEL:-llama3.1}"

mime_of() { file -b --mime-type -- "$1"; }

extract_payload() {
  local f="$1" mime; mime="$(mime_of "$f")"
  case "$mime" in
    application/pdf)
      pdftotext -nopgbrk "$f" - | head -c 8000
      ;;
    text/*)
      head -c 8000 -- "$f"
      ;;
    image/*)
      exiftool -j "$f" | jq -c '.[0]'
      ;;
    video/*|audio/*)
      ffprobe -v quiet -print_format json -show_format -show_streams "$f" | jq -c
      ;;
    *)
      echo "{\"name\":\"$(basename "$f")\",\"size\":$(stat -c%s "$f")}" 
      ;;
  esac
}

generate_meta() {
  local f="$1" content prompt
  content="$(extract_payload "$f" | tr -d '\000')"
  prompt=$'You are a file librarian. Given the content/metadata below, produce concise JSON:\n{"tags": [up to 8 short lowercase tags], "summary": "one sentence", "type": "kind of file"}\nContent:\n'"$content"
  curl -s "$OLLAMA_URL" \
    -H "Content-Type: application/json" \
    -d @- <<EOF | jq -r '.response' | jq .
{"model":"'"$MODEL"'","prompt":"'"${prompt//\"/\\\"}"'","stream":false}
EOF
}

mkdir -p "$WATCH_DIR"
echo "Watching: $WATCH_DIR"
inotifywait -m -e close_write,create --format '%w%f' "$WATCH_DIR" | while read -r f; do
  [ -f "$f" ] || continue
  out="${f}${META_SUFFIX}"
  echo "Processing $f ..."
  meta="$(generate_meta "$f" || echo '{}')"
  echo "$meta" | jq -c '{
    file: "'"$f"'", timestamp: now | todate, meta: .
  }' > "${out}.tmp" && mv "${out}.tmp" "$out"
  echo "Saved $out"
done

  • Start it with: bash ai-intake.sh /path/to/incoming

  • Output: /path/to/incoming/filename.ext.meta.json with tags/summary/type

  • Tip: write tags to extended attributes or embed XMP later if desired.

Security note: The model only sees extracted text/metadata. PDFs are truncated for performance. Tweak limits for your hardware.

2) Smarter deduplication: exact and “near-duplicate” images

First pass: exact duplicates with fdupes.

# List duplicate sets
fdupes -r /srv/data/library > /tmp/dupes.txt

# Optionally interactively delete
# fdupes -rdN /srv/data/library

Near-duplicates (images): Use perceptual hashing to find similar images (resized, minor edits).

Python helper phash_dupes.py:

#!/usr/bin/env python3
import sys, json, pathlib
from PIL import Image
import imagehash

root = pathlib.Path(sys.argv[1] if len(sys.argv) > 1 else ".")
thresh = int(sys.argv[2]) if len(sys.argv) > 2 else 5

db = {}
for p in root.rglob("*"):
    if not p.is_file(): continue
    if p.suffix.lower() not in [".jpg",".jpeg",".png",".webp",".bmp",".tiff"]: continue
    try:
        h = imagehash.phash(Image.open(p))
        db[str(p)] = str(h)
    except Exception as e:
        pass

# group similar by Hamming distance
paths = list(db.keys())
groups = []
visited = set()
for i,p in enumerate(paths):
    if p in visited: continue
    g = [p]; visited.add(p)
    for q in paths[i+1:]:
        if q in visited: continue
        d = imagehash.hex_to_hash(db[p]) - imagehash.hex_to_hash(db[q])
        if d <= thresh:
            g.append(q); visited.add(q)
    if len(g) > 1:
        groups.append(g)

print(json.dumps(groups, indent=2))

Run and quarantine:

python3 phash_dupes.py /srv/data/photos 6 > /tmp/near_dupes.json
jq -r '.[] | .[1:][]' /tmp/near_dupes.json | while read -r f; do
  q="/srv/data/quarantine/$(date +%F)"; mkdir -p "$q"
  echo "Quarantining near-duplicate: $f"
  mv -n -- "$f" "$q"/ 2>/dev/null || true
done

  • Adjust threshold (smaller = stricter). Manually review the quarantine folder before deletion.

  • Combine with rmlint if you prefer its reporting for exact/broken/empty files.

3) Predictive drive health: lightweight anomaly alerts from SMART

Pull SMART attributes daily, compute simple anomalies (z-scores) against your drive’s history, and alert via syslog (or email hook).

Collector smart-collect.sh:

#!/usr/bin/env bash
OUT="/var/log/nas_smart.csv"
DRIVES=(${DRIVES:-/dev/sd[a-z]})
[ -f "$OUT" ] || echo "ts,dev,raw_read_error_rate,realloc,pending,udma_crc,temp" > "$OUT"

ts="$(date -Is)"
for d in "${DRIVES[@]}"; do
  smartctl -A "$d" | awk -v ts="$ts" -v dev="$d" '
    $2=="Raw_Read_Error_Rate"{r=$10}
    $2=="Reallocated_Sector_Ct"{re=$10}
    $2=="Current_Pending_Sector"{p=$10}
    $2=="UDMA_CRC_Error_Count"{c=$10}
    $1=="194"||$2=="Temperature_Celsius"{t=$10}
    END{printf "%s,%s,%s,%s,%s,%s,%s\n", ts, dev, r+0, re+0, p+0, c+0, t+0}
  ' >> "$OUT"
done

Anomaly check smart-anomaly.py (no extra dependencies):

#!/usr/bin/env python3
import csv, sys, statistics, time
path = sys.argv[1] if len(sys.argv)>1 else "/var/log/nas_smart.csv"
recent_by_dev = {}
with open(path) as f:
    r = csv.DictReader(f)
    for row in r:
        recent_by_dev.setdefault(row["dev"], []).append(row)
def z(x, xs):
    xs = [float(v) for v in xs]
    if len(xs)<10: return 0.0
    mu = statistics.mean(xs); sd = statistics.pstdev(xs) or 1.0
    return (float(x)-mu)/sd
alerts=[]
for dev, rows in recent_by_dev.items():
    rows = rows[-200:]  # last 200 samples
    keys = ["realloc","pending","udma_crc","temp"]
    latest = rows[-1]
    for k in keys:
        series = [r[k] for r in rows[:-1]]
        zz = z(latest[k], series)
        if zz > 3.0:  # spike
            alerts.append(f"{dev} {k} z={zz:.1f} latest={latest[k]}")
if alerts:
    for a in alerts:
        print(f"SMART anomaly: {a}")
    sys.exit(1)
else:
    print("SMART OK")

Systemd timers (root):

/etc/systemd/system/nas-smart-collect.service

[Unit]
Description=NAS SMART collector

[Service]
Type=oneshot
Environment="DRIVES=/dev/sda /dev/sdb /dev/sdc"
ExecStart=/usr/local/sbin/smart-collect.sh

/etc/systemd/system/nas-smart-collect.timer

[Unit]
Description=Run SMART collector hourly

[Timer]
OnCalendar=hourly
Persistent=true

[Install]
WantedBy=timers.target

/etc/systemd/system/nas-smart-anomaly.service

[Unit]
Description=NAS SMART anomaly check

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/smart-anomaly.py /var/log/nas_smart.csv
StandardOutput=journal
StandardError=journal

/etc/systemd/system/nas-smart-anomaly.timer

[Unit]
Description=Run SMART anomaly check daily

[Timer]
OnCalendar=daily
Persistent=true

[Install]
WantedBy=timers.target

Enable:

sudo install -m 755 smart-collect.sh /usr/local/sbin/
sudo install -m 755 smart-anomaly.py /usr/local/sbin/
sudo systemctl daemon-reload
sudo systemctl enable --now nas-smart-collect.timer nas-smart-anomaly.timer

Hook up notifications:

  • Journal tail to email with an MTA or use a small script that sends webhook/Discord/Slack when exit code ≠ 0.

4) Safe ChatOps: natural language to shell, with guardrails

Let an LLM propose commands, but only execute after you confirm—and only from a whitelist of safe verbs/paths.

nasctl.sh:

#!/usr/bin/env bash
set -euo pipefail
PROMPT="${*:-"Summarize disk usage under /srv/data as a single safe command"}"
OLLAMA_URL="${OLLAMA_URL:-http://localhost:11434/api/generate}"
MODEL="${MODEL:-llama3.1}"

read -r -d '' SYS <<'SYS'
You are a command builder for a Linux NAS. Output ONLY JSON like:
{"cmd": "...", "reason": "..."}
Rules:

- Use ONLY these tools: ls, du, df, find, grep, sed, awk, fdupes, rclone, rsync, tar, touch, mkdir, mv, cp, rm

- Limit paths to /srv/data or /srv/backup

- Prefer read-only (no rm) unless user explicitly says delete

- One-liners; no pipes to curl|sh; no network except rclone if asked
SYS

payload=$(jq -nc --arg sys "$SYS" --arg prompt "$PROMPT" --arg model "$MODEL" \
  '{model:$model, prompt:($sys+"\nUser: "+$prompt+"\nAssistant:"), stream:false}')

resp=$(curl -s "$OLLAMA_URL" -H "Content-Type: application/json" -d "$payload" \
  | jq -r '.response' | tail -n1)

cmd=$(echo "$resp" | jq -r '.cmd // empty')
reason=$(echo "$resp" | jq -r '.reason // "n/a"')

if [ -z "$cmd" ]; then echo "No command returned."; exit 1; fi

echo "Proposed: $cmd"
echo "Reason:   $reason"
read -rp "Execute? [y/N] " yn
if [[ "$yn" =~ ^[Yy]$ ]]; then
  eval "$cmd"
else
  echo "Aborted."
fi

Examples:

./nasctl.sh "find duplicate files under /srv/data and list totals per folder"
./nasctl.sh "archive /srv/data/projects to /srv/backup/projects.tar.gz without overwriting"

You stay in control: nothing runs without your explicit confirmation.

Putting it together

  • Intake pipeline labels and summarizes new files automatically, making search and curation painless.

  • Dedup pass reclaims space by removing exact dupes and quarantining near-dupes for review.

  • SMART watcher nudges you about drift so you can migrate a drive before it dies.

  • ChatOps lets you move faster with safety rails when you can’t remember the perfect find incantation.

Optional: add rclone to sync tagged/curated content to cloud or offsite storage; encrypt with rclone crypt.

Example nightly sync:

rclone sync /srv/data remote:offsite --fast-list --transfers=8 --checkers=8 --immutable --log-file=/var/log/rclone-sync.log

Conclusion and next steps

AI doesn’t replace your NAS skills—it augments them. Start small: 1) Spin up the local LLM container. 2) Run the intake watcher on a test folder. 3) Trial the dedup scripts on copies. 4) Enable SMART timers and watch the logs for a week.

When you’re happy, roll it out to your real dataset. Consider adding:

  • XMP writing for permanent, searchable metadata

  • A small web UI to browse tags and summaries

  • A Git repo for your scripts and systemd units (infrastructure as code)

If you found this useful, subscribe and share what you automated—real-world tweaks and pitfalls help everyone level up their homelabs.

Related posts