Artificial Intelligence Agent Memory Explained (for Bash-friendly Linux users)

Ever had a terminal “AI buddy” forget the server IP you just told it, or re-ask for the same config over and over? That’s not rudeness—it’s missing memory. In this post you’ll learn what “agent memory” really is, why it matters for reliable automation, and how to bolt on a practical memory layer to your own command-line assistants using SQLite, Python embeddings, and a few simple Bash scripts.

You’ll leave with:

• A clear mental model of agent memory types and trade-offs

• A working long-term memory store you can query from Bash

• Steps to prune, segment, and harden memory

• Real-world examples you can adapt immediately

The problem and the payoff

LLMs have limited context windows and no persistent state by default. That means:

• Important details drop off as the conversation grows

• Sessions start from zero each time

• You waste tokens (and time) re-sending known facts

• Responses vary more when the model can’t anchor on consistent project conventions

A durable memory layer fixes this. It reduces cost and latency, stabilizes behavior, and lets your agent feel “stateful” while remaining auditable and privacy-conscious.

What “agent memory” actually means

“Memory” isn’t one thing. It’s a set of layers tuned for different lifetimes and purposes:

• Short-term/context window: tokens currently in the prompt. Fast, but volatile.

• Working memory/scratchpad: intermediate steps or extracted facts during a task.

• Long-term memory: durable store (e.g., SQLite + embeddings) for cross-session recall.

• Episodic vs. semantic:

  • Episodic: event-like logs (who, what, when).
  • Semantic: distilled knowledge or rules (“We deploy with Podman; not Docker.”).

• Retrieval Augmented Generation (RAG): when answering, retrieve relevant snippets from long-term memory and feed them to the model.

Below, we’ll implement a simple, robust long-term memory using SQLite and vector embeddings, then show how to keep it healthy with pruning and namespaces.

---

Quick start: Install dependencies

These are lightweight tools you likely already use: Python, SQLite, jq, curl, git. Choose the commands for your distro.

Debian/Ubuntu (apt):

sudo apt update
sudo apt install -y python3 python3-venv python3-pip sqlite3 jq git curl cron

Fedora/RHEL/CentOS (dnf):

sudo dnf install -y python3 python3-pip sqlite jq git curl cronie

openSUSE (zypper):

sudo zypper refresh
sudo zypper install -y python3 python3-pip sqlite3 jq git curl cron

Create a project and Python venv:

mkdir ai-agent-memory && cd ai-agent-memory
python3 -m venv .venv
. .venv/bin/activate
pip install --upgrade pip
pip install sentence-transformers numpy

Note: The first pip install of sentence-transformers will fetch a small, CPU-friendly model the first time it runs; it’s fine for prototyping.

---

Core idea: A minimal long-term memory you can call from Bash

We’ll create:

• A SQLite DB to store text “memories” with timestamps and namespaces

• A Python helper to embed and store memories

• A Python helper to recall the top-k most similar memories for a query

• Thin Bash wrappers you can call from any script or agent

1) Embed and store

Create embed_and_store.py:

#!/usr/bin/env python3
import sys, time, json, sqlite3, os
import numpy as np
from sentence_transformers import SentenceTransformer

# Usage: embed_and_store.py <namespace> <text...>
if len(sys.argv) < 3:
    print("Usage: embed_and_store.py <namespace> <text>")
    sys.exit(1)

NS = sys.argv[1]
TEXT = " ".join(sys.argv[2:])
DB = os.environ.get("MEMORY_DB", "memory.db")
MODEL_NAME = os.environ.get("EMBED_MODEL", "sentence-transformers/all-MiniLM-L6-v2")

model = SentenceTransformer(MODEL_NAME)
vec = model.encode([TEXT], normalize_embeddings=True)[0]  # unit vector
vec_json = json.dumps(vec.tolist())

con = sqlite3.connect(DB)
cur = con.cursor()
cur.execute("""
CREATE TABLE IF NOT EXISTS memory (
  id INTEGER PRIMARY KEY,
  ts INTEGER NOT NULL,
  namespace TEXT NOT NULL,
  text TEXT NOT NULL,
  vector TEXT NOT NULL
);
""")
cur.execute("CREATE INDEX IF NOT EXISTS idx_ns_ts ON memory(namespace, ts)")
cur.execute("INSERT INTO memory(ts, namespace, text, vector) VALUES (?,?,?,?)",
            (int(time.time()), NS, TEXT, vec_json))
con.commit()
con.close()
print("OK")

2) Recall by similarity

Create recall.py:

#!/usr/bin/env python3
import sys, json, sqlite3, os, heapq
import numpy as np
from sentence_transformers import SentenceTransformer

# Usage: recall.py <namespace> <query> [top_k]
if len(sys.argv) < 3:
    print("Usage: recall.py <namespace> <query> [top_k]")
    sys.exit(1)

NS = sys.argv[1]
QUERY = sys.argv[2]
TOPK = int(sys.argv[3]) if len(sys.argv) > 3 else 5

DB = os.environ.get("MEMORY_DB", "memory.db")
MODEL_NAME = os.environ.get("EMBED_MODEL", "sentence-transformers/all-MiniLM-L6-v2")

model = SentenceTransformer(MODEL_NAME)
qvec = model.encode([QUERY], normalize_embeddings=True)[0]

con = sqlite3.connect(DB)
cur = con.cursor()
cur.execute("SELECT id, ts, text, vector FROM memory WHERE namespace = ?", (NS,))
rows = cur.fetchall()
con.close()

def cosine(a, b):
    # vectors already normalized
    return float(np.dot(a, b))

scored = []
for (id_, ts, text, vjson) in rows:
    v = np.array(json.loads(vjson), dtype=np.float32)
    s = cosine(qvec, v)
    scored.append((s, {"id": id_, "ts": ts, "text": text, "score": s}))

top = heapq.nlargest(TOPK, scored, key=lambda x: x[0])
print(json.dumps([item for _, item in top], ensure_ascii=False, indent=2))

3) Bash wrappers with basic sanitization

Create mem_add.sh:

#!/usr/bin/env bash
set -euo pipefail

NS="${1:?namespace}"
shift
RAW_TEXT="${*:?text}"

# Basic redaction of obvious secrets before storing
SANITIZED="$(printf '%s' "$RAW_TEXT" \
  | sed -E 's/(api[-_ ]?key|token|password)\s*[:=]\s*\S+/REDACTED/gi')"

python3 embed_and_store.py "$NS" "$SANITIZED"

Create mem_search.sh:

#!/usr/bin/env bash
set -euo pipefail

NS="${1:?namespace}"
QUERY="${2:?query}"
TOPK="${3:-5}"

python3 recall.py "$NS" "$QUERY" "$TOPK" | jq -r '.[] | (.score|tostring) + "\t" + .text'

Make them executable:

chmod +x mem_add.sh mem_search.sh embed_and_store.py recall.py

Initialize the DB (optional; it’s auto-created):

sqlite3 memory.db 'PRAGMA user_version;'

Smoke test:

./mem_add.sh infra "Primary DB is 10.0.0.12; Replica is 10.0.0.13"
./mem_add.sh infra "Deployments use Podman with systemd units"
./mem_search.sh infra "Where is the database?"

You should see the DB fact bubble to the top with a high similarity score.

---

Actionable steps and patterns

1) Model your memory by lifetime and purpose

• Short-term: stay in the prompt only what you need to solve the current step.

• Long-term: store stable facts (endpoints, conventions, runbooks, decisions) in SQLite via mem_add.sh.

• Separate namespaces per project, team, or environment: infra, payments, mobile-app, etc. This reduces cross-talk and speeds up recall.

2) Keep memory healthy: prune and compact Create prune.py to keep only the newest K items per namespace:

#!/usr/bin/env python3
import sys, sqlite3

# Usage: prune.py <namespace> <keep_k>
ns = sys.argv[1]
keep_k = int(sys.argv[2])
db = "memory.db"

con = sqlite3.connect(db)
cur = con.cursor()
cur.execute("""
DELETE FROM memory
WHERE namespace = ?
AND id NOT IN (
  SELECT id FROM memory WHERE namespace = ?
  ORDER BY ts DESC LIMIT ?
)
""", (ns, ns, keep_k))
con.commit()
con.close()
print("pruned", ns, "keep", keep_k)

Run it manually or via cron:

• Debian/Ubuntu:

  (crontab -l 2>/dev/null; echo "0 3 * * * cd $HOME/ai-agent-memory && . .venv/bin/activate && python3 prune.py infra 1000") | crontab -
  

• Fedora/RHEL/CentOS (cronie):

  (crontab -l 2>/dev/null; echo "0 3 * * * cd $HOME/ai-agent-memory && . .venv/bin/activate && python3 prune.py infra 1000") | crontab -
  

• openSUSE:

  (crontab -l 2>/dev/null; echo "0 3 * * * cd $HOME/ai-agent-memory && . .venv/bin/activate && python3 prune.py infra 1000") | crontab -
  

3) Add retrieval to your agent workflow Before calling your model, fetch the top-k memories and inject into the system or user prompt. Example stub:

MEMS="$(./mem_search.sh infra "database connection details" 3 | cut -f2-)"
PROMPT="You are an ops assistant. Relevant facts:\n$MEMS\n\nUser question: How do I connect to the DB?"
# Now call your model (local or remote)

4) Guardrails: sanitize and scope

• Redact secrets before mem_add.sh (already shown).

• Add per-namespace export/import:

  sqlite3 memory.db "SELECT ts, text FROM memory WHERE namespace='infra' ORDER BY ts" > infra.tsv
  

• Keep your memory file under versioned or encrypted storage if needed (e.g., age, gpg, or OS disk encryption).

5) Observe and iterate

• Log queries and recall hits to see what actually helps the model.

• When a session goes well, extract the “lesson” as a semantic memory:

  ./mem_add.sh backend "When building on CI, pin Python 3.11 to avoid ABI mismatch"
  

• Periodically convert episodic logs into rules/conventions. This distillation shrinks token costs and boosts reliability.

---

Real-world examples

• Terminal coworker: Your shell agent remembers “Primary DB is 10.0.0.12” and “use Podman for deploys,” so it generates fewer wrong commands and less trial-and-error.

• On-call copilot: It recalls “Nginx 502s on payments are usually upstream timeouts; check systemd-journald and gunicorn workers” to shortcut incident triage.

• Code-review aide: It learns “We prefer shellcheck-clean Bash with set -euo pipefail and trap for cleanup,” and references that consistently in suggestions.

---

Why this approach works

• SQLite is battle-tested, simple, and scriptable from Bash.

• Embeddings let you retrieve relevant context without brittle keyword matches.

• Namespaces and pruning prevent slowdowns and topic mixing.

• Keeping the memory outside the model keeps you private, auditable, and portable across LLM providers (or local models).

---

Optional extensions

• Swap in a vector DB (FAISS, Chroma, Qdrant) when your memory grows to millions of items.

• Add a feedback loop: bump a “weight” when a memory is used successfully; decay unused ones.

• Use systemd timers instead of cron if you prefer:

  # Example skeleton
  systemctl --user enable --now your-timer.service
  

---

Conclusion and next steps (CTA)

You now have a practical, Bash-first memory layer: add facts once, retrieve them reliably, and keep the store tidy. Next:

• Wire mem_search.sh into your agent prompt building

• Create 2–3 namespaces for your main projects

• Add a daily prune job

• Start logging which memories actually improve answers

If you want a deeper dive, extend the store with weights and summarization, or plug in a dedicated vector DB. But don’t wait—drop this SQLite-backed memory into your scripts today and make your AI tooling actually remember what matters.