brain.md vs soul.py: Comparing AI Agent Memory Architectures

brain.md from MindMux just hit Hacker News, and it represents a fundamentally different philosophy than the RAG+RLM hybrid we built in soul.py. Both solve the same problem — AI agents forgetting everything between sessions — but they make opposite bets about how memory should work. Let’s break down what each does well and where they diverge.

The Problem Both Solve

Every AI agent has the same amnesia problem: context dies when the session ends. Decisions made today, constraints agreed upon, architectural reasoning — all of it evaporates. The next session starts from zero.

Both brain.md and soul.py attack this. But they make different bets about how memory should work.

brain.md: Structured Knowledge Through CLI Discipline

brain.md’s core insight is brutal simplicity: route all writes through a CLI, and you can’t corrupt the structure.

brain create-page --id config-as-markdown --category decision \
  --title "Store config as Markdown, not SQLite"

The brain is a directory of Markdown files with strict frontmatter. Each “page” has:

• A compiled_truth section (the current best understanding — rewritable)
• An append-only timeline (the evidence chain — immutable)

When you update the truth, the CLI atomically rewrites the understanding AND appends why it changed. The two things a validator would guard are now structurally impossible to break.

What brain.md Gets Right

1. Zero runtime dependencies. It’s Node + Markdown. No vector database, no embeddings API, no cloud service. The brain lives in your repo and travels in git.

2. Correct by construction. The CLI is the only writer. You can’t accidentally malform a page because you never hand-edit them.

3. Agent-agnostic. Any agent that can read files can use the brain. Claude Code reads BRAIN.md and learns the protocol. Codex reads it. Anything can.

4. Tamper-evident timeline. The append-only evidence chain means you can audit how understanding evolved. “Why did we decide X?” has a traceable answer.

Where brain.md Falls Short

1. No semantic search. Finding relevant memories requires knowing what you’re looking for. There’s no “show me everything related to authentication” unless you structured categories perfectly upfront.

2. Manual discipline required. Someone has to run brain create-page. The agent has to be trained (via BRAIN.md) to use the CLI. Nothing is automatic.

3. Cold start problem. An empty brain is useless. The brain-bootstrap skill helps, but you still need deliberate seeding.

soul.py: RAG + RLM Hybrid with Zero-Friction Capture

soul.py takes the opposite approach: make memory capture automatic, make retrieval semantic, and route queries intelligently.

from hybrid_agent import HybridAgent
agent = HybridAgent()
result = agent.ask("Why did we choose Tailwind?")
# result["route"] → "RAG" (sub-second retrieval)
# result["answer"] → "We chose Tailwind for X, Y, Z..."

The architecture has three layers:

1. SOUL.md — Identity and personality (who the agent is)
2. MEMORY.md — Curated long-term memory (decisions, context, lessons)
3. RAG + RLM hybrid — Vector search for focused retrieval, full-context synthesis for complex queries

What soul.py Gets Right

1. Query routing. ~90% of queries go to RAG (fast, focused). ~10% go to RLM (exhaustive, slower). The router auto-classifies — no user decision needed.

2. Semantic retrieval. “Show me auth decisions” finds relevant memories even if you didn’t categorize them under “auth.” Embeddings beat categories for recall.

3. Zero-config start. soul init scaffolds SOUL.md and MEMORY.md. The v0.1 version has zero dependencies — just markdown injection. v2.0 adds Qdrant for vector search.

4. Graceful degradation. No Qdrant? Falls back to BM25 keyword search. No Azure embeddings? Uses local alternatives. Always works, just with less precision.

Where soul.py Falls Short

1. Requires infrastructure. Full v2.0 needs Qdrant + embedding API. That’s operational overhead brain.md avoids entirely.

2. No tamper-evidence. MEMORY.md is just a file. Anyone can edit it. There’s no audit trail of why memories changed.

3. Capture isn’t automatic. You still need to write to MEMORY.md. (Mem0 integration helps here, but adds another dependency.)

The Real Philosophical Divide

brain.md says: Memory is durable knowledge. Guard it with structure.

soul.py says: Memory is semantic context. Surface it with search.

brain.md is optimized for decisions that matter in six months. It’s a project’s institutional memory — the kind of thing you’d put in an ADR (Architecture Decision Record) but richer.

soul.py is optimized for everything the agent might need to recall. Preferences, facts, context, personality. It’s closer to how human memory works — fuzzy, associative, sometimes wrong but usually helpful.

The Convergence Path

The interesting future is combining both:

Layer	Tool	Purpose
Hot RAM	SESSION-STATE.md	Survives compaction, WAL protocol
Warm Store	soul.py RAG	Semantic search, auto-recall
Cold Store	brain.md pages	Durable decisions, tamper-evident
Identity	SOUL.md	Personality, boundaries, voice

This is essentially what a layered memory architecture looks like in practice — brain.md’s CLI discipline for the cold store is stricter than a git-notes approach, while soul.py handles the semantic warm layer.

When to Use Each Tool

Use brain.md if:

• You’re working on a long-running project with architectural decisions that outlive any single session
• You want git-portable memory with zero cloud dependencies
• You value auditability and tamper-evidence over fuzzy recall

Use soul.py if:

• You want semantic search across all memories
• You need graceful degradation (works without infrastructure, better with it)
• You’re building agent identity + memory together

Use both if:

• You want the best of both: structured decisions (brain.md) + semantic context (soul.py)
• You’re willing to manage slightly more complexity for maximum recall + maximum durability

What’s Missing From Both

Neither system solves automatic extraction. You still need to deliberately capture memories. Mem0 (now integrated with soul.py v2.1) gets closer — it auto-extracts facts from conversations and deduplicates them. That’s the real unlock: memory that writes itself.

The agent memory space is heating up. brain.md, soul.py, agentmemory, Memvid — each makes different tradeoffs. The pattern that’ll win is the one that’s:

1. Zero friction to capture (automatic > manual)
2. Zero latency to recall (semantic > categorical)
3. Zero dependencies to start (file-based > cloud-required)

We’re not there yet. But we’re getting closer.

Frequently Asked Questions

What is AI agent memory and why does it matter?

AI agent memory is a system that allows AI coding assistants like Claude Code, Cursor, or Codex to remember context, decisions, and preferences across sessions. Without memory, every conversation starts from zero — the agent forgets previous architectural decisions, user preferences, and project context. Memory systems like brain.md and soul.py solve this by persisting knowledge in files the agent reads on startup.

How does brain.md store memories?

brain.md stores memories as structured Markdown pages in a brain/ directory. Each page has frontmatter metadata, a “compiled_truth” section containing the current understanding, and an append-only timeline tracking how that understanding evolved. All writes go through a CLI tool, ensuring the structure can never be corrupted by malformed edits.

How does soul.py differ from brain.md?

soul.py uses semantic vector search (RAG) to find relevant memories, while brain.md uses structured categories. soul.py is better for fuzzy recall (“find anything related to authentication”), while brain.md is better for auditable decisions (“show me exactly why we chose PostgreSQL”). soul.py requires a vector database for full functionality; brain.md has zero dependencies.

Can I use brain.md and soul.py together?

Yes. The recommended architecture uses brain.md for durable architectural decisions (cold store) and soul.py for semantic context retrieval (warm store). SESSION-STATE.md handles active working memory (hot RAM), and SOUL.md defines agent identity. This layered approach gives you both auditability and semantic search.

What AI coding assistants support these memory systems?

Both brain.md and soul.py work with any AI assistant that can read files: Claude Code, Codex, Cursor, Aider, OpenClaw, Gemini CLI, GitHub Copilot, and more. brain.md uses a BRAIN.md protocol file to teach agents how to use the CLI. soul.py injects SOUL.md and MEMORY.md into the agent’s context on startup.

Do these memory systems send my code to the cloud?

brain.md is fully local — it’s just Markdown files and a Node CLI. soul.py v0.1 is also fully local (pure markdown injection). soul.py v2.0 can use cloud embedding APIs (Azure, OpenAI) for semantic search, but falls back to local BM25 keyword search if no API is configured. Neither system requires sending code to external services.

How do I get started with AI agent memory?

For brain.md: Clone the repo, run ./setup to install skills, then run brain-setup in your project to scaffold the brain. For soul.py: Run pip install soul-agent then soul init to create SOUL.md and MEMORY.md. Both are ready to use immediately with any file-reading AI assistant.

---

brain.md: https://github.com/mindmuxai/brain.md soul.py: https://github.com/menonpg/soul.py