Memory System — Audit & Redesign Options

The agent's workspace brain: observations → beliefs → routing decisions

7 min

Diagnosis in one line: we built a fuzzy-recall system (embeddings + LLM-summarized prose beliefs + confidence decay) to do a job that is mostly structured lookup (who's the vendor for trade T at property P; does owner O need approval). The mismatch is why confidence math keeps breaking and routing flakes.

What we have What's broken The reframe Options (9) Recommendation

What we have today

Two layers. A per-phone profile (legacy, demo/pre-workspace) and the real one: a per-workspace memory graph in Supabase.

Piece	Shape	Role
observations	`summary`(embed) · raw_text · entities · salience · tags · session_id	Episodic log. Cheap, write-liberal.
beliefs	`claim`(embed) · scope · confidence · explicitness · tags · status	Durable rule-like prose. The routing surface.
entities	vendor / property / owner nodes (+ `ref_id` to PMS)	Anchors. Property→owner cascades up.
joins	observation_entities (±weight) · belief_entities · belief_evidence	Structural recall + audit trail.

Consumed where: WO intake (new-work-order-message) does deterministic recall → top-18 candidates → an LLM picks vendor_id from the roster. Chat turns (reminders) inject top beliefs for the relevant entity. Confidence: conf·0.95 + |w|·sal·factor (GAIN .15 / PENALTY .2).

What's broken

#	Problem
1	Confidence dynamics erode good beliefs. Multiplicative `DECAY=0.95` on every attach outpaces `GAIN=0.15` at mid-high conf → reinforcement plateaus low. `PENALTY (.2) > GAIN (.15)` so contradictions bite harder than support.	broken
2	Over-broad beliefs collect off-target negatives. A vendor-wide claim gets a "−" every time a legit per-subtype specialist is chosen instead → reads as contradiction, decays the truth.	broken
3	Duplicates split evidence. "handles appliances" vs "appliance repairs" → two beliefs, neither accumulates. Dedup is a prompt rule, not a DB constraint.	bug
4	Retrieval rank couples conf×evidence. `rankBelief = conf·log2(1+ev)` → a high-evidence/low-conf belief out-ranks a high-conf/low-evidence one. Ignores trade match entirely.	bug
5	No structured trade/scope. Trade lives inside free-text prose; inferred at read time by a keyword map. Can't do a clean "best vendor for plumbing at P" query — leans on embeddings + LLM judgment every time.	design
6	Two LLM hops, nondeterministic. write → belief-former(LLM) → read → decide(LLM). Generalization re-decided by a model each pass; owner-approval & preference buried in prose, not queryable facts.	design
7	Only live fix is a hack. The `pinned` tag freezes confidence entirely (no decay/penalty) to nail down known-correct rules. Escape hatch, not a model. No deprecate/forget path otherwise.	band-aid

The reframe

Your four target decisions are not "recall" — they're typed lookups with a scope hierarchy. Name them as facts and most of the breakage disappears.

Decision	Really a…	Key
Which vendor to dispatch	ranked preference lookup	`(trade, scope) → vendor[]`
Which issues need owner approval	policy lookup	`(issue_type, scope) → bool`
Which properties/owners need approval	policy lookup	`(owner\|property) → bool`
How it changes per property / WO	scope resolution	`unit → property → owner → workspace` (most specific wins)

Everything else PM-related (owner is fussy about comms, building has a gate code, tenant is the onsite manager) is soft context — that part genuinely is recall, and a belief/obs store is fine for it. The redesign question is really: should the four hard decisions stay in the fuzzy store, or move to a structured one?

Options

Ordered roughly cheapest → most ambitious. Not mutually exclusive — the recommendation combines a few.

A · patch in place

Fix the confidence math

Kill multiplicative decay; make support monotone
GAIN ≥ PENALTY; cap, don't erode
DB-unique on (vendor, trade) to stop dup splits
Rank on confidence, add trade match

Trade: ~1 day, low risk. Treats symptoms — keeps the prose-belief + embedding model that's the root mismatch (#5/#6).

B · structured rules

Typed routing & approval tables

vendor_pref(trade, scope, vendor, rank, source)
approval_rule(scope, issue_type, requires)
Lookup = SQL on resolved scope, deterministic
LLM only extracts facts into rows; no embed for routing

Trade: directly answers all 4 goals; debuggable, no decay math. Cost: schema + extraction pipeline + migrate seeds.

C · count, don't judge

Frequency layer from history

Vendor prior = actual dispatch counts per (scope, trade) + recency
"Approval asked N times for owner O" → learned policy
Deterministic aggregate, no confidence model

Trade: removes the whole confidence-math surface. Needs clean dispatch logging; counts alone miss stated exceptions (pair with B).

D · markdown per entity

CLAUDE.md-style entity docs

One human-readable doc per owner/property/vendor
Agent reads relevant docs into context, edits directly
No embeddings, no confidence, fully transparent

Trade: dead simple, easy to inspect/correct by hand. Scales to hundreds, not thousands; retrieval = "which docs to load."

E · raw-log, recall-at-read

Drop consolidation entirely

No belief-former. Keep rich observations only
At decision time, retrieve obs (entity+vector+recency)
Decision LLM reasons over raw evidence

Trade: deletes #1–#3 outright. Cost + latency move to read time; every decision re-derives from scratch (no cache).

F · one brain doc

Workspace brain, self-edited

Single sectioned markdown per workspace (vendors / approval / quirks)
Agent rewrites it incrementally, lives fully in context
Zero retrieval infra

Trade: perfect for 2 customers today. Won't scale to thousands of units/entities in one prompt; merge conflicts as it grows.

G · vote tally

Confidence = support − contradict

Replace decay math with a monotone vote count
Recency tiebreak; "stated" rules pinned on top
Keeps belief graph, fixes the dynamics by deleting them

Trade: small change, keeps current architecture. Still prose beliefs + embedding recall (#5/#6 unaddressed).

H · mine the PMS

History-first prior

Ground truth for "vendor for property X" = AppFolio WO history
Seed routing prior by mining dispatch records directly
Memory only captures overrides / stated exceptions

Trade: kills cold-start, highest signal. Needs PMS history access per customer; Propertyware shape differs.

I · hybrid

Structured routing + soft graph

B/C/H for the 4 hard decisions (deterministic)
Lightweight obs store for soft context only
Belief-former demoted to "summarize quirks," off the routing path

Trade: most surface area to build, but each piece does what it's good at. The end-state the others converge toward.

Recommendation

Short term (this week): low risk

AStop the bleeding — kill multiplicative decay + the conf×evidence rank (Options A+G). One PR, immediately stops eroding correct beliefs. The pinned hack can then retire.

Real fix (the bet): B + H + C

1Move the 4 hard decisions to typed tables (B). Vendor preference and approval become SQL lookups on resolved scope — deterministic, inspectable, no confidence math.
2Seed from PMS history (H) + keep a live dispatch count (C) as the prior. The agent's observations then only need to capture stated overrides.
3Demote the belief graph to soft context (I). Keep it for "owner quirks" / nuance, off the routing path — where fuzzy recall is actually the right tool.

Why this over "just fix the math": the math breaks because we're forcing a scope-hierarchy lookup through a single scalar confidence. Typed scope + counts represent the decision natively, so there's no scalar to erode.

Open question for you: at current scale (2 customers), is Option F (one self-edited brain doc per workspace) actually enough for the next milestone — letting us defer all the graph work? It may be the highest-leverage move even if it's not the end-state. Worth a 10-min gut check before committing to the B+H+C build.