The score that ranked Primex 4th is not salience. It's a confidence-weighted heuristic with one flat term that dominates everything: a +1 structural bonus for being entity-linked. Here's the formula, the broken confidence dynamics behind it, and why the new intake path is harder to debug.
9 minEvery candidate read_memory returns carries a score the list is sorted by. For beliefs it is computed in tools/read_memory.mjs as:
score = confidence × log₂(2 + evidence_count) + 1 if structural
Salience does not appear here at all. Salience is a property of observations, used in a different formula — rankObservation = salience × |weight| × recency_decay. Beliefs don't have salience; they have confidence + evidence_count. So for the wasp WO (where the top candidates were all beliefs), salience never entered the ranking.
Look at the ranges. The whole confidence × log₂(2+evidence) term lives roughly in [0, ~1.5] for normal beliefs (confidence caps at 1, evidence_count is usually 0–3). A flat +1 is bigger than the entire spread of the confidence term. So being entity-linked is worth more than being near-certain. A belief at 0.5 confidence that's wired to the property outranks a belief at 0.9 that was found by meaning. Linkage, not strength, decides the order.
The two beliefs at the top of the wasp-WO result, with the real numbers from the turn log plugged into the formula:
| belief | tier | conf | evidence | math | score |
|---|---|---|---|---|---|
| "onsite manager handles coordination at 221-229 Union Pl" (irrelevant to vendor pick) | structural | 0.576 | 1 | 0.576 × log₂(3) + 1 = 0.913 + 1 |
1.91 |
| "Primex handles all pest issues, preferred over Dever" (the answer) | semantic | 0.806 | 0 | 0.806 × log₂(2) + 0 = 0.806 × 1 |
0.81 |
Primex is the more confident belief (0.81 vs 0.58) and the only relevant one — yet it ranks lower. The sole reason: the onsite-manager belief is wired to the 221-229 Union Pl property entity, so it collects the +1 structural bonus. The Primex belief is a portfolio-wide claim with no link to this specific property, so it arrives via semantic recall and gets no bonus. Confidence loses to linkage by design.
Two structural side-effects make it worse: (1) the structural tier matches on property/owner edges and ignores trade — so a coordination belief and a vendor belief compete on equal footing as long as both touch the property; (2) a portfolio default can never earn the +1 on any single property, because it isn't linked to one. That's the trade-off the team took deliberately — per-property exceptions are supposed to beat portfolio defaults locally (see §3) — but here the property-linked beliefs were the irrelevant ones.
The standing project note (diagnosed 2026-06-02 from the Cross-Appliance-for-disposal misroute). This is the why behind the numbers above.
"Yonic handles ALL plumbing" took 9 of 19 negative attaches — every time the PM correctly routed a toilet to Abraham or a drain clog to The Drain Specialist, it logged as a contradiction and dragged Yonic to 0.50, even though every disposal signal was positive. Specialization reads as contradiction.
applyConfidenceDelta does next = old×0.95 + |w|×salience×GAIN(0.15). Medium-salience positive evidence asymptotes around ~0.6 — a belief can't climb to 0.9 from routine confirmations, no matter how many. (This is why the Primex belief sits at 0.81 with evidence_count 0 — it was seeded, not earned upward.)
A duplicate "Yonic handles plumbing incl disposals" was created rather than attaching to the seed belief, splitting the evidence across two rows. Violates its own consolidation rule.
The same conf × log₂(…) sort means evidence_count outweighs confidence, and the structural tier surfaces by property/owner edge ignoring trade — so a vendor with any owner-linked history at a property ranks high there regardless of whether it does that trade. This is exactly the wasp bug.
pinned tagA pinned tag freezes a belief's confidence in belief-former.mjs applyOp — evidence still recorded, but no decay and no penalty. The pattern that holds it together: portfolio defaults live in a pinned, trade-scoped belief (semantic, no +1); per-property exceptions live in property-linked beliefs (structural, +1 → beat the default locally). Pinned so far:
2d210fb0Yonic handles garbage disposals, never Cross Appliance — @ 0.96b0fdf77Guox preferred at 6337 Primrose Ave — @ 0.9, scope {} (fires on any Primrose issue)a9773cd6Yonic = portfolio default plumber (drains/disposals/heaters/faucets) — @ 0.9, scope {plumbing}; delinked from 824 11th St so the local Drain-Specialist exception still wins thereNote: there is no pinned pest belief. The Primex belief (668a4a96) sits unpinned at 0.81 with scope {trade: pest, problem_subtype: rodents} — exactly the gap the wasp WO fell through. A one-line fix is to pin it and broaden the scope to portfolio pest.
Rework observation→belief so: (a) beliefs are scoped narrow enough that specialization isn't contradiction, (b) reinforcement can reach high confidence, (c) retrieval ranks by trade-relevance, not raw evidence_count. The Jan seed graph was good; the live-turn pipeline is what degrades it.
The wasp WO ran on the old orchestrator path. The live path is now new-work-order-message.mjs — a single structured-JSON call. Same bug surface, less visibility:
update_issue(vendor_id) tool call — a real model action you can point at in steps[].vendor_id is one field in a forced json_schema output. No separate decision step, no reasoning token — reasoning is null and there's no room for chain-of-thought.compactMemory → top-18 by score before the model sees it. A belief below the cutoff is invisible to the model — a new silent-truncation failure mode the old path didn't have.Net: debugging goes from "read the tool trace, see the ranked list and the discrete pick" to "re-run the whole intake and stare at a single completion's inputs, with no reasoning to inspect and a top-18 cutoff that may have hidden the right answer entirely." None of that requires re-adding the loop to fix — see §5.
Keeping intake out of the orchestrator is the right call, and it's an asset here: a pure single-shot decideMessage({issue, vendors, memory}) is more replayable and assertable than a streaming multi-tool turn. The loop never gave the visibility — discrete logged steps did. So don't re-add the loop; add the signal back into the pipeline. Five changes, highest leverage first.
Add reasoning fields to schema() and put them before vendor_id. Structured outputs generate fields in schema order, so a reason that precedes the answer is real chain-of-thought; one that follows it is post-hoc narration the pick can't depend on.
properties: {
title, urgent, urgency_reason, issue_sentence,
vendor_reasoning: { type: 'string' }, // ← BEFORE vendor_id: quote the provenance you used
considered: { type: 'string' }, // runner-up + why rejected
vendor_id, vendor_display
}
// strict mode: every key must be in `required` (use anyOf[T,null] for optional)Why: the trace now reads "picked Primex — belief 668a4a96 'preferred over Dever', conf 0.81" or, in the bug, "picked Dever — name match, no history." That string IS the answer to "why". A few output tokens, zero structural change.
Have decideMessage return the exact messages it sent and the raw response, and store them on the turn. Today the step log shows the full memory and hand-built toolCall(...) wrappers — not the compactMemory top-18 the model actually saw.
return { decision, sent: messages, raw: content }; // from decideMessage
// then in writeCompletedTurn: steps.push({ i, llm_io: { sent, raw } })Why: kills the "synthetic trace" and "truncated input invisible" problems at once — you see the exact bytes in and out, including the 18 candidates the model was actually given.
compactMemory slices top-18 silently. Record how many were shown vs total, and the rank + score of the belief that supports the chosen vendor — and whether it made the cut.
{ memory_total: 22, memory_shown: 18,
chosen_support: { id: '668a4a96', rank: 4, score: 0.81, shown: true } }Why: tells you which bug you have. Right belief at rank 19 (cut) = a truncation bug. Rank 4 and ignored = an adherence bug. Different fixes — and today you can't tell them apart without re-running.
After the decision, a pure check (no model, no loop): does a pinned / high-confidence belief name a different vendor for this trade? If so, stamp the draft and turn with a flag and show ⚠ in the queue.
if (pinnedVendorForTrade(memory, trade) &&
pinnedVendorForTrade(memory, trade) !== decision.vendor_id) {
draft.flags = ['memory_override']; // renders ⚠ "overrode memory" in the UI
}Why: turns a silent override into a loud one. The wasp WO would have been flagged the moment it queued, instead of caught by eye in a screenshot the next day. Highest-value item after #1.
The turn already stores trigger.payload (issue + candidate vendors). A small scripts/replay-intake.mjs <turn_id> reconstructs the inputs and re-runs decideMessage, printing prompt → reasoning → pick. Pin temperature: 0 in the intake call so replay reproduces the same decision.
Why: because it's a pure function outside the loop, replay is exact and cheap — the payoff of the architecture you're keeping. Iterate on the prompt against a real failing turn without waiting for a live WO.
The throughline: out-of-loop isn't the debugging problem, silence is. Reasoning-in-schema (#1) + literal I/O logging (#2) + visible truncation (#3) + an override flag (#4) restore everything the old tool-steps gave you, while keeping the clean single-shot pipeline. The deterministic trade × workspace → vendor lookup from §3 still sits on top as the correctness fix; #4 is its debugging twin — one prevents the wrong pick, the other makes any remaining wrong pick loud.