Prompt + Context Architecture — Review
An honest read on the current prompt stack + build-context.mjs, tuned for gpt-5.4 on the Responses API. You asked me to push on your calls — so I did.
assistant/user history, markdown developer messages, one clean builder, action-framed tools — all correct and modern. The problems are two: (1) you've handed a reasoning model a stack of rules and prohibitions where it needs examples, and (2) the message array fragments the instructions into five developer turns while burying the one fact that decides every turn — the open-WO list — five messages away from the message it has to interpret.
What's right (don't touch it)
- Real conversation history as
assistant/userturns, not<system-reminder>XML. Correct, and what the API wants. - Markdown developer messages over XML wrappers. Right.
- One
build-context.mjsthat owns the whole array; orchestrator stays dumb. Clean seam. - Killed the skill router +
use_skill. The model-picks-its-own-skill thing was pure ceremony for one workflow. Good riddance. - Action-framed tools (
dispatch_vendor/text_tenant) with the draft staging hidden under the tool. The agent should believe it acts. Correct instinct.
Where I'd push back
work_orders.md is 186 words of state-machine spec (states, transitions) with zero worked examples. messages.md is mostly prohibitions — "avoid / don't / never." A reasoning model reads abstract rules and interprets them, differently each time — which is the exact source of the "sometimes asks, sometimes approves all" inconsistency. You can't rule your way to judgment or voice.
FIX Replace the prose rules with ~6–12 real worked exchanges spanning the range (approve / override / "i got it" / batch yes / genuine ambiguity → ask / casual non-WO → just chat). Examples pin behavior; rules suggest it.
Factual context ships headers like Open Work Orders (oldest first; recent bundle sent close together) and a ← most recent tag. That burst/recency marker is a real signal — but nothing in the prompt tells the model what to do with it. So it guesses. That's the inconsistency, surfaced: a hint with no key.
FIX Either teach it ("if WOs were sent close together, a bare 'yes' is ambiguous → ask which") or drop the markers and let the assistant/user history carry recency. Don't ship an unkeyed signal.
The open-WO list sits at array position 4, ahead of the entire conversation. Then history, then the latest "yes" last. To interpret "yes," the model needs that list — but it's 5+ messages back. Reasoning models weight recent context heavily; you've put the most decision-relevant data the farthest from the decision.
FIX Restate the open WOs (compact) adjacent to the latest user message — e.g. a short "context for this reply" right before the final user turn. Test it; this is the kind of move that visibly changes the ambiguity behavior.
soul.md, messages.md, work_orders.md are three separate developer turns. They're all "who you are + how to act" — the model reads them as one block anyway, so the split buys nothing and risks them reading as detachable. The clean boundary isn't soul-vs-messages-vs-workorders; it's static instructions (cacheable, identical every turn) vs per-turn facts (change every turn).
FIX Collapse the three static files into one developer "system" message (keep the files on disk, concatenate at load). Keep factual context as the second developer message. Two developer turns, not five. Cleaner for the model and for prompt caching once the static block grows with examples.
{role:'developer', content:'# This is your current conversation with the user'} is a standalone developer turn wedged between the facts and the history. The assistant/user roles already say "this is the conversation." Inserting a developer turn mid-flow is unusual and can muddy turn-taking. "The 8/8 smoke test included it" proves it didn't break — not that it helps.
FIX Drop it, or fold the one line into the end of the factual-context message. Don't spend a role turn on a header.
We already caught draft_tenant's description steering the workflow ("use this together with…"). Tool descriptions are part of the live prompt; when they imply behavior, they compete with work_orders.md and the model gets mixed signals.
FIX One source of truth: behavior in the prompt, tool descriptions limited to mechanics (what it does, params). Audit the rest of the tools for stray policy.
soul.md contradicts messages.mdsoul.md:43 "Do not say an action happened unless a tool confirmed it" reads as permission to confirm; messages.md says "avoid narrating your actions." A reasoning model surfaces contradictions as inconsistent behavior — this is part of the "Sent." problem.
FIX Make 43 a pure truth-guard ("never claim an action you didn't take") and let messages.md own the narration rule. No overlap.
The array — now vs how I'd shape it
Now (5 developer turns)
developer: soul.md
developer: messages.md
developer: work_orders.md
developer: factual context
developer: "# This is your current
conversation…"
assistant: …
user: … (history)
user: latest ← facts are 5 back
Proposed (2 developer turns)
developer: SYSTEM soul + voice + WO
behavior + FEW-SHOT
examples — static, cached
developer: FACTS runtime + open WOs
assistant: …
user: … (history)
user: latest + compact open-WO
restate, inline ← facts adjacent
The two structural moves: fold static instructions into one cached block, and pull the open-WO facts next to the message they decide. The behavioral move (the big one): the SYSTEM block carries examples, not just rules.
gpt-5.4 specifics
- You're fighting the reasoning default. Reasoning models are tuned to complete tasks and report — terse, "did it, done." That disposition is why it says "Sent." and skips asking. Rules nudge it; they don't override it. Only two things reliably do: strong few-shot examples of the target voice, or a warmer base model for the chat turn. If "texts like a human" is core, decide consciously whether a reasoning model is even right for this surface.
- Temperature is inert (we tested — identical output at temp 0 and 1). So variation can't come from sampling, and the eval-mode
temperature: 0means the demo shows you a frozen sample — don't over-fit the prompt to one output. - Reasoning isn't carried across turns (
store:false, encrypted CoT echoed only within a turn). So every turn's quality rests entirely on whatbuildMessagesassembles — which makes the history fidelity and the fact placement above matter more, not less. - Contradictions cost more here. A reasoning model that finds soul.md and messages.md disagreeing will "reason" its way to an answer — and you'll see that as nondeterministic-looking behavior. Internal consistency is a feature, not polish.
What I'd do, in order
- 1 · Pin behavior as ~10 scenarios (decisions/tools, not words) — the act of writing them settles the multi-WO question. These become the eval set.
- 2 · Write the SYSTEM block with few-shot examples — real text threads (yours), spanning act / ask / chat. This is ~80% of both the voice and the consistency fix.
- 3 · Collapse 5 developer turns → 2 (static SYSTEM + per-turn FACTS); drop the delimiter.
- 4 · Move the open-WO restate next to the latest message and run the ambiguity scenarios to confirm it flips.
- 5 · Teach or drop the burst marker, reconcile the soul/messages contradiction, de-policy the tool descriptions.
- 6 · Keep few-shot examples and eval scenarios disjoint — or you're grading on your own answer key.
Reviewed against current: core/context/build-context.mjs, core/prompts/{soul,messages,work_orders}.md, the Responses-API orchestrator loop.