This started out as a quick project to try to include an LLM as part of a working application, and grew into me writing four separate agent harnesses to learn how it all works.
The basic idea was to simulate a tabletop roleplaying game, with an LLM in the role of “dungeon master”. This is an interesting problem because the LLM can handle the narration and storytelling part pretty well; but on its own it will lose track of the details (such as “what items is the player carrying” or “which room are we in right now”). So you need conventional code to keep track of those details and feed them back to the AI as needed; and you need a way for the AI to modify that game state when the player moves to a new room or picks up a new item.
With frontier AI models this is very straightforward: just say to the AI “hey let’s play D&D” and the AI says “sure let’s go”. General-purpose agents with a really smart LLM model are, it turns out, pretty decent at this already. But it’s not terribly interesting from a coding point of view.
Instead I’m working with open-source models [1], and using them directly instead of through a general-purpose agentic workflow. Which means what I’m building is a custom agentic workflow. (This is a little bit of an oversimplification, but the LLM, the large language model, is the magic bag of weighted numbers that can understand English. The conventional code that wraps around that in various ways is the “agent”). Free models tend to be a lot less capable of keeping track of all the details over a period of time, making the agentic wrapper much more important.
So if you start playing you’ll see four “modes” at the top of the screen. You can switch between these at any time, even in the middle of the story; which means you’re switching between four agent harnesses:
- “Freeform” doesn’t even try to keep track of game state; whatever you say gets passed to the model, along with a summary of what happened before; the LLM says what happens next.
- “Direct” makes a similar call to the LLM, and in effect asks it to include in its response special signals that will tell the conventional code to add a room or add a new item. Conventional code handles those signals and includes the result in the next LLM call. (So for example if the player picks up a sword, the LLM will include in its response a signal that means “Add sword to inventory”, conventional code will look for that signal, add a sword to the player’s inventory, and then in the next step of the story will list the sword as part of the palyer’s inventory. Special handling is built in for the map locations, items, and non-player-characters.
- Direct mode fails often, because we’re asking the LLM to do a lot of things at once. “Managed” mode works similarly but breaks things up into separate LLM calls: an item manager looks specifically for items and decides if they should exist, then reports that back to the conventional code; a location manager looks specifically for changes in location; and so on, each as a separate LLM loop. Finally a narrator LLM call is made to write the text explaining what just happened to the player.
- Finally, “Tool-using” mode uses a different code pattern to do the same thing. Instead of the conventional code looking for signals in the LLM output that it should do inventory or map management, the LLM is provided with an array of “tools” it’s allowed to use. These tools are basically the same code functions as before, but written in such a way that the LLM is choosing which ones to use and when, and invoking them directly instead of signaling the conventional code that it needs to run them.
Detailed sequence diagrams are below, if you’re curious. You can also open the “Turn progress” collapsible while playing to see exactly what LLM calls are being made, what they’re being provided, and how they respond, to compare the results.
So which one’s the best?
Freeform is pretty obviously just winging it. It starts off well but quickly goes off track.
Direct mode just barely works. Open source LLM models can’t really keep track of everything at once, so it’ll frequently lose track of where you are or what you’re carrying.
Managed mode is the slowest, because it involves a lot of separate LLM calls, but probably works the best from a technical standpoint. The main problem is that the story often fails to advance: everything thrums with portent and meaning, but little actually happens — this is because the various managers make their decisions about items, locations, and NPCs before the narrator gets involved, so the narration is highly constrained by what the managers have already decided. If I were writing this from scratch I’d probably have the narrator act first, then have the managers update state according to the narrator’s decisions.
Tool-using mode is the most AI-native technique — the code patterns for “tools” are fine-tuned directly into the LLM model, so it already “understands” how to use this format. But this pattern leaves a lot of control in the AI’s hands instead of the program, since the AI is the one who decides when or if to call each tool. So it’ll often “forget” to change your location or to add an item to your inventory even if you’re directly interacting with it.
The hairy details
Freeform Mode
Freeform treats the LLM as a storyteller rather than a world-state manager. The narrator receives the player’s action and a continuity summary, or the opening narration on the first turn, and returns plain prose. It receives no tracked world state, JSON output schema, or tools. Apart from preparing missing history, each turn needs just one LLM call.
Only the action and narration are added to history. Tracked locations, inventory, NPCs, and game-over status remain unchanged, so the sidebar can become stale as the story develops. This makes Freeform the least constrained mode, but switching to a stateful mode does not reconstruct those untracked changes.
sequenceDiagram
participant FF as Freeform orchestrator
participant GS as History preparation
participant H as LLMCall harness
participant P as Provider transport
participant L as LLM
FF->>GS: Obtain current historical summary
alt summary is current
GS-->>FF: saved summary, no LLM call
else fewer than two events
GS-->>FF: empty summary
FF->>FF: Use opening narration as background
else summary is stale
GS->>H: HistorySummarizerCall.run(completed history)
H->>H: Build system prompt and user message
H->>P: chat(messages, summary JSON schema)
P->>L: Structured-output request
L-->>P: summary JSON
P-->>H: response, provider, model
H->>H: Log, parse JSON, validate summary
H-->>GS: LLMResult with validated summary
GS-->>FF: summary
end
FF->>H: FreeformNarratorCall.run(action and background)
H->>H: Build system prompt and user message
H->>P: chat(messages, no schema)
P->>L: Plain-text narration request
L-->>P: story prose
P-->>H: response, provider, model
H->>H: Log, strip prose, validate non-empty narration
H-->>FF: LLMResult with narration
Note over FF,L: Provider or validation failures propagate without returning a successful turn.
Note over FF,L: No manager calls, comparator calls, or tools in Freeform.
Direct Mode
Direct extends Freeform’s storytelling with explicit world-state updates. One narrator LLM receives the action, continuity summary, player ID, and a projection of the tracked state, then returns structured JSON containing narration, the complete updated world, action success, and game-over status.
The harness validates the response, and conventional code checks world references before applying it to a draft. There are no specialist managers, semantic comparator calls, or incremental tool operations. This keeps orchestration simple but asks one model to maintain entity identity and world consistency in a single response. Managed refines this approach by separating interpretation, reconciliation, and narration.
sequenceDiagram
participant D as Direct orchestrator
participant GS as History preparation
participant H as LLMCall harness
participant P as Provider transport
participant L as LLM
D->>GS: Obtain current historical summary
opt Missing or stale summary with at least two events
GS->>H: HistorySummarizerCall.run(completed history)
H->>H: Build system prompt and user message
H->>P: chat(messages, summary JSON schema)
P->>L: Structured-output request
L-->>P: summary JSON
P-->>H: response, provider, model
H->>H: Log, parse JSON, validate summary
H-->>GS: LLMResult with validated summary
end
GS-->>D: Current summary or empty initial background
D->>H: DirectNarratorCall.run(action, summary, player ID, projected world)
H->>H: Build system prompt and user message
H->>P: chat(messages, direct narrator JSON schema)
P->>L: Decide outcome and return the complete updated world
L-->>P: narration, updated world, action success, game over
P-->>H: response, provider, model
H->>H: Log, parse JSON, validate response
H-->>D: LLMResult with narration and world
D->>D: Validate world references and apply to draft
Note over D,L: No managers, comparator calls, or native tools.
Note over D,L: Provider or validation failures prevent a successful turn.
Managed Mode
Managed refines Direct’s single-call world updates by splitting interpretation across specialist LLM calls. Inventory, movement intent, and NPC managers run in parallel, then conventional code reconciles their proposals into a draft world. When identity is uncertain, the comparator receives an eligible candidate list and resolves it to an existing entity, a new identity, or an ambiguity. A separate movement resolver handles destinations that cannot be determined through deterministic shortcuts.
The narrator runs last and describes the reconciled outcomes rather than choosing world changes itself as it does in Direct. Unlike Tool Using, the LLM does not choose which functions to invoke: the engine owns the workflow and interprets structured JSON responses. This provides explicit state constraints and specialist reasoning, at the cost of more LLM calls per turn.
sequenceDiagram
participant M as Managed orchestrator
participant GS as History preparation
participant H as LLMCall harness
participant P as Provider transport
participant L as LLM
M->>GS: Obtain current historical summary
opt Missing or stale summary with at least two events
GS->>H: HistorySummarizerCall.run(completed history)
H->>P: chat(messages, summary JSON schema)
P->>L: Structured-output request
L-->>P: summary JSON
P-->>H: response, provider, model
H-->>GS: Parsed and validated summary
end
GS-->>M: Current summary or empty initial background
par Inventory interpretation
M->>H: InventoryManagerCall.run(action, summary, local state)
H->>P: chat(messages, inventory JSON schema)
P->>L: Identify item interactions
L-->>P: item proposals
P-->>H: response
H-->>M: Validated inventory result
and Movement intent
M->>H: MapManagerCall.run(action only)
H->>P: chat(messages, movement intent JSON schema)
P->>L: Interpret movement intent
L-->>P: movement intent
P-->>H: response
H-->>M: Validated movement intent
and NPC interpretation
M->>H: NPCManagerCall.run(action, summary, local NPCs)
H->>P: chat(messages, NPC JSON schema)
P->>L: Identify NPC interactions
L-->>P: NPC proposals
P-->>H: response
H-->>M: Validated NPC result
end
M->>M: Reconcile inventory proposals
opt Identity requires semantic resolution
M->>H: ComparatorCall.run(proposal, eligible candidates, context)
H->>P: chat(messages, comparator JSON schema)
P->>L: Resolve entity identity against candidate list
L-->>P: existing, new, or ambiguous
P-->>H: response
H-->>M: Validated identity outcome
end
opt Movement requires contextual resolution
M->>H: MovementResolverCall.run(intent and location context)
H->>P: chat(messages, destination JSON schema)
P->>L: Resolve destination or clarification
L-->>P: destination result
P-->>H: response
H-->>M: Validated destination result
end
M->>M: Reconcile destination and NPC proposals
Note over M,H: Reconciliation may make further candidate-list comparator calls.
Note over M,H: Exact identity matches and deterministic movement shortcuts avoid LLM calls.
M->>H: NarratorCall.run(action, summary, reconciled outcomes)
H->>P: chat(messages, narrator JSON schema)
P->>L: Narrate authoritative outcomes
L-->>P: narration, action success, game over
P-->>H: response, provider, model
H-->>M: Validated narration result
Note over M,H: Each harness call builds messages, logs, parses JSON, and validates the result.
Note over M,L: No native tools are offered to the LLM in Managed mode.
Tool Using Mode
Tool Using lets a game-master LLM choose the sequence of world changes. It receives the action, continuity summary, and current scene alongside native tool schemas for moving the player and creating, moving, acquiring, dropping, or destroying entities. Each request offers all currently applicable tools together, with eligible IDs constrained by the draft state.
Conventional tool functions validate and apply changes, invoking the semantic comparator when creation needs identity resolution. Tool results and the updated scene go back into the conversation, allowing further LLM rounds before final plain-text narration. Unlike Managed’s fixed pipeline or Direct’s whole-world replacement, the model controls incremental operations. Rejected tools return errors without changing the draft, while provider failures or conversation-limit failures abort the turn.
sequenceDiagram
participant T as Tool turn orchestrator
participant GS as History preparation
participant H as LLMCall harness
participant GM as GameMasterCall
participant W as WorldTools executor
participant P as Provider transport
participant L as LLM
T->>GS: Obtain current historical summary
opt Missing or stale summary with at least two events
GS->>H: HistorySummarizerCall.run(completed history)
H->>P: chat(messages, summary JSON schema)
P->>L: Structured-output request
L-->>P: summary JSON
P-->>H: response
H-->>GS: Parsed and validated summary
end
GS-->>T: Current summary or empty initial background
T->>GM: Build system prompt from harness definitions
T->>T: Start conversation with action, summary, and current scene
loop Until final prose or conversation limit
T->>W: Select state-applicable tool schemas
W-->>T: Tools with eligible entity IDs
T->>GM: respond(conversation, offered tools)
GM->>P: chat(messages, native tools)
P->>L: Game master request with all applicable tools
L-->>P: Assistant tool calls or final prose
P-->>GM: Normalized assistant message, provider, model
GM-->>T: Logged response
alt Assistant requests tools
loop Each requested tool in order
T->>W: execute(tool call)
W->>W: Validate arguments and state constraints
opt Creation requires semantic identity resolution
W->>H: ComparatorCall.run(proposal, candidates, context)
H->>P: chat(messages, comparator JSON schema)
P->>L: Resolve existing entity or new identity
L-->>P: existing, new, or ambiguous
P-->>H: response
H-->>W: Parsed and validated identity outcome
end
W->>W: Apply accepted change or roll back rejected tool
W-->>T: Tool result or error with authoritative scene
T->>T: Append tool result to conversation
end
T->>T: Append execution checkpoint for next LLM round
else Assistant returns final prose
T->>T: Validate non-empty story text and finish loop
end
end
Note over GM,H: GameMasterCall reuses prompt loading but uses respond instead of LLMCall.run.
Note over T,L: Tool calls and nested comparator calls are logged in execution order.
Note over T,L: Provider failures or exhausted limits abort the draft turn.
After a successful turn in any mode, separate history preparation can run
HistorySummarizerCall through the same harness and provider transport. It does
not delay returning the story. The next turn reuses the prepared summary when
current, or regenerates it before proceeding.
↩gemma4turned out to be best suited
