What moved LoCoMo: the claim layer — and where the sheaf comes in
How donto's recent substrate work changed the LoCoMo agent-memory benchmark, reported honestly — including what it did not do yet.
LoCoMo asks an agent questions about long multi-session conversations. The standard way to win it is episodic stuffing: dump the relevant dialogue turns into the reader's context and let a strong model read them. That works (≈84% here) but it is token-heavy and it doesn't test anything donto-specific.
donto's thesis is different: extract an evidence-anchored claim layer from the conversation, then answer from claims — a fraction of the tokens, and the substrate's actual job. The question this report answers: did the recent substrate program move that claims-only number, and by what mechanism?
The result
Same 4 LoCoMo categories, same reader and judge (holo3.1 on Hyades), three retrieval strategies over the same conversations:
| config | recall | n | accuracy | median ctx | multi-hop | temporal | open-dom | single-hop |
|---|---|---|---|---|---|---|---|---|
| Claims-only lexical (C) | FTS over triples | 160 | 15.6% | 1,435 tok | 10% | 18% | 8% | 28% |
| Claims-only semantic (C-sem) | bge-small ANN over claims | 200 | 37.5% | 913 tok | 20% | 72% | 16% | 42% |
| Episodic baseline | dialogue turns | 258 | 83.7% | 19,029 tok | 74% | 90% | 47% | 93% |
Two things stand out, and both are donto's whole point:
- Semantic claim recall is 2.4× the lexical claims-only baseline (37.5% vs 15.6%) — at fewer tokens (913 vs 1,435). Recall, not the reader, was the bottleneck; the answer-bearing claim usually existed but lexical top-K didn't surface it.
- Claims-only answers at ~1/21 the context of episodic (913 vs 19,029 tokens) for ~45% of its accuracy. That ratio — not the raw score — is the scaling story: donto stays cheap where episodic stuffing blows the context budget on
_s/_m.
What actually moved the needle
Not the sheaf contradiction-math. Being precise about mechanism:
- The claim-embedding layer. The lexical→semantic jump is entirely the embedded claim layer doing its job. The recent program took claim-embedding coverage to 100% (every
ctx:claims/locomoclaim has a bge-small vector indonto_claim_embedding), so semantic ANN can reach any answer-bearing claim instead of only the lexically-obvious ones. With partial coverage, semantic recall silently misses the unembedded claims; at full coverage it runs at strength. That is the single change behind the +21.9 points. - The bitemporal claim layer. Temporal accuracy jumped 18% → 72% — the biggest category move. Each claim carries its session date (valid-time), so "when did X happen / what changed" questions are answerable from the triples alone. This is donto's bitemporality paying off directly, no episodic dialogue required.
Both are substrate properties — evidence-anchored, embedded, bitemporal claims — that the broader program hardened. The benchmark is the instrument that made the improvement legible, not a grade.
Where the sheaf actually comes in (and hasn't yet)
Honest accounting: the sheaf engine — H⁰/H¹, cocycles, contradiction pressure, the Stage-2 query-time reconciliation — is not in the LoCoMo recall path today. It did not produce the numbers above. The Stage-2 reconciliation ships dark and re-rank-neutral; the LoCoMo harness uses its own semantic claim recall, which the sheaf layer does not yet touch.
But it is the natural next lever, precisely on LoCoMo's weak spots:
- Knowledge-update / temporal questions hinge on which version of a fact is current. donto holds the superseded and the current claim side by side (paraconsistent). Contradiction-pressure-aware reranking — prefer the low-pressure, non-superseded claim, surface the conflict instead of averaging — is exactly what a sheaf reading of the recalled claim set provides. The Stage-2 reconciliation already computes per-claim pressure; wiring it into recall ordering is the experiment.
- Multi-hop (20%) needs a bridge entity the question never names. The cellular sheaf is built over the claim graph; a graph walk to the bridge is the same machinery that localizes cocycles.
So the precise statement is: the claim-layer work (embeddings + bitemporality) lifted LoCoMo; the sheaf contradiction-engine is the teed-up next step toward closing the remaining gap. We will measure it, not assert it.
The road to 100% (no shortcuts)
The honest gap analysis from C-sem's per-category numbers:
- temporal 72% — strong; bitemporal claims work. Push with contradiction-aware reranking.
- single-hop 42%, open-domain 16%, multi-hop 20% — the work. Open-domain often needs world knowledge the conversation never states (a ceiling for any memory system); multi-hop needs claim-graph bridging; single-hop needs deeper recall + better extraction coverage.
"No shortcuts" means: every point comes from a general donto capability — better extraction coverage, claim-graph multi-hop recall, contradiction-aware reranking, finer claim granularity — never a LoCoMo-specific cache or a prompt that games the judge. When a benchmark-specific fix is tempting, the rule is to find the substrate gap it exposes and fix that, so every consumer benefits. The curve continues in the live runs table.
Every run above is reproducible and browsable — drill into any one for the full reader→judge transcript. Reader/judge run on Hyades (holo3.1); claim extraction runs on the same gateway, serialized so a producer and a consumer never share it.