Tracing Agent Memory Through Changing Source Documents

An agent can remember a policy accurately and still apply an outdated rule. When the source changes, builders need to know which remembered conclusions depended on the changed passage.

In Building Evidence-Grounded Agent Memory with OpenWiki and Graphs, LangChain's Eugene Liu describes compiling documents into a source-linked wiki. Claims retain references to source lines and hashes, fingerprints used to detect changes. A changed fingerprint flags a claim for model review; it does not determine whether the claim is now false.

The team adds a graph connecting claims to passages and recording relationships such as one document superseding another. In its synthetic insurance library, reported accuracy on questions spanning documents increased from 0.52 with the wiki to 0.66 with the graph added. Simple lookups had shown little benefit from the wiki alone. Reported answering costs exclude building and maintaining the memory, and the article does not identify the evaluated model or grader sufficiently for an exact reproduction.

Follow the rule through its revisions

Consider a hypothetical manufacturer's purchasing agent checking which inspection certificate a supplier must provide. Its memory summarizes an older purchasing manual. A later amendment changes the requirement for one class of component while leaving the rest of the manual in force.

A link to the manual's title gives the agent a starting point. Links to the particular requirement, its amendment and their relationship give it a way to inspect which version applies to this purchase.

The distinction matters when documents change only part of an earlier rule. Treating an amendment as replacing the entire manual could discard requirements that remain applicable. Repeatedly citing the old passage could produce a consistent but outdated answer.

Our memory-provenance note asks where remembered advice came from. This account adds a way to follow dependencies between a conclusion and changing source material.

Check the conclusion as well as the links

The graph still depends on the accuracy of the citations from which it was built. A working link establishes where a passage is; it does not establish that the passage supports the conclusion or applies to the current supplier.

For WTK's catalog packages compiled into target runtimes, we would want a memory-informed decision to identify the supporting passages, relevant revisions and reason the rule applies. Our memory-use note separates useful context from evidence that can justify a decision.

People can define the accepted sources and purchasing criteria while routine checks proceed automatically. A source change would direct review toward the conclusions it affects, with unresolved conflicts visible before they determine a purchasing decision.

Source: Eugene Liu, Building Evidence-Grounded Agent Memory with OpenWiki and Graphs, LangChain, October 28, 2026. The manufacturing example and WTK interpretation are ours.