Store knowledge the way it changes
Decisions often move faster than the documents that record them. People fill that gap from memory. AI agents only have what they can read, so what they read should change when the decision does.
Ankur Garg · Co-founder and Head of AI, Pyrana · October 20, 2026 · 7 min read
Think about the last time your team changed how something gets done.
Where was that decided? And has the document, or whatever your AI agents read, caught up yet?
The people on that call will use the new rule on Monday. The knowledge article will say 50 for a while yet. So will the AI assistant that drafts replies for the support team, because the article is what it reads.
Decisions stack up. Documents catch up later.
I've been on both sides of this. Sometimes I made the call. Sometimes I was the one who was supposed to update the document. Either way, the rewrite sat on someone's list. It caught up when it caught up.
The gap is much older than AI. Safety researchers have studied it for a long time. Erik Hollnagel describes it as the difference between work as imagined and work as done, and Steven Shorrock writes that the written version is often "just not how things are really done."
We cover the gap from memory. Someone was on the call. Someone mentioned it in the hallway. For people, that mostly works.
It doesn't work for AI agents. Sometimes an agent can get a transcript of the call. Often it can't, because the call wasn't recorded, or couldn't be. Even when a transcript exists, often nothing routes it to the agents that need it, and the one decision is buried in an hour of everything else. Julie Mohr at Forrester wrote in May: "Documentation written for a human reader who interprets ambiguity is not sufficient input for an agent that executes on what it reads."
Transcripts have a second problem. Not everything said on a call is a decision. "We've been skipping the second review lately" describes how work is being done. It doesn't change what the policy is. The policy should stay what it is until someone with the authority to change it does.
The rule changed in a meeting. The document caught up days later, or months later, or never. The agent ran in between.
And once the document does catch up, you still have to ask: what did the agent do in the meantime?
The document is the wrong size
This isn't a find and replace. The policy needs a new exception for late orders, worded so it fits the approval and escalation rules around it. The same rule also sits in the knowledge article, a few saved replies and maybe a training deck, often with different owners. Then the policy goes through review, sign-off and a new version. Nobody wants to do that for one change.
So decisions wait. They stack up until there are enough of them to justify a revision. The backlog grows, the person who made the call moves on to the next one, and the revision ships weeks or months after the first decision. Sometimes it never ships. And when it does, the new version has the new rule. It usually doesn't say who decided, when it started, or why.
One rule changed. The whole document has to move.
Imagine knowledge the size of a decision
Now picture the same policy kept differently: as a set of Context Units, each holding one rule.
A Context Unit holds one claim that stands on its own, with what it needs attached: the exact words it came from, who decided it and when, why the rule exists, and where it applies. One unit is about one decision. (More on how Context Units work.)
The unit becomes the thing you maintain. It is what gets updated when a decision changes, what keeps the history, and what an agent points to when it acts. The document can still be updated, on its own schedule, and the unit stays linked to it. You just no longer have to wait for it.
Now run the support call again, the way it could work.
After the call, the Agentic App the support team works in picks up what the team already produces: the meeting transcript, the follow-up email, the action item someone logged. It spots the decision and drafts a change to one unit: late-order credits up to 100 USD, starting Monday. It doesn't change anything on its own. The draft goes to the person who owns that rule, the head of support.
She isn't reviewing it because software flagged it. She owns the rule, so changes to it go through her, the same as a policy edit would. The difference is the size. She sees the old rule, the new one and the line from the call it came from, and she approves it, edits it or sends it back. Once she approves, the reps and the AI agents who look it up work from the new rule on Monday. Nobody had to reopen the document.
Most teams don't have this step today. It's one of the pieces we build into an Agentic App at Pyrana.
If your business runs on formal change control, this is still change control. The approval just happens at the size of the decision instead of the size of the document.
Until the document catches up, the two disagree, and that should be visible. The unit links back to the document and flags that it is behind, and the approved change can be drafted into the document for its owner to publish.
Knowledge changes one decision at a time, so store it that way.
Keep the old version
When a unit changes, the old one is not edited or deleted. A new unit replaces it, points to the one it replaced, and both keep their dates.
Engineers who build ledgers have worked this way for a long time. Pat Helland, who has spent four decades designing database and transaction systems, compares it to accounting: nothing in a ledger is erased, and a correction is a new entry. Knowledge for agents can work the same way.
So what did the agent do in the meantime? The new rule took effect Monday. If the AI assistant told a customer with a late order on Wednesday that a 75 USD credit needed a lead's approval, it was working from the old version, and the reply records which version it used. You can pull every late-order reply that relied on the old version after Monday and put them right.
It works the other way too. A reply the week before that sent a 75 USD late-order credit to a lead was correct at the time, and the history can show that.
Pinecone's team wrote in August that "a fluent answer grounded in the wrong version of a policy scores zero." Keeping every version, and recording which one each answer used, is how you find out.
What changes when knowledge works this way
| When knowledge lives in documents | When knowledge is the size of a decision | |
|---|---|---|
| When a decision reaches the agent | When the document is next revised | As soon as it takes effect |
| Who gets the change | Whoever was in the room | Everyone who reads that rule |
| Impact analysis of work done under the old rule | Hours of digging | Minutes: pull the results that used it |
| Explaining an agent's answer | Reconstructed after the fact | Recorded at the time, down to the version |
| Who changed a rule, and why | Spread across emails and meeting notes | An owner, a date and a source |
| What you can hand to agents | What you can check by hand | More, as the record builds trust |
A rule changes when someone decides, not when nobody uses it
There is a fair objection here. If updates are cheap, knowledge could get messy.
It will if anyone can make a change stick. A change should be easy to propose. Approving it should be a real decision, made by the person who owns that rule.
The same caution applies to systems that learn from use. Cognition's coding agent, Devin, runs a nightly "dreaming" session that tidies its memory, including removing "stale notes that no session has used." For a developer's notes, that makes sense. For company policy, it is the wrong signal. A rule nobody needed this month is still the rule.
What's still hard
Smaller pieces also mean more of them. Getting one unit right is manageable. Keeping thousands of them healthy is harder, and we are still working through it.
- Turning conversations into proposals. Decisions arrive in messy forms: a transcript, an email, a comment in passing. Spotting the one decision and drafting the right change takes judgment.
- Approval load. The people who own the rules, like a head of support, a finance controller or a policy owner, are busy. If every proposed change waits for one of them, proposals pile up.
- Staleness from outside. Some units go out of date without anyone inside the company deciding anything. A vendor changes its terms, a regulation moves, a system gets replaced.
Jaya Gupta at Foundation Capital described the open question as knowing "what precedents to remove and when."
Ideas we are exploring
- An exception queue. Changes that clearly match a decision the owner already signed off on go through with a notice. Rule owners only review the ones that need a human call.
- Background review that proposes instead of deciding. Andrej Karpathy suggests periodically asking a model to health-check a knowledge base and flag claims that newer sources have replaced. We think that is right, as long as the result is a proposal to the rule's owner.
- Staleness signals tied to sources, not usage. The decision log moved, the policy got a new version, or a related unit changed. Those are reasons to look again. Low usage is not.
What we believe
When we build Agentic Apps at Pyrana, we treat knowledge as part of the app, not a folder it happens to read. It has owners, versions and a history, the same as everything else the app depends on.
Try it on one rule your agents follow today:
- When did it last change, where was that decided, and how long did it take to reach what your agents read?
- If an agent acted on the old version in between, could you list every case by tomorrow morning?
- Who is allowed to change that rule, and would you know if someone did?
An agent acts on what it reads. Make sure what it reads changes when the decision does.
Sources
- Steven Shorrock, "The Varieties of Human Work," Humanistic Systems, December 5, 2016.
- Julie Mohr, Forrester, "Zendesk Relate 2026 Showed Why Agentic Customer Service Starts With Knowledge," May 26, 2026.
- Pat Helland, "Immutability Changes Everything," Communications of the ACM, January 2016.
- Gujral and Ragavan, Pinecone, "Nexus GA: It's the Knowledge, Not the Models," August 6, 2026.
- Cognition, "Memory and Dreaming," Devin documentation, October 2026.
- Jaya Gupta, interviewed by Ramp, "Context graphs: what capturing the reasoning behind decisions unlocks for finance," April 28, 2026.
- Andrej Karpathy, "llm-wiki," April 4, 2026.
More for business leaders
Bring your hardest process.
We will run it through admission, approval, and the record, and show you what comes out.