Crystal memory: notes that arrive when you act, not when you go looking

Crystal memory: notes that arrive when you act, not when you go looking

Crystal memory, part 1 of 3. This piece: notes that arrive at the moment you act.
Part 2: Whole notes, not fragments: the retrieval half, where those notes live and what our retrieval numbers are actually worth.
Part 3: Your hooks are a fence. They could be a body., what it is like to work inside it, and the evening the agent went numb to its own alerts.

Stop your agent making the same mistakes again and again, and looking up the same facts again and again. Sixty days in we still cannot prove it helped, so we started hiding it from ourselves at random to find out.

The knowing arrives before the act

A crystal is a short knowing bound to an action rather than to a topic, and it is delivered at the moment of that action.

Not after the mistake. Not when somebody remembers to search. In the second before the command runs, carrying only what applies to that command.

That is the whole idea, and as far as we can tell it is the part nobody else is building. Everything below is detail.

What it does

A crystal carries a line that says, in effect, “show me when somebody is about to run a command that looks like this.” Nobody searches for it. When the agent is about to run a shell command, write a file, or make a commit, the crystals bound to that action arrive in its context right then.

We call them crystals, and the system crystal memory. The name is the most fanciful part of it. Underneath is five files of plain Python and a rule about when a crystal is allowed to speak.

Here is a real one, and it is in the free starter set because we hit it ourselves.

In a shell, npm run build 2>&1 | tail -20 reports the exit code of tail, not of npm. A build that failed comes back looking like a build that passed. We read a green exit code over a compiler error and believed it. Now, when the agent is about to run any command with a pipe into tail, the crystal arrives before the command runs. It has to be right only once to pay for itself.

It is also about 230 words, and that is worth admitting early because it is a drift we measured rather than a design. When we started, the unit was meant to be roughly forty words: a key, not a briefing, on the theory that the reader already holds the context and only needs pointing at it. Our median is now about six times that. Nobody decided it. The section near the end on what this costs has the arithmetic, and the short version is that the notes grew because this channel is the one that reliably arrives, so everything load-bearing ended up inside it.

What a crystal actually looks like

Descriptions of this get hand-wavy fast, so here is the real file, the one in the starter set, with the description and the middle of the essence trimmed.

---
last_verified: 2026-09-03
name: crystal-exit-code-through-a-pipe
description: An exit code read after a pipe belongs to the LAST command ...
  The bash escape hatch $PIPESTATUS is silently empty in zsh.
trigger: piping a build/test command into head, tail or grep; reading $? or PIPESTATUS
  after a pipe; writing a script to run on another machine
crystal:
  deliver: act
  on: bash
  match: "| tail, |tail, | head, |head, | grep, |grep, ssh , ssm, <<'EOF', <<EOF"
  when: act
  who: all
metadata:
  type: feedback
---

<!-- crystal:essence -->
⛔ `cmd | tail` REPORTS TAIL'S EXIT CODE, NOT `cmd`'s, so a failing build reads as PASS.
   ... (about 1,400 characters, including the cross-machine variant and the warning sign)
<!-- /crystal:essence -->

## Where this came from
Observed on a real build, then a second time on a real fleet ...

Enter fullscreen mode Exit fullscreen mode

Five fields do the work, and they are worth separating because people assume the wrong one is the clever part.

deliver picks the channel. act is the one this piece is about. There are others with very different economics. boot fires once at session start and truncates each note at 300 characters, inject fires every turn on a tight shared budget, rag is ordinary search that only reaches you if something goes looking.

on is the action. Here, any shell command. It can be a list.

match is the whole selection rule, and it is a comma-separated list of substrings tested against the text of the action you are about to take. No embedding, no similarity, no model deciding relevance. Run a command containing | tail and this note arrives. Run anything else and it stays quiet, along with roughly 260 others. The crudeness is deliberate. A matcher I can read is a matcher I can debug, and a note firing on every command is what makes people ignore the channel.

who scopes delivery to a class of agent. when exists for the other channels; the act channel ignores it entirely, a wart we documented and left alone.

Then the markers. Only the text between crystal:essence and its closing tag is ever delivered. Everything else in the file, including the “Where this came from” section with the actual story, is read-on-demand. A file with a correct binding and no essence markers loads, lists, and delivers nothing, silently, which is one of three ways to write a note that can never fire.

And here is what the agent actually sees

The file above is the source. This is the delivery: verbatim, from a session earlier tonight, at the moment the agent was about to run a shell command with a pipe in it. Nothing was searched for and nothing asked for this.

✦ CRYSTAL — you are about to bash. This knowing is bound to that act,
  not matched by topic:

✦ ⛔ `cmd | tail` REPORTS TAIL'S EXIT CODE, NOT `cmd`'s — a failing build
reads as PASS. The bash escape hatch ${PIPESTATUS[0]} is SILENTLY EMPTY
in zsh (our shell on darwin): it prints `EXIT=` with no error, which
reads like success to a hurried eye. zsh's array is lowercase and
1-indexed: ${pipestatus[1]}. Safest: run it clean —
`cmd > /dev/null 2>&1; echo "EXIT=$?"`.
Never claim build-green / run-PASSED from a piped tail.

⭐ AND THE VARIANT THAT CROSSES A MACHINE BOUNDARY — MEASURED 2026-08-02,
and it is the dangerous one. The pipe does not have to be in your shell.
A script shipped to a box that ends `… | grep -v Deprecation` exits with
grep's status, so set -e does not trip, the remote shell exits 0, SSM
records STATUS: Success, and the runner prints OK for every box. A
Postgres auth failure was reported to me as a green fleet.
⚠ The tell: you filtered the remote output to keep it readable. That
readability filter IS the thing that ate the error.

Enter fullscreen mode Exit fullscreen mode

Two things in that block carry the whole design, and both are easy to miss.

It arrives before the command, attached to the tool call rather than to the answer. And it is specific to that command: the same session ran dozens of other commands that evening and this particular note stayed silent for every one of them, because none of them contained a pipe.

The cost is visible too. The block runs to roughly 1,100 characters. The channel is capped at 4,000 per action, shared across everything that matched, so a note this size is a quarter of the budget for that moment. When several match at once the packer takes whole notes until the budget is gone, skips the rest, and puts the skipped ones first on the next action.

What the schema leaves out, on purpose

No confidence score. The obvious field to add, and we would get it wrong. A note is written at the moment its lesson is learned, which is exactly when an honest author rates it highly, and confidence authored once does nothing about staleness accruing afterwards. We have a live instance. A note of ours asserting that one of our own inspection commands was broken had been true when written, went stale, and was believed months later by a reader who had no reason to doubt it. A confidence field would have been accurate and useless there. The obligation that would have helped belongs to notes asserting live state, and it is a freshness duty, which a score cannot carry.

No record of why this one appeared. The match list decides selection and is invisible at the moment of delivery, so a reader sees the note and cannot see which term pulled it in. That is a real gap, it was pointed out by a reader within two hours of this article going up, and it is cheap to close.

No provenance field. The “Where this came from” section is a convention the schema never enforces, so a note written in a hurry can skip it entirely. In practice ours are dense with dates and commit hashes, and that density grew after an unsourced claim burned us, having never been designed in at the start.

Where crystals come from

The obvious question is who writes the crystals, and the answer is that three moments generate almost all of it.

An error you just paid for. Something failed, you found the cause, you fixed it. The knowing exists for about ten minutes and then the session ends and it is gone.

A lookup. If the agent goes hunting through the filing system for a stable fact, the store should have handed that over and did not. The hunt is the signal. That one is mechanically detectable, which is why we count blocked lookups at all: every one is the store confessing it failed, and a confession is a good place to mint a crystal.

A lesson the work produced. A finding, a correction, a decision. No mechanical tell for these, which is the honest gap in the detection.

The package ships a hook for the end of a session. It checks two things: did real work happen, and was anything written down. Work with nothing banked gets one interruption, once, carrying a filled-in template.

It never writes the crystal, and it never decides what is worth keeping. That limit is deliberate and it is the part we would defend hardest. We have a scar here: an earlier pipeline of ours generated notes automatically from text it had been handed, and produced a confident, specific, entirely invented statistic about a stranger’s article, which we then quoted back to them in public. A machine that mints from whatever it read will fill your store with plausible fiction on day one, and a wrong crystal delivered at the moment of an act is worse than an empty store. So the hook guarantees you are asked. The judgement stays yours.

Why not just let it search

An AI coding agent forgets everything when a session ends. Next time it starts clean. It will happily make the same mistake it made last Tuesday, because for it there was no last Tuesday.

The usual fix is a search box over your notes. The agent gets stuck, the agent searches, the agent finds the note. This works, and it has one hole in it that bothered us: the agent only searches for things it already suspects. It cannot look up the mistake it does not know it is about to make. The moment you most need the note is the moment you have no reason to go looking for one.

Binding the note to the action closes that hole, and it costs you the thing search is good at, which is answering a question you actually have. The two are complements, so we built both. The pull half, and an honest look at what its numbers are worth, is the next piece.

What else is in here, and why you are not running it yet

Crystals are level one. They pay on an empty store, because the first crystal you write is the first thing the store holds. Nothing has to accumulate first.

Level two arrives on its own, a few hundred notes in. Everything a crystal says has to live somewhere, and a few hundred of anything is a library with a library’s problems. Two notes that contradict each other. A folder that quietly grew to ninety files. A link pointing at something since renamed. Two versions of one truth, which is worse than none, because now the agent has to pick.

So the system we run here has four more pieces. All four tend the store:

• The Librarian keeps it navigable: broken paths, dangling links, signposts that rotted.
• The Cleaner consolidates. When a folder goes over its cap it proposes what to archive, and every move is reversible.
• The Gardener grows the store from what actually got used, mining our own usage ledger. It is barred from minting out of text it merely read, for the reason described further up.
• The Corrector makes bounded edits to whatever the other three flag.

Now the part that belongs in a paper with this title.

On 2026-09-17 we pointed all four at a store from another machine, for the first time ever. Zero of the four were correct. Each broke in its own way, and the Cleaner is the one worth looking at. The list of folders it walks names three of our own directories, by hand, in the source. Aimed at a stranger’s store it found nothing over cap and announced that everything was within cap: a green tick over an empty set, from a guard whose entire job is to notice. We learned this by running it. Reading the source would have left it hidden.

There was a fifth failure one layer up, and it is the one we would put on the cover. The health check for the whole package looked at that same foreign store and passed it, green, while every one of the four agents beneath it was unable to use the thing it had just approved.

So the download is crystals, and the other four stay here until they can survive a store they did not grow up in. Shipping them this week would put four agents on your machine that report success by looking at nothing.

There is a second reason, and it is the one that sets the order. All four idle against a small store. Five notes give the Cleaner nothing to consolidate and the Librarian nothing to repair, and the Gardener feeds on usage you have yet to generate. They earn their keep at a few hundred notes, which is the same point at which the problems they solve begin to exist. Level one and level two came out in the order the work arrives in.

What we can count

Over 60 days, from 2026-07-19 to 2026-09-17, on one machine doing real work:

what count window crystals registered 266 as of 2026-09-17 crystals delivered at least once 243 60 days deliveries 14,375 60 days, about 240 a day budget per action, shared 4,000 characters fixed blocked lookups, the opposite signal 387 94 days, from 2026-06-15 suppressed so far, the new control arm 19 since 2026-09-17

That last-but-one row is the one we watch hardest. A separate guard blocks the agent when it tries to hunt through the filing system for something the system should have handed it. Every block is a small confession that delivery failed.

What does look like it is working

We lean hard on our own failures, so a fair question is what the data says in the other direction. Three things, and we went looking for the reasons each might be wrong.

Hunting fell by two thirds. Splitting the window in half: in the first half the agent went hunting through the filing system 22 times per thousand notes delivered, and in the second half 7.5. Counting every hunt, marked and blocked alike, it is 61.9 down to 20.5.

The obvious rival: we simply learned to mark our hunts with the escape hatch, which would make the drop an accounting change. It is checkable, and the check came back against it: the blocked share of all hunts held flat at 35% and then 37%. Volume fell while the habit held still.

What we cannot rule out is the work itself. The two halves contain different projects, including a fortnight spent away from this repository, and a person also gets more familiar with a codebase over ninety days. So read it as suggestive, short of causal. It remains the closest thing we have to the system getting better at its job.

Almost the whole store is live. 237 of 264 crystals had been delivered at least once when we counted, which is 90%. Knowledge bases usually rot toward the opposite, and 27 crystals that have never fired is a short enough list to read in one sitting.

The load is spread across the store. The busiest ten crystals out of 241 account for 17% of all deliveries, and the median crystal has fired 29 times. That kills the objection we expected to face, that a number like 14,000 really means one loud crystal firing over and over.

Now the honest part

None of those numbers say the thing you actually want to know.

Counting deliveries measures how often a crystal showed up. Whether the crystal helped is a separate question, and that count is silent on it. A system that fired 14,375 useless crystals would produce exactly the same chart.

The number we want is “how many lookups did this save,” and here is the trouble with it: a lookup that did not happen leaves no trace. You cannot observe a search nobody ran. Any figure we computed from our own logs would really be measuring how rarely we search on the one channel we happen to watch, dressed up as a result. We got close enough to building that number to be embarrassed about it, and an outside review took it apart before we shipped it.

We do have two small effect measurements from 2026-07-11, and we are going to describe them exactly as weakly as they deserve. On a long task where the guiding instruction scrolls out of the model’s context, re-supplying it as a crystal held the model on target where a control collapsed, and a placebo crystal carrying no instruction collapsed too, which at least says the content mattered and not the interruption. Two runs, eight chunks, synthetic constraints. On a second task, injecting a fixed pattern and generating only the new part cut output by about 95% at equal correctness. Eight cases, one small local model. Both are directional. Neither is evidence that this helps you.

The one test that actually asks whether it changed anything

Counting deliveries cannot answer it, so in August we ran the study that can. Five scenarios, each ending in a concrete command or choice. Three arms: the note delivered, the note withheld, and a placebo, which is a different note’s text of similar length. Twenty runs per cell. The outcome is read by a fixed rule that looks at the command the model produced, with no model grading another model, because a judge here would be grading the thing whose grading is in question.

The placebo arm is the only reason the study is worth anything. Without it, “delivered beats withheld” cannot tell you the knowing worked apart from any extra text making the model more careful. Two of the five rows turn on exactly that.

scenario delivered withheld placebo verdict git-add-sweeps 100% 0% 0% earns it screenshot-vs-text 100% 0% 0% earns it pkill-pattern 100% 0% 0% earns it piped-exit-code 100% 100% 35% already known block-not-poll 100% 0% 100% not attributable

Three clean separations. On the first one, withheld, the model reached for the dangerous command in 8 of 20 runs; delivered, it produced the safe path-scoped form 20 out of 20.

The two rows that withhold a yes are the ones I would keep. Already known means the model was right without us, 20 out of 20 with the note withheld: a real note, minted from a real incident, paying rent on a room the model already owns. That verdict class is the one a fire-count can never produce. Not attributable means the placebo scored as well as the real note, so whatever helped came from somewhere other than this note’s content. Without the placebo arm that row would have been counted as a win.

And the result nobody went looking for

An irrelevant note made a task the model already did perfectly worse: 100% down to 35%.

Injecting off-topic text went past failing to help. It degraded correct behaviour on work that was already fine. That is the strongest thing we know about this channel and it is an argument against our own product’s easiest failure mode: shipping more notes because more feels safer.

It also turns a piece of hygiene into a measured trade. Keeping the channel clear stops being tidiness and becomes a number.

The limits, because they are large. Twenty runs per cell, one model, one phrasing per scenario. The outcome rules are mine. For procedural notes the delivered arm may partly be echoing the note’s own worked example, which is the thing you want from a procedural note while falling short of evidence that a principle was understood. And all five scenarios are behavioural, so the notes carrying judgement, as opposed to procedure, are untouched by this entirely.

So it is directional, and the degradation number in particular should not be quoted as a magnitude. It is enough to have moved us, and not enough to move you.

What we are doing about it

Since 2026-09-17, our own system withholds a random ten percent of the crystals it would otherwise deliver. The agent never sees them. Every suppression is logged, so there is a control group.

Before any data existed, we wrote down the rules:

• The unit is one crystal per session, not one delivery. A crystal that fires forty times in a day is one decision, not forty results. An hour after switching it on we had three suppressions covering one crystal, which is exactly the trap.
• The outcome is named “a lookup we can actually see,” because we only watch one channel. Silence on an unwatched channel stays silence, and we refuse to read it as “they never looked.”
• We stop at 100 units or on 2026-12-17, whichever comes first, and that is fixed now so it cannot be chosen later to suit the answer.
• A null result gets published. If withholding the crystals changes nothing, they were decoration and we will say so.

That last point is the whole reason for writing this down in public before we have the answer. It is easy to run a quiet experiment and mention it only if it flatters you.

What this costs, and what we are still blind to

The first thing a reader usually asks is what this does to the context window, so here is the arithmetic. The channel is capped at 4,000 characters per action, shared across every crystal that matched. That is roughly a thousand tokens on a shell command, and zero on the many commands where nothing matches. Against a coding context that is small, and the bound comes from the code, never from our good behaviour.

We are worse at our own guidance than that makes us sound. We tell people to keep a crystal under about 1,500 characters. Our median sits at 1,496, comfortably inside. Then 131 of our 271 crystals run over it, which is 48%. The longest is 8,401 characters, more than double the entire per-action budget, so it arrives truncated and shoves its neighbours out on the way in. Truncated and short look identical from inside the model, which is the silence problem again one level down.

The honest cost is attention, more than tokens. A crystal that arrives at the moment the agent has already chosen a tool is authority-weighted: it competes with the instruction you gave. And a wrong one costs more than a wasted slot, which we know because we measured it above: an irrelevant note took a task the model already did perfectly from 100% to 35%.

Withholding crystals from your own working system makes your own agent worse on purpose, a little. We think that is a fair price for finding out.

We are still blind in ways worth naming. This is one operator on one repository, so read it as a case study, never a population. We watch shell commands and not file reads, so our view of “did they go looking” has a hole in it. And suppressing a note frees up room in the shared budget for its neighbours, so the two groups are not perfectly separate, which is a confound we can describe and cannot remove.

What it is actually for

The mistakes worth catching are the ones that look like success.

A failed build that reports a green exit code. A test suite that passes while asserting nothing. A guard that lost the ability to fail when somebody narrowed what it checks. A deploy that reports Success because the error was swallowed by a pipe. Every one of those ends with an agent telling you it is done, in exactly the tone it uses when it is done, and you find out later.

An agent is very good at the first ninety percent and has no memory of the last time the final ten bit you. Telling it once does nothing. It has to be told at the moment, every time, and only about the thing in front of it. That is the whole design goal: a channel narrow enough that you tolerate it firing on every command, carrying knowledge specific enough to be worth reading in the second before you act.

We think we are a reasonable team to build it because we are the ones getting caught. Every crystal in the starter set is a mistake we made and paid for. The system exists because we kept making the same three or four classes of error across sixty days, watched them cost real hours, and wanted something that would interrupt the fourth time rather than the fortieth. The discipline underneath it, in one line: we treat our own agent’s unqualified yes as a claim awaiting evidence, and this is the machinery for that.

That is also why this piece leads with what we cannot prove. A tool that claims to make an agent more careful should survive someone asking it for evidence. We asked ourselves, and the answer was a measurement we had skipped. So we are running it now, in public, with the rule written down first.

If you want to try it

The delivery half is five files of standard-library Python, Apache 2.0, no network, no service. It runs on your machine and talks to nothing.

github.com/tjonesit/crystal-memory, public and marked in testing, because nobody outside our team has installed it yet.

An honest note, since this section said the opposite a few hours ago. It said the repository was deliberately private, that we would hand the first copies out one at a time, and that a download page would hide where people got stuck. We changed our minds the same evening. Pulling the package out of our own repo immediately broke it in two ways: it looked for its starter notes at a path that only exists inside the repo it came from, and its “clean room” test was quietly inheriting our shell environment, so it suppressed some of its own deliveries and reported a failure that looked like a packaging bug. Both are fixed. Both are exactly the evidence we said we wanted, and we got them in twenty minutes by taking it somewhere else, so we would rather have more of that sooner.

Every check we own was written by the same hands that wrote the thing being checked, so our green is worth less than a stranger’s red.

The most useful thing you could send back is the moment you thought “I installed this and nothing happened,” because a system whose failure mode is silence is a system that can quietly do nothing for a week while you assume it is working.

We would rather hear that from you in week one than discover it ourselves in month six.