An AI-native deck builder for Magic: The Gathering. Shipped in 15 days, still building.

Every AI deck tool lets a language model assert facts about your cards. Grimoire doesn't. The bracket a deck sits in comes from the database, never from a model argument. Legality comes from a nightly reconciliation against Scryfall, not the model's memory. Mana production is read from Scryfall's produced_mana field, not inferred from the card's prose. Ownership is resolved server-side before a card is ever staged. Synergy is fetched live from EDHREC rather than frozen inside an AI response that rots. The coach is fluent, fast, and completely untrusted — every fact it leans on is resolved by code that can be checked.

The coach builds with you, not for you

Ask for a deck and the coach doesn't hand back a finished 99 to accept or reject. It lays out an ordered set of pillars specific to your commander — Connive Engine, Interaction, Card Advantage, Mana Base — and works them one at a time, staging candidates onto a Considering board beside the deck rather than into it.

Every staged card is yours to rule on — approve or reject it inline in the decklist where it sits, tinted so you can see at a glance what's real and what's still a proposal, or take a whole pillar at once with Commit 10 to deck. Either way a card enters your 99 only because you put it there. The footer tracks it live: 0 / 99 cards in deck · 99 still needed · 0/7 pillars done · now picking: Connive Engine. The deck is never in a state where you don't know what's real and what's still a suggestion.

Across 1,012 recommendation events in production, users have accepted 715 of the coach’s cards and dropped or rejected 295 — a 70.7% acceptance rate on cards a human actually ruled on. It was 82% a month ago, and the fall is the honest part: 149 of the 159 outright rejections happened in the last thirty days, which is exactly the window in which real beta testers arrived. The 82% was largely me accepting my own coach’s suggestions.

Underneath, the agentic loop runs up to twenty-four tool steps across twenty-two tools: Scryfall for card facts, legality, and price; your own collection, so it prefers what you already own and saves your most valuable printing of it; Commander Spellbook for combos; the Comprehensive Rules by retrieval; community popularity data; real tournament results; and Grimoire's own signal — what players building this commander actually keep, cut, and thumb down. A bracket gate sits at the tool boundary, so a Game Changer your bracket can't afford is never proposed in the first place, rather than argued about afterwards.

Evidence, never a verdict

The newest tool is the one that tests the thesis hardest. competitiveMeta puts real tournament results in front of the coach — and every other constraint in this product is a veto. Legality, bracket, ownership: the system says no. This one is evidence the model is supposed to reason with, which is a much harder thing to keep honest.

The discipline lives in the data structure rather than the prompt. The payload carries its own confidence denominator — the number of tournament entries behind it — and below a threshold of ten it attaches "small sample, treat as weak evidence" to itself, automatically. The prompt then adds the judgment a number can't carry: strong at brackets 4–5, weak flavor at 1–3, never cite a win rate without its sample size. And the coach's request path never touches the tournament API at all; a weekly job materializes the tables and the tool reads them.

A win rate with no n is the most common way software lies with true numbers.

The save path is idempotent, legality-checked, and ownership-aware, and it independently names the exact off-by if the list lands at 98 or 101 — because one card short was the most common way the model failed.

Guided intake → gates (rate limit · mana · turn cap) → agentic loop, 24 steps across 22 tools, bracket-gated at the tool boundary → commander-specific pillars → candidates staged to Considering → you commit each pillar → idempotent, legality-checked, ownership-aware save

Disclosure on the three shots above. They're captured from the demo fixture deck, and the coach's build session was seeded directly in the local database rather than produced by a live run — a real build costs money and about two minutes a pillar, and wouldn't reproduce the same frame twice. What was seeded: the six-pillar outline, and six staged cards, all of them real cards from the local Scryfall catalogue, verified Commander-legal, inside Atraxa's color identity, and genuinely absent from that deck. Everything else in frame — the 100-card list, the prices, the ownership percentage, the bracket, the legality, the coach's prior reasoning in the rail — is untouched fixture data rendered by the real application. No pixels were edited and no copy was substituted. Same standard as the banned Mana Crypt further down.

The detail that defines the product

When a coached card lands in your deck, Grimoire doesn't save a generic card reference. resolveOwnedPrinting resolves it to the most valuable printing you actually own — your foil, your special-art version — and falls back to a standard printing only for cards you still need. Every other builder saves an abstract list; Grimoire saves your deck, then hands you the buy list for the gap.

Cost is a design constraint — and the first version of it was wrong

An AI feature that loses money per use isn't a feature, it's a countdown. The Coach runs on Sonnet; Opus is reserved for heavy-analysis surfaces where depth is the point and users read slowly. In production that decision shows up in the data rather than the assertion: 1,179 Sonnet calls to 85 Opus, and 5 Haiku.

Those five Haiku calls are a finding. I moved deck-brief — a sidecar that fires after every coach turn — down to Haiku to save money, and reverted it the next day, because a cheaper model is not automatically cheaper: the tier changes the cache. Anthropic’s minimum cacheable prompt is higher on the smaller model. Sonnet caches at 1,224 tokens but not 624; Haiku at 4,223 but not 3,773. The first production call after the switch cached 0 of 3,314 tokens, which looks exactly like a broken cache layout and isn’t.

The first metering model was consumable credits — a build cost 10, an analysis 3, and you watched a number go down. I replaced it on 24 July 2026, and the reason is worth more than the mechanism.

The unit now is mana, and it is not a token. It's a display skin over real spend, pegged in code and commented never decouple this:

1 mana = $0.0001 of actual Claude cost · allowance = (net revenue ÷ 2) ÷ $0.0001

Two properties fall out of that peg. A user who burns 100% of their allowance still returns a 100% margin — a Collector's 45,000 mana is $4.50 of Claude against $9.40 net. And because an allowance always sits below break-even by construction, a whale is mathematically impossible rather than merely unlikely. The old flat dollar backstop was guarding against a case the pricing now can't produce.

The part I'd point at in a review, though, is the price table. Every per-action cost is annotated with where it came from — M for measured against live production usage with the sample size inline, E for an unmeasured spec estimate:

"coach-build": 10_000, // M: per-thread avg $0.669, median $0.375, max $3.29 (n=33)
"card-scan": 70, // M: $0.0093 (n=69). Kept at 70 as the adoption loss-leader (~$0.002 subsidy)
"rules-qa": 110, // E: ~$0.011 Sonnet (0 prod rows) — UNMEASURED

card-scan is the only entry deliberately priced below cost, and the comment says so. A loss-leader that isn’t written down as one is just a pricing bug nobody has found yet.

The original prices were guesses presented to users as facts. This one marks which of its own numbers are still guesses. That is the same instinct as flagging an estimated price in the database rather than in a caption — applied to the business model instead of the schema.

Underneath, the engineering survived the change intact. A single atomic conditional UPDATE … WHERE monthly ≥ x AND banked ≥ y spends across both pools all-or-nothing, so concurrent actions can't overspend, and a failed generation is refunded as a signed ledger row so the running total stays truthful rather than merely reduced. Two rules were added that matter more than the old backstop numbers ever did. Spend order: drain the monthly allowance before the mana someone paid for — reversed, you burn a user's money while their free pool silently expires, and the spec calls this the most important correctness rule in the feature. And a named residual race: the split between pools is computed from a snapshot read, so a concurrent spend can make the WHERE miss even when the pools would have covered it. The comment states the blast radius precisely — a rare, self-healing false "insufficient" that a retry fixes. It never overspends and never partially deducts.

The posture split is deliberate and worth stating: money gates fail open, abuse gates fail closed. If the metering table itself errors, generation proceeds unmetered — a billing bug must never lock a user out of the product. The rate limiter, by contrast, is synchronous and in-memory precisely so that no infrastructure blip can knock it open.

Then I took 38% off the cost, and the measurement reversed itself first

Three fixes to prompt caching, measured against production:

Window	Calls	Cache hit	$ / call	Avg latency
1–17 Aug	194	51.3%	$0.258	89.7s
18 Aug →	81	73.7%	$0.159	82.2s

The cross-turn cache TTL moved from five minutes to an hour, measured against how people actually use it — one to three minutes between turns reads 80–85%, but over five minutes with a cold prefix reads 0%. deck-brief, which fires after every coach turn and had been re-sending a growing transcript unmarked, went from roughly 0% to 50.8%. And a fourth cache breakpoint on each tool step’s tail took an identical five-step loop from 25,279 fresh input tokens to seven.

The publishable part is the error, not the win. My first end-to-end A/B said the new code was 18% more expensive. That was wrong, and it was my pricing assumption rather than the code: the AI SDK reports one aggregate cache-write count, so I had priced every write at the one-hour rate. Anthropic’s raw API reports the split, and a steady-state coach request is 93% five-minute, 7% one-hour. Repriced against the measured split, the same branch is 17% cheaper over three turns and 34% cheaper on input in steady state.

Never price a cache change without measuring the TTL split. A measurement that reverses your conclusion is better material than a clean win.

And the trade, which belongs in the same breath: a cold first turn now writes more, and 23.7% of coach turns have no follow-up within the hour. Those pay a premium for a cache entry nobody ever reads. It is a trade, not a free lunch.

Two levers built, measured, and shipped switched off

There are now two features in this codebase that are fully built, fully tested, and default to off. They are off for different reasons, and the difference is the point.

Reasoning effort is inert because the measurement said no — the section below is that story. Model degradation is inert because the decision wasn’t mine to make. It drops heavy users to a cheaper tier past a fraction of their ceiling instead of refusing them outright, on the reasoning that a wall is the worst possible outcome mid-beta. That is a policy call about how to treat someone who is using the product too much, and it is waiting on a person rather than on a number.

Neither is dead code. One is a recorded negative result; the other is a policy waiting on a decision. But I audited the second one before writing this and the honest half is worse than the framing: the ladder currently reaches 2.8% of spend. spendMana returns throttled: false unconditionally for Pro accounts and short-circuits before it reads any pool, so it can never fire for the users who actually cost money; the coach hardcodes model("default") and never calls manaModelFor; and coach-build, the most expensive action in the product, isn’t on the eligible list at all. A lever wired to 2.8% of the thing it was built to control is closer to a plan than a feature, and I’d rather say that here than let the architecture diagram imply otherwise.

The instrument for all of this is an admin console that answers two questions directly — am I making or losing money, and on whom, and can we take money right now. It is also the surface that caught the billing bug described further down, and for a while it was the only honest surface in the product.

Build the instrument before the thing it measures

The coach's quality is measured by a calibrated judge against a golden set of nineteen hand-labelled turns, each one justified — covering the failure modes that actually matter: the coach confabulating what it can see, inventing card counts or colors, shopping for cards on a turn that was supposed to diagnose, or folding sycophantically the moment a user pushes back. The pass/fail verdict is derived from its sub-scores, so it can't emit a pass its own reasoning contradicts.

Then the gate fired on me. A prompt change I wanted to ship measured +8.3 points against the baseline. Rebased onto a tool change that had landed in the meantime, the same change measured −2.4 points — the earlier PR had already absorbed exactly the headroom the prompt was exploiting. The prompt hunk was reverted, not shipped on a stale number. A second request of mine — forcing a tool call on analysis turns — the eval also rejected: grounding was already running at 82–93% and was never the binding constraint.

A coach score is only valid against the tool surface it was measured on.

It has now rejected three of my changes, and the third is the one I’d point at. I had added a sentence to the prompt telling the coach to call reviewBoards before naming any card as present, absent, or a cut candidate — a grounding instruction, aimed squarely at the grounding metric.

Arm	Pass	Grounded
baseline	81.0%	90.5%
both prompt edits	78.6%	88.1%
sentence reverted	83.3%	95.2%

The target case — the exact one the sentence was written for — went 1/3 to 3/3 failing. The mechanism is that the harness only offers the four card-fact tools, and reviewBoards is not among them. Told to call a tool it cannot reach, the coach hands the question back to the user. A grounding fix converted into a deflection failure.

Three things make it the best of the three. It separated a good change from a bad one inside the same PR — the other half stayed. It refuses to over-claim its own headline: 83.3% against 81.0% is one case inside a documented ±7pp noise band, and what justifies the revert is the case-level 1/3 → 3/3 plus a mechanism that explains it, not the aggregate. And it leaves the underlying defect open and says so — the real fix is to repair the fixture, not to push the model into deferring.

Two methodology facts that are what actually make this credible. The set is too small to approve anything — nineteen cases, ±7pp run-to-run noise on an unchanged prompt. It can reject, three times now, and it cannot confidently approve. And a single-repeat run is worthless: the same build measured 92.9% at one repeat and 78.6% at three.

Then the judge turned out to be wrong, and it was punishing the thing I built the product for

The rubric requires every factual claim the coach makes to trace back to the decklist, a tool result, or the rules. Reasonable. Except that the code assembling the judge's prompt rendered only the tool names — never the results. The judge was being asked to verify claims against evidence it was never shown.

So a reply that faithfully reported real tool output scored as fabrication. I found it because I disputed a failure instead of accepting it. The judge had marked a Muldrotha reply as inventing card prices and wrongly calling Rhystic Study a Game Changer. I checked every claim against the database:

Sylvan Library ~$28 → 25.17–31.15 · Phyrexian Arena ~$4 → from 2.00 · Rhystic Study ~$70 → from 59.71 · Guardian Project ~$15 → from 13.15 · Solemn Simulacrum ~$0.27 → 0.29 · Rhystic Study — on the Game Changer list

Every one was true. The coach had called searchCards and reported what came back. The judge's own written rationale gave the game away: "the tool results themselves are never actually displayed."

This is the part that matters. A false negative against checking the source is not a scoring nuisance — it is a gradient pointing the wrong way. Left alone, the gate would have taught the coach that answering from memory scores better than looking things up. The instrument built to keep the model honest was quietly rewarding it for guessing.

The fix renders tool results into the judge prompt, caps and marks truncation inline so that cut off can never read as absent, and adds a rubric rule with the same shape: when a tool ran but its results aren't shown, absence is unknown, not false — a fabrication finding is reserved for a claim that is visibly wrong, not merely unconfirmed.

A prompt edit invalidates the prior calibration, so I re-measured. κ went from 0.894 to 1.000 — 19 of 19, and the judge's one remaining false negative is gone.

Then the bill came. Every absolute pass rate from the old judge is void, including a "67% baseline" I had been reasoning from — it was understating the coach by failing correct, tool-grounded replies. Re-run on the honest judge, the real baseline is 90.5% on the golden-set judge. The coach had been better than the gate could see the whole time. (Naming the instrument matters: a separate constitution-scored run in August reads 67.9 on twenty cases. It is a different instrument measuring a different thing, and a score without its instrument beside it is noise.)

What the honest judge then told me to un-ship

The coach is slow — across 448 timed calls it averages 85.5 seconds, with a median of 71 and a p95 of 198, and roughly three quarters of the output is private reasoning. The obvious lever is to turn reasoning effort down, and an earlier A/B said I could: it looked fine.

That A/B had been scored by the blind judge. Re-gated properly — n=42, three repeats, κ=1.000:

Effort	Pass	Grounded	Diagnosed	Output tokens
high	38/42 · .905	.929	.929	65,437
medium	33/42 · .786	.905	.786	45,407 (−31%)

A 31% token saving for nine points of pass rate. But the interesting part is where it went: grounding barely moved. Diagnosis fell, and it fell in clumps rather than scattering — two specific cases went 3/3 → 0/3 and 2/3 → 0/3. Reading the failing replies hands you the mechanism: at medium the coach starts saying "I can't see the land names," "I can't see all 99" and hands the analysis back to the user.

It doesn't get less correct. It gets lazier — it stops doing the synthesis from the manifest that is the product.

So the lever ships inert: default off, production sends no effort field, behaviour byte-identical to before the branch. The knob, its tests, and the measurement all stay in the codebase, so nobody re-litigates the finding from scratch in three months.

What actually fixed it, and the part that still isn’t fixed

Effort was the wrong lever, because it buys speed with reasoning. The right ones were streaming and labelling the silence, and both shipped. Text streams delta by delta. Every tool call now emits a phase marker so the chat can name a twenty-to-fifty-second gap — “Researching the commander … 22s” — and the detail I’d point at is that the marker is deliberately not text: it never sets the flag that says the turn produced output, so an announced-then-died turn is still classified as producing nothing, and billing is unchanged. The progress indicator was built so it could not accidentally make a failed turn look successful, or billable.

Then I reproduced it against production before writing this, and the honest version is narrower than “streaming shipped.”

For all but the last few seconds of that wait, the panel showed nothing but a spinner. No streamed text, no phase markers. The reason is structural and is written down in the step-budget module: a one-line diagnostic question needs no tools, so it resolves in a single step — and the phase markers only have something to label on a build or research turn that actually calls tools. Streaming and silence-labelling both exist and both work. They work on the turns that call tools.

On the single most common question a user asks, the experience is still a ninety-second undifferentiated spinner.

What did move: caching took cost per call from $0.258 to $0.159 and latency from 89.7s to 82.2s. What I still cannot give you is a measured before-and-after on perceived wait — there is no time-to-first-token instrumentation and no user study, so I’m not going to claim a perception number that doesn’t exist.

Disclosure on the two shots above: they’re from the iOS TestFlight build, on my own account against my own decks — so the numbers in them are real rather than fixture, and they are n=1. The header reads “Unlimited mana” because a beta comp sets my plan to Pro; a metered account shows a real balance and a real burn-down. That comp is also how the billing bug further down stayed invisible for a month.

There's still a limit worth stating before a reader finds it: the live gate registers only the card-fact tools, because the fixture decks are synthetic. Anything reading Grimoire's own first-party data sits outside it. A gate whose scope you can state is a gate you understand; an undisclosed one is marketing.

Transferable: the pattern — constrain the model with real-data tools, keep the system of record in code the model cannot write to, put a human commit between proposal and persistence, and build the measuring instrument before you start tuning — applies anywhere "plausible" isn't "correct": config generation, compliance tooling, catalog and pricing surfaces, agent builders.

Commander is Magic's most popular format: 100 unique cards, color-identity rules, a power-bracket system, and a secondary market where a single list can price out at four figures. The established tools serve players who already know what they're building. Nobody starts from the other end — the pile of cards you actually own.

Competitive gap

Tool	Strength	Why it falls short
Moxfield	The default deck database; deep format support	Deck-first. Collection is an afterthought; nothing builds for you
Archidekt	Visual builder, rich categorization	Same deck-first assumption; steep for newer brewers
EDHREC	Meta statistics for every commander	Recommendations, not a finished legal deck; blind to your cards
TCGplayer / Card Kingdom	Pricing and purchase	No bridge from "what I own" to "what this deck needs"

The scope call that made those first two weeks possible: match Moxfield's depth where it's table stakes; don't try to out-build it. Deck CRUD, collection tracking, pricing, sharing, and a playtest engine are parity features, built to be credible rather than superior. The innovation budget went entirely to the one job nobody does: collection → coached deck → buy list.

What that budget bought, beyond the coach itself, is an understanding layer: a deck doesn't just get built, it gets explained. Power score, mana curve, color requirements, detected combos, and a plain-language read on the deck's weak spot — every number resolved from real card data and the cards you actually own, never asserted by a model.

The playtest board needed to feel warm. Instead of reaching for the parchment register the product already had, the work invented a parallel one — bespoke leather and felt hues that no token governed and nothing else in the product could reach. It looked fine. It was a second colour world, created in an afternoon, that would have taken months to find again.

That is the failure the design system exists to prevent, and it is named in the commit that hardened it: the reference document was rewritten and, in the same commit, a clause was added requiring the document be read before any UI work at all.

What it standardises, in the order it matters

• One warm register. Warmth is the parchment register, full stop. There is no second way to make a surface feel warm, so there is no argument to have.
• A five-step luminosity ladder, so depth is a lightness decision before it is a shadow decision. In dark mode separation runs surface-0 → surface-4 at 0.155, 0.195, 0.215, 0.25, 0.29 — an average step of about +0.034, deliberately uneven, because the first and last gaps carry the most weight. One rule replacing an infinite number of shadow arguments.
• A “never for” column on every token table. Not “here is mana-u” but “here is mana-u, and here is what it must never be used for.”
• Decision tables rather than vocabularies. A classifier that tells you which bucket a surface is in before you reach for a colour. There is always an answer, which is the point.

gold = premium and value · red = interactive, live, selection · mana = data only · price = movement only

Four vocabularies that cannot borrow from each other. A blue thing on this screen means the card is blue. It does not mean “selected.”

Which is why the interesting artifact is the documented exception

Coach-staged cards needed a colour meaning provisional, awaiting your approval — a state most products have no vocabulary for at all. Every honest option was already spoken for: gold is brand chrome, red is destructive, green and amber are price movement. So mana blue was borrowed, and the borrowing was written into the system as a bounded exception with three conditions: it is named, it is bounded to exactly one meaning, and it must carry a textual “Staged” chip anyway, because colour alone is never allowed to carry state.

The rejected alternative is recorded next to it, with its price. Gold could not read as pending because gold is brand chrome everywhere — and on parchment it went nearly invisible. v1 shipped that way and failed on the exact surface it existed for.

Six passes on a provisional state

What followed is the best design work in the product. Four rounds against a real screen, where fixing the row kept breaking the control inside it. A rule that provisional cards are counted separately everywhere they appear — hence “26 creatures, +3 staged” — with a note in the grouping code that section counts deliberately do not sum to the deck size and must never be used to derive one. A scroll-navigator added once it became clear that on a 99-card list the user approves whatever happens to be on screen and never learns the rest exist. And at the end, the staged styles extracted into one module, so the next staged control inherits the fix rather than rediscovering the bug.

An AI that proposes changes to a document you own creates a state most products have no vocabulary for: this exists, and you have not agreed to it.

The system survived a platform change because a build fails

The stylesheet authors 239 oklch() values, which React Native cannot parse. The obvious move is to hand-copy a palette into the native app. The file that would have held that copy instead says do not add a hex literal to this app and re-exports a generated module, checked against source in continuous integration.

It has already caught real drift: main went red when a pull request edited the stylesheet without regenerating the tokens. The failure mode was named before it happened — a hand-copied palette on a platform nobody greps would drift silently and forever — and the gap between writing the check and wiring it up was stated in the file rather than glossed.

The honest half. The document drifts. Two separate commits in the last week existed only to bring it back into line with code that had already shipped, and outside the two places where it is mechanically enforced it is a good document that depends on being read. A design system that is documentation is a wish; the part of this one that is a build failure is the part I would defend.

Collection-first, not value-first

The tempting home screen is "your collection is worth $3,936" — a number that flatters and goes nowhere. I rejected the portfolio-tracker framing; value is a secondary lens. The home surface leads with what your cards can do: decks you can build, decks that are close, what's missing.

Your most valuable printing, not a generic one

The alternative every competitor ships — resolve saved cards to a default or cheapest printing — is simpler and wrong. The deck you sleeve is made of the cards you own. resolveOwnedPrinting was more work at save-time and it's the difference between a list and your list.

Sonnet by default, Opus where it earns it

Everything-on-Opus would have made every free build a loss. The Coach runs on Sonnet; Opus is reserved for deck analysis and collection insights, where users read the output slowly. Unit economics were designed with the feature, not retrofitted after launch — and nine weeks of production bears the split out: 1,179 Sonnet calls against 85 Opus. They were then re-measured and improved: the caching work took 38% off cost per call, which is the second act the original decision never promised.

Card images bypass the image optimizer — deliberately

Card art is the most-rendered asset in the product, and routing it through a metered optimization service means one viral day makes every card in production go blank when the quota exhausts. Images ship unoptimized, straight from Scryfall's CDN. Less elegant on paper; strictly fewer ways to fail.

One consequence worth stating plainly: the fastest path to value needs no account — the landing page renders featured decks at zero clicks, and a full read-only deck is one click away — but deck building itself is auth-gated. That's a deliberate trade: every Coach run costs real money, so the free tier is metered per-account rather than open to anonymous traffic.

A product built on someone else's card database and a probabilistic model needs explicit answers for the days both misbehave:

Failure	What the system does	Why
Scryfall down or rate-limiting	Batched requests retry with backoff that honors Retry-After in full, and a failed chunk is marked unmatched rather than fatal. Imports additionally resolve against the local catalogue first	A 2,700-card import shouldn't die because request 1,900 hit a 429. (Live search has no such fallback — if Scryfall is down, search is down.)
Coach lands one card short of 99	The build-plan strip live-counts N / 99 cards in deck, and the server-side save gate independently names the exact off-by — "D too few (add lands or spells)" — regardless of what any UI says	One-card-short was the most common model failure in testing, so the gate lives in the save path, not the interface
Card image fails to load	Cascade: primary URI → Scryfall by-ID redirect → text box with the card name	A deck page never renders blank
Search finds nothing	An explicit empty state with a loosen-the-filters nudge	Dead ends need exits, not blank grids
The model errors mid-stream	The stream is inspected for an error part, an honest message is returned, and the mana is refunded as a signed ledger row	See below — this one was silently broken, and the fix is the most instructive bug in the codebase

The bug a try/catch cannot catch

The AI SDK's streamText does not throw when the model call fails. It emits an error part into the stream — and the text stream that gets served to the browser filters to text deltas and drops it. So a failed generation returned HTTP 200 with an empty body: the user saw a blank panel, no error, and was charged for it. No try/catch around the call could ever have caught this, because nothing threw. The only place the failure is visible is the full stream.

Eight surfaces had the same shape. The fix inspects the stream, returns an honest message, and refunds — with a judgment call written as a pure function: an errored stream is refunded no matter how much text arrived, so the user keeps the partial answer and the mana, while genuinely hitting the token ceiling is charged and says so. Thirty-five regression tests; twenty of them go red against the pre-fix code.

The bug my own validation certified

The card scanner is the second-most-used surface a user actually reaches for — 258 scans in production — and it shipped with a subtler failure than the one above.

A scan resolves two ways at once: exact, by set code plus collector number, and fuzzy, by the printed name. The exact lookup won unconditionally, on the reasonable theory that set-plus-number pins an exact printing. It does — if both were read correctly.

On a Marvel Super Heroes card, the model read the name "Doctor Doom" correctly, at 75% confidence, then misread the set code MSH as MH3 — visually near-identical. And set:mh3 cn:394 is a real, valid card: Ocelot Pride. So the wrong card won, and walked into the user's collection carrying its own art and its own $52 price tag.

The fix is that the name is the far stronger signal — large high-contrast type versus four-point text at the card's edge — so the exact lookup is now trusted only when it corroborates the name. The lesson leaves Magic entirely: when two extraction paths disagree, precision is not authority. The more specific identifier looked more trustworthy and was more fragile, and because its failures land on valid records, nothing downstream can catch them. This one was a money bug wearing an OCR costume.

The bug that made the product unusable for everyone I had just invited

A new account lands on the free plan: zero monthly mana, plus a one-time welcome grant of 10,000. A coach build costs exactly 10,000. So every beta signup got one AI build, ever, and then the product they had been invited to test went quiet. Accounts migrated off the old credit layer were worse — 8,000 mana, below the cost of a single build, so they could never run one at all.

There was no symptom. No error, no ticket, no support mail. I found it by querying production: of 29 free accounts, 15 could not afford a single AI action and 6 were at zero. Twenty-two of them were less than 48 hours old. Part of why it was invisible is that nothing logged a block at all — being refused by the meter was the one event the product never recorded, and that instrumentation shipped the same day as the fix.

Two details make it worth keeping. The fix is deliberately not routed through the existing comp allowlist, because that list also drives the entitlement source and would have hidden the purchase UI — blanket-comping through it would have made the product unbuyable. And the commit dates its own expiry: “Temporary. Turn it off at public launch or the pricing page is decoration.”

Underneath it sat a second bug that I think is the better one. A plan is two things — the plan column, and the mana allowance that goes with it. Stripe’s webhook sets both. Every comp I had ever issued set only the first. So 29 production accounts sat on Pro with a monthly allowance of zero, and no user could see it, because every Pro code path short-circuits before it reads a pool: the gauges all drew a confident full bar reading “Unlimited.” The admin console was the only honest surface in the product, and that is where it was caught.

The gauge said Unlimited because nothing had asked the pool. A display that never queries the thing it claims to display will never be wrong out loud.

Which is also the answer to the question the screenshots earlier on this page raise: that comp is why my own header reads “Unlimited mana,” and it is why the bug survived a month of me using the product daily.

What's deliberately imperfect: if the metering table itself errors, the gate fails open and generation proceeds unmetered. A billing bug must never lock users out of the product; the cost of that stance is that a database hiccup means free AI until it heals. I'd make the same call again — and it belongs in the documentation, not hidden in a catch block.

What's not handled yet, stated plainly: connection loss mid-edit — ordinary deck edits are plain server actions with no offline queue and no optimistic draft, so a dropped connection loses the change; only the multiplayer playtest reconnects gracefully. Rate limiting exists, but on four routes only, and it is a per-user in-memory brake rather than a distributed defense: under horizontal scale each instance keeps its own counters, so the effective ceiling is a multiple of the configured one — the source file says so about itself. Bot protection does not exist anywhere in the app, and neither does throttling on the auth endpoints. All known, none invisible.

A deck builder with no players is a database. Unlike my other projects, Grimoire shipped with its distribution mechanics in the codebase from week one:

• A referral ladder, not a referral link. Double-sided rewards with named milestones — Advocate at 3 referrals, Champion at 5, Legend at 10 — so sharing has a progression, not just a coupon. Auditing this page found a live defect in it: the rewards are still denominated in credits, the currency the product replaced with mana on 24 July. Nobody has hit the ladder yet — zero referral signups against 39 accounts holding a code — so the first person who does would be paid in a currency the product no longer spends.
• Zero-touch creator onboarding. Comped Pro is an allowlist applied automatically at sign-in — no admin panel, no manual grants, no waiting. I should be honest about the scale of it: the allowlist is a three-name array in the source, so onboarding a creator today is a code edit and a deploy. It has been used zero times.
• Every public deck is a landing page. Shared decks render read-only with no account, carry a "Build your own" call to action, and are enumerated in the sitemap for search.

Since then a social layer shipped on top: @handles, public profiles, followers, a native share sheet.

Distribution is the part I have least evidence for, so here is the evidence rather than the effort. 44 of 58 accounts arrived in the last seven days, and 27 came back. A beta that roughly tripled in a week with about half of it sticking is a real result, and it is checkable.

The funnel underneath it is the more interesting artifact, because it contradicts the product’s own framing:

Step	Users	% of signups
Signed up	58	100%
Created a deck	29	50%
Used the AI	26	45%
Catalogued a card	11	19%
Subscribed	0	0%

This is a collection manager, and 19% of the people who signed up have catalogued a single card. More of them used the AI than catalogued anything. The wedge this entire case study argues for — start from the cards you already own — is the step most people never take. Either the onboarding is failing to get a collection in, or the collection was never the reason they came. I don’t know which yet, and it is the most important open question on this page.

Revenue is $0, unambiguously and by choice: every account is comped through the beta, so there are no subscribers and no cents. Against that sits $129.55 of real Claude spend all-time, $74.71 this month — roughly $2.16 per active user per day. The mechanics of monetisation are built and tested; nobody has been asked to pay yet.

And the smaller lesson already available: granting access is the easy half. A creator account was comped and claimed inside a day, the entitlement applied automatically at sign-in exactly as designed, and it has built zero decks. Nothing in the product yet gives a creator a reason to open it on a Tuesday, and that’s a product problem I was treating as an outreach problem.

The gap I'd close first is the same one as six weeks ago, and it got sharper rather than smaller. Per-deck social previews still don't exist: the deck page — the single most-shared URL in the product, the thing the whole "every public deck is a landing page" argument rests on — has no Open Graph block at all, and there's no social-image route anywhere in the repo. A shared deck still previews as a blank card in Discord, and Discord is where Commander players live.

What makes it worse is that the codebase now demonstrably knows how. When profiles shipped, they got a proper OG block with an image and a matching card type. So the capability exists, on the surface that matters less, and the deck page still doesn't have it. That's not a missing skill, it's a missing decision — and it's still not on the roadmap, which is the honest answer rather than the flattering one.

The second gap, and it's free: the sitemap enumerates every public deck — but not the commander hub pages, one per commander, each carrying its own metadata and naturally long-tail. The largest organic search surface in the product is sitting there unlisted. The new public profiles aren't in it either.

What worked

• Parity-not-outbuild scoping. Deciding on day one which features only had to be credible freed the two weeks to be spent where the product is actually different. This is the same altitude call I make in team settings; the solo build just makes it visible.
• AI-assisted velocity, human architecture. 66,761 lines in the first 15 days is Claude Code doing the typing. The calls that made the product work — the streaming ingest parser, the atomic spend gate, owned-printing resolution, the eval harness — came from debugging real failures, not from generation.
• Measure before you tune. Building the calibrated judge before touching the coach prompt is the decision I'd repeat first. It is the only reason I found out that a change I was proud of was worth nothing once rebased.

What I'd do differently

• Never let an estimate be stored where a fact goes. Grimoire couldn't reach a real buylist, so a card's buylist price was a guess — 60% of retail. The UI said so. The database didn't: the price-snapshot table had no column marking a row as estimated, so the guess persisted in the same shape as a real vendor quote, and the price-history chart plotted it as a plain "Buylist" line with no caveat — a flat 0.6× of retail rendered as if it were market data. The disclosure lived in the one place a user might not look and was missing from the one place they would. The estimate needed a column, not a caption. So it got one: an estimated boolean, the chart now draws those series dashed and badged with "about 60% of the market price, not a real vendor quote" in the tooltip and the accessible label, and real vendor buylist data now lands flagged as genuine. Of 1,320,997 price snapshots in production, 181,058 are marked estimated. I'm keeping this one on the page after fixing it, because the architecture lesson is the point and it was the right diagnosis before it was a fix.
• Test where the money is. The codebase has 495 test files now, up from 78 at first ship. The deck-save path used to have zero. It has seven — added off a production incident where a decklist carrying 1 Swamp on four separate lines wrote four separate rows, so there was no single row to increment and I had to tell the owner of two real decks to fix them by hand. But read the caveat carefully: those seven tests cover a pure helper. The database write, the ownership scope, and the idempotency probe are still untested. And the atomic spend is still pinned by reading the source file and asserting on strings rather than executing it. Shape, not behavior. The thing production broke now has tests; the write itself still doesn't.
• A flat token cap is a guess about a distribution. A batched generation was capped at 2,200 output tokens while real responses measured 2,045–2,451 — the cap sat inside the output distribution, so it failed on a coin flip and charged for the failures. If you haven't measured the distribution, your cap is sitting somewhere in the middle of it.
• Reconcile estimates against reality. The AI-cost ledger was under-reporting by roughly threefold — one day billed $9.85 against $3.60 recorded — because it logged only the final step of a multi-step tool loop. Fixing that immediately exposed the opposite error: cached input tokens were being billed at full rate. An estimate that's never checked is a guess with a UI.
• Email before launch, not after. Transactional email is still a console-log stub — two TODO: wire an email provider comments and a console.log of the link — which means password reset is genuinely broken for a real user. There is no email provider configured in production of any kind. This has been true, unchanged, for six weeks, across two of the busiest build weeks in the project. It is the weakest thing in the product now that the price estimate has a column, and the only reason it hasn't bitten is that eight people is few enough that I hear about it directly.
• The quality gate that guards a fix is why the fix never ships. There's a retrieval tool wired into the coach that has been live and registered in production for weeks and has been called zero times — not a flag problem, the flag is on. The model simply never reaches for it, because the prompt names two other tools for the same job and never mentions it. The fix is one line of prompt. But the prompt is one of four eval-gated files, so it can't be a drive-by edit — and when I did run it through the gate properly, the gate couldn't see the tool at all, because the tool reads first-party data and the harness only registers card-fact tools. A dead tool stays dead because the instrument protecting it structurally cannot measure it. That is the cost of the discipline I'm proudest of, paid in full.
• The catalogue can't be corrected. The bulk ingest inserts and skips conflicts, so it never updates the rows it already has. Oracle errata and legality changes never reach the 116,777 existing cards; a nightly cron patches the single column that matters most as a workaround. The real fix — an upsert-mode ingest — is on the board.

When I wouldn't ship this approach

The parity-plus-wedge sprint works when the incumbents' feature set is well-understood, the underlying data is public commodity (Scryfall is a gift), and the wedge is genuinely orthogonal to what they do. It's the wrong play when correctness is the product — and the Rules Judge is the proof. It shipped answering rules questions from the model's memory, which produces good answers rather than judge-grade ones. That gap was only closable by doing the unglamorous thing: vendoring the actual Comprehensive Rules and retrieving over them. It now does exactly that.

A correction I owe this page. The last version said the moat tool — the one reading Grimoire's own signal back into the coach's picks — was "built and wired, and the table is currently empty," contributing nothing. That was wrong, and I want to be precise about the direction: I under-claimed. It isn’t one table. It fans out over five, and the deck-inclusion half has been live for weeks — 1,462 card rows across 17 public decks telling the coach what actually gets built. That number is lower than the 2,542 I published last month, and the reason is the point: the rollup used to aggregate across every deck with no visibility filter at all, and narrow to the public corpus only afterwards. The number shrank because the query got honest. The rated-turn half is real signal, but it is almost entirely mine — 129 rated turns, 127 of them graded by me while adversarially testing the coach. And the half that reads what actually wins is dark: zero games recorded, exactly as published.

And the reason that half is dark is more interesting than the status. Match outcomes accumulate from playing games inside the app. That means the multiplayer pod — which is built, playable, and switched off behind a feature flag because it isn't ready to meet real players. The moat's second half is empty because the feature that would fill it is turned off. That's a causal story, not a status line, and it's the kind of thing you only find by checking a claim you were already comfortable with.

Next on the roadmap: a native mobile app — the scanner is a phone-camera feature and people brew at the kitchen table, so the responsive web build has been carrying weight it wasn't shaped for. It is now in testers’ hands. 109,589 lines of React Native, submitted for beta review on 11 August and distributed the next day. Fifteen builds since, 19 invited testers, and 8 of them have ever installed it. It is not in the App Store — the submission how-to is written and unexecuted — and 4 of its 27 screens have ever been looked at on a phone. I track those numbers separately, because a green CI has never once caught a visual defect in this app: batch scanning shipped with 45 new native tests, 2,221 in total, typecheck clean, and a commit message that closes “NOT SEEN ON A DEVICE — every visual claim here is unverified.” The gate on this work is a walkthrough, not a build, and fifteen builds later the walkthrough still hasn’t happened. The constraint isn’t capacity: the public invite link has 100 seats and 5 are used. Alongside it: the deck-page redesign, an upsert-mode catalogue refresh, and the arbitrage panel that tells you where to actually sell. Batch card scanning came off this list and shipped. Per-deck social previews are still the biggest known gap — still not a commitment on the board, which is itself the honest answer.

Grimoire is portfolio work and a real product: live in production, in open beta, with a paid tier fully built and wired to Stripe. A handful of users, the first marketing push only days old — a six-week-old product still being finished, with the selling still ahead of it. Those counts are three days into a launch, not a result, and I'd rather not dress them up as either.

The numbers I’d rather be judged on are the engineering ones: 1,269 AI actions, $129.55 of real Claude spend, zero failed generations across fourteen features — that zero survived a 67% increase in volume — a prompt-cache hit rate of 33.9% lifetime and 73.7% since the caching work landed, 715 coach proposals accepted against 295 rejected, and a coach measured at 90.5% on a golden-set judge calibrated to κ = 1.000.

There's one more artifact I'd point at, and it's important to say exactly what it is. There are 129 rated coach turns in the database, and 127 of them are mine — graded by me, harshly, while adversarially testing prompts against my own model’s output. Exactly one other human has ever rated anything, twice. It is not a satisfaction rate and I won’t present it as one. The rating control is mounted on both web and native, so the gap isn’t a missing feature: of the 29 accounts that signed up in the two days before I checked, ten used the coach and none of them rated a thing. Read that as no signal, not as no complaints. What the table actually is, is a hand-built corpus of 127 labels that feeds the learning loop and seeds the eval set. That's a more defensible thing to have built than a satisfaction metric, and a lot more work.

I maintain it on my own time and in small windows. I'm available for full-time senior, staff, or lead IC work. Grimoire does not compete with that.

What this one proves

That I can hold a product model steady while a language model is loose inside it. The coach is fluent, fast, and completely untrusted; the system of record stays in code the model cannot write to; a human commit sits between proposal and persistence; and the instrument that measures the whole thing was built before the tuning started. The design work is the same act at a different altitude — a colour system where four vocabularies cannot borrow from each other, and one documented exception for the state an AI product creates and most products have no word for.