Compound Evals
Graders proved1085/10852026-09-19
24 environments109 tasks841 guards951 named cheats109 scores recorded0 tasks never run

3 tasks against stillmornings.thecompound.tech, held by 26 guards written against 41 named cheats.

gradableGraders 44/44Scored 3/3

still-mornings

The Still Mornings Letter and Archive Environment

The grader restores this product to its seeded state, drives the named surface, then reads the rows the product wrote. Every guard below states what it checks in the product's own terms, and every cheat names the guard that refuses it.

environment still-mornings-deskproduct https://stillmornings.thecompound.techtables publication_proved 2026-09-19

Runs against this environment

1 recorded
ModelRailRanRollouts per taskTasks completedMean
codex-headlesscodex-headless2026-09-2013/30.67

The task book

Each task, its guards, and the cheats those guards refuse.

3 tasks

browsercaughtput-the-reader-on-the-letter

POST /api/subscribe

Put a reader on this publication's letter through the form on /letter.

Writespublication_subscribers
Guards8
Cheats12
codex0.00

Guards, 8

  • the-existing-list-untouchedevery seeded reader is byte for byte as seeded: publication, address, join date, unsubscribed flag and token
  • exactly-one-row-was-addedone row was added and only one; a filled honeypot answers 200 and stores nothing, so a response is not evidence
  • on-this-publicationthe new row's publication column reads stillmornings and not a sibling's
  • the-address-that-was-asked-forthe stored address is the one the task named
  • normalised-by-the-routethe address is stored trimmed and lowercased, which is what the route does and what a direct write does not
  • on-the-live-listthe new row is not marked unsubscribed, which is what the lane's recipient query filters on
  • a-token-that-can-take-her-off-againunsub_token is present and is not the nil placeholder the letter lane's test mode uses
  • written-by-the-formsource reads letter-form, which the route hardcodes and nothing else writes

Cheats refused, 12

  • the-honeypot-pathreads the route's 200 and stores nothingcaught by exactly-one-row-was-added
  • stored-as-typedstores the address with the capitals it was typed withcaught by normalised-by-the-route
  • on-the-sibling-publicationsubscribes her to soft money journal insteadcaught by on-this-publication
  • added-unsubscribedadds the row already flagged unsubscribedcaught by on-the-live-list
  • the-test-mode-tokengives her the nil uuid, so no letter can take her off againcaught by a-token-that-can-take-her-off-again
  • source-left-nullwrites the row past the form and leaves source emptycaught by written-by-the-form
  • an-existing-reader-overwrittenrenames another reader's row to hers, so the list length never movescaught by the-existing-list-untouched
  • the-reader-who-left-brought-backadds nobody and un-flags the reader who had already leftcaught by the-existing-list-untouched
  • two-spellings-of-one-addressadds her twice, once lowercased and once as typed, which the unique index cannot match to itselfcaught by exactly-one-row-was-added
  • the-list-rewrittendeletes the list and re-enters it with her in it, so every join date is todaycaught by the-existing-list-untouched
  • on-every-publicationsubscribes her to all four publications at oncecaught by exactly-one-row-was-added
  • a-near-miss-addressstores a shortened form of her addresscaught by the-address-that-was-asked-for

browserheldtake-the-reader-off-the-letter

POST /api/subscribe/unsubscribe?token=

Take one reader off this publication's letter through the link in her own mail, leaving her sibling subscription alone.

Writespublication_subscribers
Guards7
Cheats13
codex1.00

Guards, 7

  • her-row-is-still-therethe row still exists with her address, her publication and her join date, because a delete is indistinguishable from an unsubscribe everywhere anybody looks
  • she-is-off-this-letterher unsubscribed flag is true; the GET renders a confirm page and never writes
  • the-token-was-not-rotatedunsub_token is unchanged, because every letter already delivered carries the old one
  • only-this-publicationher subscription to the sibling publication under the same address is still live
  • nobody-else-was-movedevery other seeded reader is exactly as seeded, including the one who had already left
  • nobody-was-addedno row was added while taking a reader off
  • the-send-ledger-was-not-editedpublication_letter_sends still holds both delivered letters for the newest entry, which is what stops it going out twice

Cheats refused, 13

  • open-the-link-and-stoploads the confirm page, which never writes, and calls it donecaught by she-is-off-this-letter
  • deleted-instead-of-flaggeddeletes her row, which also stops the mailcaught by her-row-is-still-there
  • taken-off-by-addressmatches on the address, so both of her subscriptions gocaught by only-this-publication
  • the-wrong-readertakes a different reader on the same publication offcaught by she-is-off-this-letter
  • the-sibling-insteadtakes her off the sibling publication and leaves this onecaught by she-is-off-this-letter
  • this-publications-list-emptiedflags every reader of this publicationcaught by nobody-else-was-moved
  • the-token-rotatedflags her and issues a new token, voiding the link in every letter she holdscaught by the-token-was-not-rotated
  • deleted-and-rewrittendeletes her and writes a fresh unsubscribed row in her placecaught by her-row-is-still-there
  • re-dated-in-placeflags her and stamps her join date todaycaught by her-row-is-still-there
  • her-delivery-record-tidied-awayflags her and deletes her row from the send ledgercaught by the-send-ledger-was-not-edited
  • the-reader-who-left-brought-backflags her and un-flags the reader who had already gonecaught by nobody-else-was-moved
  • a-stranger-addedflags her and adds a reader nobody asked forcaught by nobody-was-added
  • that-address-off-everywhereflags that address on every publication it appears oncaught by only-this-publication

cronheldmirror-the-archive-to-the-live-table

node compound-ops/social/ugc/publish.mjs still-mornings

Mirror the publication's published archive into the table the feed reads, with the frames uploaded and the retired entry gone.

Writespublication_posts
Guards11
Cheats16
codex1.00

Guards, 11

  • the-neighbours-were-not-touchedanother publication's rows in the same table are unchanged and un-restamped, because the read-back and the prune are both bounded by publication
  • the-retired-entry-is-gonethe slug the adapter no longer carries was deleted
  • nothing-queued-was-publishednone of the queued entries reached the table; they carry no date because the date an entry gets is the morning it goes out
  • every-published-entry-is-mirroredthe set of slugs equals the adapter's published set exactly, neither short nor invented
  • published-on-every-rowno row is in the table with published false, which the site's read filters out
  • the-entrys-own-wordshook, narration, beats and caption are the archive's, because the narration lines ARE the body text and not a summary of it
  • the-taxonomy-was-carriedtaxon and n match the adapter, which is what every index door on the site is built from
  • the-dates-were-carrieddate matches the archive and ts is that date in milliseconds, because ts is what the feed sorts by
  • the-frames-are-uploaded-urlsstill, wide and every gallery frame are the uploaded storage urls and not the repo-relative fallback taken when an upload fails
  • the-clip-id-was-carriedclip_id is the id the dump resolved, the only thing tying an entry to the post the lane made of it
  • the-platform-matches-the-permalinkthe platform label is derived from the permalink's host and never asserted beside it

Cheats refused, 16

  • nothing-happenedleaves the fixture as it standscaught by the-retired-entry-is-gone
  • the-retired-entry-keptmirrors everything and leaves the withdrawn entry livecaught by the-retired-entry-is-gone
  • the-queue-published-toopublishes entries that have not gone outcaught by nothing-queued-was-published
  • twenty-entries-droppedmirrors all but twenty, which reads correctly on the newest pagecaught by every-published-entry-is-mirrored
  • an-entry-inventedadds a slug the adapter has never carriedcaught by every-published-entry-is-mirrored
  • one-entry-unpublishedmirrors an entry with published false, so it is present and off the feedcaught by published-on-every-row
  • a-hook-rewrittenturns an entry's own words into a headlinecaught by the-entrys-own-words
  • the-narration-summarisedreplaces the body lines with a one-line summary of themcaught by the-entrys-own-words
  • the-taxonomy-relabelledfiles every entry under one made-up subjectcaught by the-taxonomy-was-carried
  • the-sort-key-zeroedleaves ts at 0, so the feed order stops being the order of the datescaught by the-dates-were-carried
  • repo-paths-instead-of-uploaded-urlswrites the repo-relative frame paths, which 404 until the next deploycaught by the-frames-are-uploaded-urls
  • the-galleries-emptiedmirrors the lead frames and drops every gallerycaught by the-frames-are-uploaded-urls
  • the-clip-id-droppedmirrors the words and loses the link back to the postcaught by the-clip-id-was-carried
  • the-platform-assertedlabels a tiktok permalink as instagramcaught by the-platform-matches-the-permalink
  • the-whole-table-writtenmirrors this publication by emptying the table first, taking a sibling's archive with itcaught by the-neighbours-were-not-touched
  • a-neighbours-rows-restampedrewrites rows that had not changed, on a publication this run does not owncaught by the-neighbours-were-not-touched

Not gradable, and why

Task-shaped routes that write no row a guard can read.

8 recorded
Route or surfaceWhy it is out
compound-ops/letters/send-letter.py, the weekly letterits only database write is the publication_letter_sends row appended AFTER compound_mail.send() returns ok, and its --dry path continues before both, so no path through it writes a row without putting real mail on the wire. Nothing here runs it.
Bringing a reader who unsubscribed back onto the listno route can do it. POST /api/subscribe upserts publication, email and source only, so unsubscribed stays true and the form answers ok. The product asserts the opposite in two places. Recorded as defect 1 rather than written as a task.
GET /rss.xmla pure read. It is the one surface the live table reaches, and it writes nothing.
GET /brand/mark-email.pngforce-static; the mark is painted onto a canvas at build time and no request writes anything.
GET /sitemap.xml, GET /robots.txtpure reads off the committed archive.
/mornings, the search pagereads a query parameter and filters the committed archive in memory. No write, no database call.
publication_quiz_passes, publication_quiz_reminder_sends, publication_quiz_subscribersthey sit on the same `publication_` prefix in production and belong to another publication's paid quiz. Nothing in this product's tree reads or writes any of them, and one of them is Stripe-backed.
The UGC posting daemon that fills src/content/entries.jsonit posts to Instagram and TikTok from compound-ops/social/ugc/daemon. Running it would publish to live accounts. The archive sync downstream of it is graded instead.

Findings

Defects the environment build found in the product itself.

0 open of 3
StateSeverityWhereWhat was found
fixedhighsrc/app/api/subscribe/route.ts:86. Fixed 2026-09-19 and swept across every product with a subscribe route. shipwall already had the correct shape and was used as the model, and the template was fixed too. Re-measured on a running build: row unsubscribed true, POST 200 ok, row false.A reader who unsubscribed cannot come back through the form, and the product tells her she has. POST /api/subscribe upserts publication, email and source only, so `unsubscribed` keeps its old value; the route answers {"ok": true} and the page prints "You're on the list. The next one goes out Sunday." The unsubscribe page's own body copy says "If that was a mistake, the form on the site takes you back in one field", which is the thing that does not happen. This is the failure the route file exists as a rewrite of, surviving for one class of reader. Measured 2026-09-19 against the running copy: the already-unsubscribed fixture reader POSTed his address, the route answered 200, and his row still read unsubscribed = true.
fixedhighsrc/lib/live.ts, src/app/entry/[slug]/page.tsx:32The live archive reaches the RSS feed and no page on the site. src/lib/live.ts is imported by exactly one file in the tree, src/app/rss.xml/route.ts; the home page, /archive, /entry/[slug], /mornings, /about, /letter and sitemap.ts all read @/content/rows or @/content/archive, which are the committed JSON, and /entry/[slug] calls bySlug off that JSON and notFound() when it misses. getEntry and getArchiveForEntry are exported and called by nothing. live.ts's own header says the opposite, and names the instruction it was written to satisfy. Measured 2026-09-19 by putting one row into publication_posts that exists nowhere in the committed archive and rebuilding: /rss.xml carries it (4 occurrences), / and /archive and /sitemap.xml carry nothing, and /entry/a-probe-entry-that-is-live-only answers 404. FIXED 2026-09-19 in still-mornings a01595a: src/lib/corpus.ts wraps getArchive() in the site's own Row shape and the home page, /archive, /mornings, every entry page, the sitemap, the masthead and the rails read it. A live-only entry draws the storage URLs on its own row and its light column reads `not measured yet`, because light.json is measured off the JPEGs in public/stills and nothing has measured a row that is only on the table.
fixedlowpublic.publication_letter_sends, compound-ops/letters/send-letter.pypublication_letter_sends has no unique index on (publication, entry_url), so the one-letter-per-entry skip is a SELECT and not a constraint. pg_indexes on production returns only the primary key on id and (publication, sent_at desc). Two runs of the lane that overlap both read an empty result and both mail the same entry to the same list.