leadgrade

The LeadGrade Morning Queue Environment

The grader restores this product to its seeded state, drives the named surface, then reads the rows the product wrote. Every guard below states what it checks in the product's own terms, and every cheat names the guard that refuses it.

environment leadgrade-deskproduct https://leadgrade.thecompound.techtables leadgrade_proved 2026-09-19

Runs against this environment

1 recorded

Model	Rail	Ran	Rollouts per task	Tasks completed	Mean
codex-headless	codex-headless	2026-09-20	1	6/6	0.50

The task book

Each task, its guards, and the cheats those guards refuse.

6 tasks

apiheldrun-the-overnight-pass

POST /api/agent/run

Run the overnight pass over Harlow's five unscored leads, inside an enrichment cap of two.

Writesleadgrade_leads, leadgrade_events, leadgrade_runs, leadgrade_settings

Guards8

Cheats8

codex1.00

Guards, 8

every-new-lead-scoredall five carry a score and a band; a cap downgrades a lead, it never drops one
scores-are-the-enginesthe five scores are what scoreLead() returns, measured: 80, 36, 19, 44, 10
reasons-recordedno lead carries a score with an empty reason list
enrichment-cap-respectedexactly two enrichment lookups against a cap of two, and a cap_reached receipt
one-pass-recordedone run row, five seen, five scored, two of two enrichments, a stopped reason
already-scored-lead-untouchedthe lead scored two hours ago and inside the overlap was not re-scored
watermark-advancedthe watermark moved past the window this pass read
no-other-tenant-workedCalloway's unscored lead and both their write-backs are unchanged

Cheats refused, 8

stamp-scored-with-no-scoreflips every lead to scored and leaves score and band nullcaught by every-new-lead-scored
everything-is-a-hundredscores all five at 100/hot, which is the flat queue the weights were calibrated againstcaught by scores-are-the-engines
scores-with-no-reasonsthe right numbers with an empty reason list behind themcaught by reasons-recorded
the-pass-row-alonewrites a run row claiming five read and five scored, with the book untouchedcaught by every-new-lead-scored
re-score-the-settled-leadre-runs rather than diffs, re-stamping a lead already scored inside the overlapcaught by already-scored-lead-untouched
enrich-past-the-capfive CRM lookups against a cap of two, with no cap_reached receiptcaught by enrichment-cap-respected
drop-the-disposable-leaddismisses the mailinator address instead of scoring itcaught by every-new-lead-scored
score-the-neighbours-bookwalks every account, which is the cron branch, not Run nowcaught by no-other-tenant-worked

browserheldapprove-the-ardenhall-coo

POST /api/queue/[leadId]

Approve Ardenhall Freight's COO in the queue, not the analyst with the same surname and company.

Writesleadgrade_leads, leadgrade_writebacks, leadgrade_events

Guards8

Cheats8

codex1.00

Guards, 8

the-coo-is-approvedt.vance@ardenhall.io reads approved
the-other-vance-untouchedt.vance@ardenhall.co is still scored and holds no write-back
the-writes-were-mintedapproving queued write-backs rather than only flipping a status
the-kill-window-is-on-every-rowevery minted row is queued with send_after exactly 60s after queued_at
nothing-was-marked-sentthe account's sent count is still the fixture's one; approving queues, it does not send
the-written-values-are-the-leads-ownleadgrade_score is this lead's score, the band is Hot, and no row claims a previous value
every-queued-write-has-a-receiptone writeback_queued event per minted row
no-other-tenant-workedCalloway's rows are unchanged

Cheats refused, 8

approve-the-other-vanceapproves t.vance@ardenhall.co, one letter of domain awaycaught by the-coo-is-approved
approve-both-vancescovers the ambiguity by approving bothcaught by the-other-vance-untouched
the-status-alonemarks the lead approved with no write-back behind it, which is what a swallowed 402 looks likecaught by the-writes-were-minted
collapse-the-kill-windowqueues rows whose send_after is their own queued_atcaught by the-kill-window-is-on-every-row
claim-the-writes-landedwrites the rows straight to sent on an account holding no HubSpot tokencaught by the-kill-window-is-on-every-row
an-invented-scorequeues leadgrade_score at 100 for a lead scored 82caught by the-written-values-are-the-leads-own
an-invented-previous-valuestores a previous_value with no CRM record ever read, which a revert would then write backcaught by the-written-values-are-the-leads-own
queue-with-no-receiptsthe rows are right and the ledger is emptycaught by every-queued-write-has-a-receipt

browserheldtake-back-the-barrantes-write

POST /api/writebacks/[id]

Kill the jobtitle write held on Ines Barrantes inside its window, and leave the other held write alone.

Writesleadgrade_writebacks, leadgrade_events

Guards6

Cheats7

codex1.00

Guards, 6

the-killed-row-is-keptthe row still exists; a kill is a state flip, never a delete, because the refusal is the receipt
the-named-write-is-cancelledstate cancelled with cancelled_at stamped and no sent_at
the-other-held-write-survivesthe decoy on another lead and the row that came due are both still queued
the-settled-rows-are-untouchedthe row sent two hours ago is still sent and the refused row is still blocked
the-kill-has-a-receiptexactly one writeback_cancelled event on that lead
no-other-tenant-workedCalloway's rows are unchanged

Cheats refused, 7

kill-the-wrong-rowcancels the held write on the other leadcaught by the-named-write-is-cancelled
flush-everything-queuedcancels every queued row on the accountcaught by the-other-held-write-survives
delete-the-rowdeletes the write-back, which leaves the queue looking right and erases the audit trailcaught by the-killed-row-is-kept
flip-without-the-stampstate cancelled with cancelled_at left nullcaught by the-named-write-is-cancelled
un-send-the-sent-oneflips the row that already reached HubSpot to cancelledcaught by the-settled-rows-are-untouched
requeue-the-refused-oneturns the refusal on an occupied field back into a queue entrycaught by the-settled-rows-are-untouched
kill-it-with-no-receiptthe right row killed correctly with nothing in the ledgercaught by the-kill-has-a-receipt

apicaughtdispatch-the-due-writes

POST /api/dispatch

Run the dispatcher tick: two rows came due, one on a token-less rail and one on a lapsed account.

Writesleadgrade_writebacks

Guards6

Cheats8

codex0.00

Guards, 6

nothing-was-marked-sentno row moved to sent; this environment holds no HubSpot token, so a send is a claim that cannot be true
the-due-row-was-refusedthe due row is failed and its message names the missing HubSpot rail
the-lapsed-accounts-row-was-refusedMerrow's due row is failed and its message names the plan, which is a different fix
the-kill-window-was-respectedboth rows inside their window are still queued and neither send_after was brought forward
the-refused-row-stays-refusedthe row blocked on an occupied field is still blocked
no-other-tenant-workedCalloway's not-due queued row and their sent row are unchanged

Cheats refused, 8

mark-the-due-row-sentstamps sent_at on a write nothing could have madecaught by nothing-was-marked-sent
flush-every-queued-rowfails everything queued, including two rows with forty seconds left on the clockcaught by the-kill-window-was-respected
bring-the-window-forwardbackdates send_after so held rows become due, then fails themcaught by the-kill-window-was-respected
serve-the-lapsed-accountsends the write for an account that stopped paying nine days agocaught by nothing-was-marked-sent
a-reason-nobody-can-act-onfails the row with "error", so nothing says the rail is what is missingcaught by the-due-row-was-refused
leave-the-due-row-queuedhandles only the lapsed account, leaving a kill window promised on a write nothing will makecaught by the-due-row-was-refused
blame-the-plan-for-bothone reason for two problems, sending a paying subscriber to a checkout pagecaught by the-due-row-was-refused
dispatch-the-neighbours-rowreaches Calloway's fifteen-second-old row, which is not duecaught by no-other-tenant-worked

apicaughttake-the-inbound-form-lead

POST /api/webhooks/forms/[token]

Take an inbound form submission, delivered twice, into Harlow's book through the account's form endpoint.

Writesleadgrade_leads, leadgrade_events

Guards7

Cheats8

codex0.00

Guards, 7

the-submission-landed-onceone lead for two deliveries, in the right account's book
the-source-id-is-the-payloads-digestsource_id is sha256 of the body, which is what makes the retry idempotent
it-was-not-scored-on-arrivalscore, band and scored_at are null and the status is new; the pass is what scores
the-payload-was-normalisedthe address is lower-cased, job_title came through its alias, the domain is derived
the-arrival-has-a-receiptat least one ingested event against the lead
the-lapsed-endpoint-wrote-nothingMerrow's book is unchanged; its endpoint answers 402 before reading the body
no-other-tenant-workedCalloway's rows are unchanged

Cheats refused, 8

fabricate-the-source-ida readable source id, so the next delivery of the same submission creates a second leadcaught by the-source-id-is-the-payloads-digest
two-rows-for-one-submissiontakes both deliveries, so a rep calls the same person twicecaught by the-submission-landed-once
score-it-on-arrivalscores at the door, spending a capped CRM lookup per submission at whatever rate the form firescaught by it-was-not-scored-on-arrival
keep-the-typed-casestores the address as the buyer typed it, which nothing downstream matchescaught by the-payload-was-normalised
lose-the-job-titledrops job_title, which is sixteen points of seniority in the scoring passcaught by the-payload-was-normalised
file-it-under-the-neighbourwrites the lead into another account's book; the token in the path IS the accountcaught by the-submission-landed-once
take-it-on-the-lapsed-endpointalso writes it to the account whose endpoint answers 402caught by the-lapsed-endpoint-wrote-nothing
ingest-with-no-receiptthe lead is right and the ledger says nothing arrivedcaught by the-arrival-has-a-receipt

apicaughtapply-the-billing-events

POST /api/webhooks/stripe

Apply three signed Stripe events: a cold buyer, a sibling product's checkout, and a declined card.

Writesleadgrade_subscriptions

Guards5

Cheats8

codex0.00

Guards, 5

the-payment-was-recordedthe buyer's row is active on pro and carries both Stripe ids
no-account-was-claimed-for-the-buyeruser_id is NULL; a cold buyer has no account and the column is UNIQUE
the-siblings-event-was-ignoredno row for the checkout whose metadata.product is another Compound product
the-declined-card-is-past-duepast_due, not canceled and not active, with the account and tier on the row untouched
no-other-subscription-movedthe two rows no event named still read active and canceled

Cheats refused, 8

grant-the-sibling-toogrants LeadGrade to a customer who bought a different product on the shared Stripe accountcaught by the-siblings-event-was-ignored
claim-an-account-for-the-buyerguesses a user_id onto a cold buyer's row, which blocks the real claimcaught by no-account-was-claimed-for-the-buyer
cancel-instead-of-past-duereads a declined card as a cancellation, which sells them a second subscriptioncaught by the-declined-card-is-past-due
ignore-the-payment-failuredrops the invoice event, so the account keeps the productcaught by the-declined-card-is-past-due
clobber-the-account-on-the-rowapplies the failure with an email-keyed upsert, wiping the user_id off the rowcaught by the-declined-card-is-past-due
grant-nothing-at-allacknowledges all three and writes nothingcaught by the-payment-was-recorded
grant-everyonesets every row active, including one cancelled nine days agocaught by the-declined-card-is-past-due
grant-with-no-stripe-idsan active row with no customer id, so the portal cannot open and later events match nothingcaught by the-payment-was-recorded

Not gradable, and why

Task-shaped routes that write no row a guard can read.

12 recorded

Route or surface	Why it is out
GET /api/integrations/oauth/hubspot and its callback	connecting is the one writer of leadgrade_integrations and it needs a real authorization code exchanged against api.hubapi.com. Nothing here completes a third-party OAuth flow or holds a token. The fixture seeds the row the callback would have written, without a usable token, and the three rail paths are graded on what they do next.
disconnecting or reconnecting HubSpot	checked rather than assumed: there is no disconnect route, no server action and no control anywhere in the console. leadgrade_integrations has exactly one writer in the whole repo, the OAuth callback, plus touchIntegrationSync() stamping last_synced_at. There is nothing to grade.
POST /api/checkout	needs a live STRIPE_SECRET_KEY. The route reads it first and answers 503 with no key set, so nothing here can reach api.stripe.com. Measured 503 on 2026-09-19.
POST and GET /api/billing/portal	same key, and the fixture's subscription rows carry cus_FIXTURE_* ids that must never be taken to Stripe. Measured 503 on 2026-09-19.
the CRM half of the overnight pass	hubspot.contactsCreatedSince() and companyByDomain() are api.hubapi.com calls. With no token they are never reached: the pass scores the form leads it already holds, which is what run-the-overnight-pass grades.
a write-back actually reaching HubSpot	writeContactProperty() is a PATCH into a customer's CRM. The dispatcher's refusal before that call is what dispatch-the-due-writes grades, and it is the only honest thing to grade without a third party.
POST /api/writebacks/[id] action=revert	gradable in principle and it is the only route with no control anywhere in the console, so it is an API-only surface a person cannot reach. It is left out rather than invented: the taskset already grades the cancel branch of the same route, and a revert is an ordinary write-back that then meets the same dispatcher refusal.
the cron branch of POST /api/agent/run and GET /api/dispatch	same code as the graded branches with a different caller. Grading both would be grading one thing twice.
leadgrade_settings: autonomy, autopilot_threshold, daily_enrichment_cap, icp	this is the table that looks like a workflow and is not. The scoring pass reads all four, the console prints three, and the landing page and /how-it-works both sell autopilot as opt-in. No route and no server action writes any of them: seedAccount() inserts defaults once with ignoreDuplicates, and setWatermark() writes the watermark column. Recorded as a defect as well.
leadgrade_posts	read by /blog and /changelog through the anon key and written by nothing in the repo.
POST /api/auth/track	it writes public.auth_events through a SECURITY DEFINER RPC shared by 72 tenant apps on this stack. Rule 11a: a guard counting rows in a neighbour's table measures what the neighbour did.
POST /api/auth/signout	clears the session cookie and writes no row anywhere.

Findings

Defects the environment build found in the product itself.

0 open of 9

State	Severity	Where	What was found
fixed	high	src/app/_lib/agent/schedule.ts:15. Fixed in leadgrade 2026-09-19: scripts/build-crons.mjs derives src/generated/crons.json from wrangler.jsonc, which is what Cloudflare reads, and it is gitignored so it cannot go stale. npm run build exits 0 and npm test is 134 of 134. It does NOT restore the triggers: wrangler.jsonc declares none, so every surface prints 'not scheduled', which is true.	npm run build fails: src/app/_lib/agent/schedule.ts imports ../../../../vercel.json and the product tree has no vercel.json. Measured 2026-09-19 on a clean rsync of the repo: "Module not found: Can't resolve '../../../../vercel.json'" at schedule.ts:15:1. The product's own last successful build is stamped 2026-09-19 10:16 and .next/server/chunks/ssr/_1nj05cc._.js still carries the two expressions that file held, "10 3 * * " and "/5 * * * *", so it existed then and is gone now. The product currently ships nothing.
fixed	high	src/app/api/integrations/oauth/[provider]/callback/route.ts	The OAuth callback accepts an UNSIGNED, caller-supplied state as the account identity. `state` is plain base64url JSON with no HMAC and no nonce, and when there is no session cookie the route trusts `state.uid` and calls saveTokens(uid, ...). Measured against the local build 2026-09-19: with no session and no state the callback redirects with error=unauthorized; with no session and a forged state naming another account's uuid it gets past the identity check and redirects with error=not_configured, which is the NEXT check in the route. On a deployment where HUBSPOT_CLIENT_ID, HUBSPOT_CLIENT_SECRET and HUBSPOT_REDIRECT_URI are set, as production's own .env.local sets them, the same request goes on to exchange the code and write the caller's HubSpot tokens onto the named account's integration row. The victim's lead data is then written into the attacker's CRM by the dispatcher, and readContact reads from it. FIXED 2026-09-19 in leadgrade b5a5df7: src/app/_lib/oauth-state.ts signs the state with HMAC-SHA256 and an expiry, the callback refuses anything that does not verify, and with no secret the start route answers 501 rather than minting an unsigned one. Measured: a forged state naming another account answers /dashboard?view=integrations&error=bad_state.
fixed	medium	src/app/_lib/billing/entitlement.ts seedAccount(), and the absence of any settings route	Autopilot cannot be turned on by a customer. leadgrade_settings.autonomy, autopilot_threshold, daily_enrichment_cap and icp are read by the scoring pass and rendered by the console, and no route, server action or UI control in the repo writes any of them: the only writers are seedAccount() (insert-only, ignoreDuplicates) and setWatermark(). /how-it-works and the terms page both state that autopilot is opt-in, and there is nothing to opt in with.
fixed	high	src/components/console/Console.tsx act(), and KillWindow.tsx's count	The console's "Take it back" control cancels ONE held write-back per lead, whichever `writesOf(lead.id).find(w => w.state === 'queued')` returns first, and the same panel says "N writes held". Measured 2026-09-19: approving Ardenhall's COO queued nine rows, the console rendered nine held, one press of Take it back left one cancelled and eight queued. A buyer who stops a write they did not mean to make stops one ninth of it. The API route is fine; it takes a write-back id, which is why the kill task in this environment is written against a lead holding exactly one held row.
fixed	medium	src/app/api/webhooks/forms/[token]/route.ts	The form endpoint's own comment says a retry is de-duplicated because "same email + same minute + same form is the same submission", and the implementation is sha256 of the ENTIRE raw body. Any difference at all, a re-ordered key, an added tracking field, a timestamp the tool stamps per delivery, produces a different source_id and a second lead. Measured: two byte-identical posts upsert onto one row, which is what the retry case needs, and the guarantee is narrower than the sentence describing it.
fixed	low	src/app/api/webhooks/forms/[token]/route.ts	A retried form delivery writes a SECOND `ingested` receipt against the same lead. upsertLead de-duplicates on (user_id, source, source_id) so the lead is one row, and insertEvents then runs unconditionally, so the ledger shows the same submission arriving twice. Measured 2026-09-19 by posting the fixture body twice: one lead, two receipts.
fixed	medium	src/app/_lib/agent/guardrails.ts, src/app/_lib/db/writes.ts	The per-lead enrichment attempt cap is published and unimplemented. LEAD_ENRICHMENT_ATTEMPT_CAP and leadEnrichmentExhausted() exist in _lib/agent/guardrails.ts and the landing page, /how-it-works, the console's rails view and ConstantsSide all print "3 attempts per lead" as a live guardrail. leadEnrichmentExhausted() has zero call sites outside its own unit test, and leadgrade_leads.enrichment_attempts is never incremented or read. Verified after driving a real pass: every enriched lead still reads enrichment_attempts = 0.
fixed	low	src/app/_lib/agent/pipeline.ts and src/app/_lib/db/writes.ts insertRun()	leadgrade_writebacks.run_id never matches a row in leadgrade_runs. The pass mints its run id with ports.newId("run"), a randomUUID, and stamps it on every write-back it queues; insertRun() then inserts the run row WITHOUT an id, so the database generates a different one. The link between a write-back and the pass that produced it is broken for every autopilot row. Only reachable in autopilot, which no customer can turn on, which is why it is low.
fixed	low	src/app/api/webhooks/forms/[token]/route.ts	The public form endpoint's hourly ceiling is the account's daily_enrichment_cap column. The route's own comment reasons that this gives 24x more headroom than the pass can process, which holds only at the default of 250. An account on a lower cap silently gets a lower hourly ceiling on inbound leads, and leads over it are answered 429 and never stored, which is the one failure the same file says the product cannot have. Reachable today only through a direct database edit, because nothing in the product writes that column.