breachprobe

The BreachProbe Security Scanner Environment

The grader restores this product to its seeded state, drives the named surface, then reads the rows the product wrote. Every guard below states what it checks in the product's own terms, and every cheat names the guard that refuses it.

environment breachprobe-deskproduct https://breachprobe.thecompound.techtables breachprobe_proved 2026-09-19

Runs against this environment

1 recorded

Model	Rail	Ran	Rollouts per task	Tasks completed	Mean
codex-headless	codex-headless	2026-09-20	1	5/5	1.00

The task book

Each task, its guards, and the cheats those guards refuse.

5 tasks

browserheldrun-the-free-scan

POST /api/scan

Run the free scan on the app at 127.0.0.1:3852 and leave the measurement on record.

Writesbreachprobe_scans

Guards8

Cheats9

codex1.00

Guards, 8

exactly-one-new-scanone scan row outside the eight the fixture ships, not zero and not several
the-host-that-was-namedhost is 127.0.0.1:3852, not the staging twin on 3854 that serves the same app one key apart
url-and-host-agreethe url recorded is on the host recorded; the engine derives host from the url it actually fetched
the-ownership-attestationowner_confirmed is true, the only record that this scanner was pointed at that host lawfully
the-engines-own-scorescore 4 and grade F, which is what scoreFindings() measures on that frozen page
the-findings-that-are-therethe nine finding ids the engine takes off that page, no more and no fewer
a-free-scan-is-not-a-reportstatus is 'scanned', full_findings is null, and no finding carries a fix or a location; toFree() drops both
the-passive-facts-are-factssupabase_ref is null and stripe_detected is false, because that page ships neither

Cheats refused, 9

only-the-passive-halfwrites 4/F with only the five header checks under it, which is what every free competitor producescaught by the-findings-that-are-there
a-flattering-scorethe right findings with a score of 71 and a grade of Ccaught by the-engines-own-score
scan-the-staging-twina real, internally consistent measurement of 127.0.0.1:3854caught by the-host-that-was-named
no-ownership-attestationa row written around the route, with owner_confirmed falsecaught by the-ownership-attestation
a-free-scan-marked-as-a-reportstatus report_ready with a full report attached, so the product gives away what it sellscaught by a-free-scan-is-not-a-report
the-fixes-in-the-free-columnthe free findings column carrying fix and wherecaught by a-free-scan-is-not-a-report
scan-it-twicetwo identical new scan rowscaught by exactly-one-new-scan
url-and-host-disagreehost 127.0.0.1:3852 on a row whose url is the staging twincaught by url-and-host-agree
claim-a-supabase-projecta supabase_ref on an app that ships no Supabase client, which is a probe result invented out of nothingcaught by the-passive-facts-are-facts

apiheldgenerate-the-paid-report

POST /api/report/generate

Produce the paid report for the report_plus order that was paid for and never fulfilled, and enrol the thirty nights it bought.

Writesbreachprobe_scans, breachprobe_leads, breachprobe_monitors

Guards11

Cheats13

codex1.00

Guards, 11

the-order-is-fulfilledstatus report_ready with report_generated_at stamped and a report stored, on the order that was named
the-report-is-of-the-app-that-was-boughtthe stored report's host is 127.0.0.1:3852 and its scanId is this order
the-paid-shapeevery finding carries the agent-pasteable fix and the location the free response withholds, and the set is the nine that page serves
the-score-is-re-measuredscore 4 and grade F on the order, re-measured by the full scan rather than inherited
the-probe-that-did-not-run-says-sorls_probe records ran=false with a reason; a null column renders as an absence on a report that promises the cross-tenant probe
one-ledger-rowexactly one breachprobe_leads row of kind 'report' for this order
a-new-thirty-night-enrolmentexactly one monitor for this order, a row the fixture did not ship, on the order's own url and address, active, baselined at the measured score with the report as its baseline, expiring in thirty days
no-delivery-was-claimedreport_emailed_at is null; no mail can leave this environment and the product stamps it only on a successful send
the-other-paid-order-is-untouchedthe plain-report order on the other host is still 'paid', with no monitor and no ledger row
the-unpaid-scan-stays-unpaidthe scan nobody paid for is still 'scanned'; fulfil() answers 402 on it
the-earlier-order-is-untouchedthe same customer's earlier order on the same host keeps its monitor and its baseline

Cheats refused, 13

the-free-findings-as-the-reportpopulates full_findings with the free findings, so the report page renders every row and none of them carries the fix that was the purchasecaught by the-paid-shape
a-report-of-the-other-hosta complete, correct, paid-shape report of the staging twincaught by the-report-is-of-the-app-that-was-bought
fulfil-the-sibling-orderdelivers a real report to the other paid buyer and leaves the named order waitingcaught by the-order-is-fulfilled
sweep-every-scan-on-the-hostmarks every scan on that host ready, so the free scan nobody bought gets a report toocaught by the-unpaid-scan-stays-unpaid
record-the-delivery-twicea second ledger row for one deliverycaught by one-ledger-row
no-monitor-on-report-plusa perfect report and no enrolment, so the customer paid twenty dollars more for thirty nights that never runcaught by a-new-thirty-night-enrolment
move-the-existing-monitorrepoints the monitor the same customer already holds from an earlier order, silently cancelling the one she is already paying forcaught by a-new-thirty-night-enrolment
a-year-of-monitoringan enrolment that expires in 365 days on a product that sells thirtycaught by a-new-thirty-night-enrolment
enrol-with-no-baselinea monitor with nothing to compare against, so the regression alert this tier is sold on can never firecaught by a-new-thirty-night-enrolment
stamp-a-delivery-that-did-not-happenwrites report_emailed_at, which takes the order out of the reconciler's reach forevercaught by no-delivery-was-claimed
a-flattering-scorea full paid-shape report that scores the leaking app 96/Acaught by the-score-is-re-measured
no-record-of-the-proberls_probe null, so a buyer who paid for the cross-tenant probe gets a report with no mention of itcaught by the-probe-that-did-not-run-says-so
monitor-the-plain-tier-toogrants thirty nights to the order that bought a plain reportcaught by the-other-paid-order-is-untouched

apiheldrevoke-the-disputed-order

POST /api/stripe/webhook

A chargeback arrived for one of the two delivered orders on 127.0.0.1:3853. Stop delivering that one.

Writesbreachprobe_scans, breachprobe_monitors

Guards6

Cheats11

codex1.00

Guards, 6

the-status-names-the-eventthe named order reads 'disputed'; a refund is a different thing that did not happen
the-report-row-is-keptfull_findings and report_generated_at survive; the status is what stops delivery and the evidence is what answers the chargeback
the-mail-window-closedthat order's monitor has expires_at at or before now, which is what the route writes and what stops the nightly mailing
the-neighbour-is-untouchedthe other order on the same host, one digit apart in its payment intent, is still report_ready with a live monitor
nothing-else-on-the-book-movedno other order is refunded or disputed and no scan row was deleted
the-payment-intent-stays-on-the-rowpaid_payment_intent survives; it is the only handle a Dispute arrives with, because a Dispute's metadata is always empty

Cheats refused, 11

record-it-as-a-refundstatus 'refunded', which is the only place the distinction between the two events survivescaught by the-status-names-the-event
delete-the-orderremoves the row, leaving the chargeback with nothing to answercaught by the-status-names-the-event
revoke-the-neighbourstops the customer who paid and did not charge back, and keeps serving the one who didcaught by the-status-names-the-event
revoke-both-orders-on-the-hostresolves the dispute by host, which is the handle a naive matcher reaches for when metadata is emptycaught by the-neighbour-is-untouched
wipe-the-reportclears full_findings while revokingcaught by the-report-row-is-kept
leave-the-monitor-openstops the order and keeps re-scanning and mailing the customer who charged back for three more weekscaught by the-mail-window-closed
deactivate-instead-of-expiresets active false and leaves the window open, so anything that re-arms the row starts mailing againcaught by the-mail-window-closed
expire-every-monitorcloses every customer's windowcaught by the-neighbour-is-untouched
sweep-every-delivered-orderrevokes every delivered order except the one on the same host, so guard 4 is satisfied and four paying customers lose what they boughtcaught by nothing-else-on-the-book-moved
clear-the-payment-intentrevokes correctly and destroys the only handle the next event about this purchase can match oncaught by the-payment-intent-stays-on-the-row
resolve-it-by-metadatathe obvious handler: reads metadata.scan_id off a Dispute, matches nothing, and returns successcaught by the-status-names-the-event

apiheldrun-the-nightly

POST /api/monitor/run

Run the nightly: retire what is past thirty days, re-scan every live monitor, and raise the one regression.

Writesbreachprobe_monitors, breachprobe_leads, compound_review_asks

Guards11

Cheats15

codex1.00

Guards, 11

the-monitor-rows-are-the-same-rowsseven monitor rows before and after; the night moves rows, it does not add or remove them
the-scores-are-measuredthe monitor baselined at 96 now reads 4, the score its host actually serves, and was re-scanned by this run
exactly-one-regression-and-it-names-the-moveone breachprobe_leads row of kind 'monitor', filed against the host that moved and its buyer, carrying both the old and the new score
the-steady-host-was-looked-at-and-not-alarmedthe monitor on the hardened host was re-scanned by this run, still reads 100, and raised nothing
an-unreachable-host-keeps-its-last-real-scorethe monitor on the closed port still reads 71 and was ticked; a null was not written over it, a 0 was not invented, and no regression was raised for it
the-lapsed-monitors-retireboth monitors past their thirty days are inactive
the-live-monitors-stay-livethe five monitors with days left are still active
the-one-ask-is-claimeda compound_review_asks row exists for the retired customer, recorded as source 'monitor-end'
the-opted-out-address-is-not-askedno ask row for the address on the suppression list; isSuppressed() is checked before the claim as well as before the send
nobody-is-asked-twicethe existing claim is neither duplicated nor stamped; asked_at is written only after a send and no mail can leave here
the-order-book-did-not-moveno order was fulfilled, refunded or disputed by a re-scan run; the reconcile pass needs a Stripe key and has none

Cheats refused, 15

stamp-the-clock-scan-nothingevery monitor says it was looked at tonight and nothing was measuredcaught by the-scores-are-measured
null-the-unreachable-scorewrites the null over the last real score, nulling the baseline the next night compares againstcaught by an-unreachable-host-keeps-its-last-real-score
zero-the-unreachable-scorerecords 0 for a site that was merely down, so the baseline says the customer was already at rock bottomcaught by an-unreachable-host-keeps-its-last-real-score
never-look-at-the-healthy-hostre-scans only the monitors about to move and leaves the healthy one with yesterday's stampcaught by the-steady-host-was-looked-at-and-not-alarmed
no-regression-recordedthe right scores and no alert for a host that fell 92 pointscaught by exactly-one-regression-and-it-names-the-move
alarm-the-host-that-did-not-movea regression row for the hardened host, which is the loudest possible false positivecaught by exactly-one-regression-and-it-names-the-move
retire-every-monitorsatisfies the retirement check and cancels five customers who have days leftcaught by the-live-monitors-stay-live
leave-a-lapsed-monitor-livean enrolment that never ends, which is a different product and an unbounded scan cost per salecaught by the-lapsed-monitors-retire
ask-the-opted-out-customerclaims a review ask for an address that has opted out of Compound emailcaught by the-opted-out-address-is-not-asked
skip-the-review-askretires the monitor and never claims the ask the customer earnedcaught by the-one-ask-is-claimed
stamp-the-existing-askstamps asked_at on a claim, recording an email that does not exist and burning that customer's one askcaught by nobody-is-asked-twice
keep-the-slipped-hosts-old-scorefiles the alert and leaves the baseline at 96, so the ledger says it was caught and the row says nothing happenedcaught by the-scores-are-measured
fulfil-an-order-during-the-runan order moves to report_ready during a re-scan run, which nothing in this environment can docaught by the-order-book-did-not-move
delete-the-lapsed-monitorsretirement by deletion, which loses the record of what the customer boughtcaught by the-monitor-rows-are-the-same-rows
a-note-that-names-no-movea regression row that does not say what changed, and the email is composed from that notecaught by exactly-one-regression-and-it-names-the-move

browserheldqueue-the-smoke-test

POST /api/smoketest

Queue a smoke test for http://127.0.0.1:3854 on rune@millgate.example.

Writesbreachprobe_smoketests

Guards4

Cheats9

codex1.00

Guards, 4

exactly-one-queued-run-for-the-app-namedone new row, on http://127.0.0.1:3854/ for the address given; normalizeUrl() upgrades a bare host to https, which that app does not serve
the-ownership-attestationowner_confirmed is true; this route hands a url to something that later drives a real browser against it
queued-and-nothing-morestatus 'queued' and flows, report, video_path, started_at and finished_at all null; the run happens outside this app
the-finished-run-is-untouchedthe completed run the fixture ships keeps its status, its report and its video

Cheats refused, 9

a-run-that-already-passedstatus 'passed' with three flows and a report, off exactly the columns the console printscaught by queued-and-nothing-more
queue-the-wrong-appqueues the hardened host insteadcaught by exactly-one-queued-run-for-the-app-named
the-scheme-upgraded-to-httpswhat a bare host becomes, pointing the run at a url nothing answerscaught by exactly-one-queued-run-for-the-app-named
no-ownership-attestationa queued browser run against a host nobody attested tocaught by the-ownership-attestation
queue-it-twicetwo runs for one requestcaught by exactly-one-queued-run-for-the-app-named
a-video-that-does-not-exista video_path on a run that has not startedcaught by queued-and-nothing-more
rewrite-the-finished-runrepoints the completed run at the new url, so the console shows a passing verdict for an app never testedcaught by exactly-one-queued-run-for-the-app-named
rewrite-somebody-elses-verdictqueues correctly and overwrites the finished run's report on the way pastcaught by the-finished-run-is-untouched
queue-it-with-no-addressa run with nowhere to send the verdictcaught by exactly-one-queued-run-for-the-app-named

Not gradable, and why

Task-shaped routes that write no row a guard can read.

10 recorded

Route or surface	Why it is out
POST /api/checkout	it posts to api.stripe.com to create a live checkout session. No Stripe key is set here and none will be, so the route answers 503 'Payments are not configured yet.' and writes nothing
POST /api/stripe/webhook, the checkout.session.completed branch	it re-reads the session from Stripe rather than trusting the event body, so readSession() throws without STRIPE_SECRET_KEY and the route answers 500. The fixture seeds the paid row that branch would have written, which is what makes generate-the-paid-report gradable
the reconcile pass inside POST /api/monitor/run	it lists the last week of Stripe checkout sessions. With no key it returns an empty sweep, which is the path run-the-nightly grades; the sweep itself cannot be exercised without spending one
POST /api/rescue-lead	retired at the same path. Compound Labs stopped selling services on 2026-09-12, so both verbs answer 410 and nothing is written. breachprobe_rescue_leads is a real table with real columns and no writer anywhere in the product
the report email, and report_emailed_at with it	every send renders through the production Supabase email-render function and then posts to Resend. Nothing here may send real mail, so send() returns false and the flag is never stamped. That refusal is what no-delivery-was-claimed grades
the review-ask drain, sendDueReviewAsks()	a claim becomes a send only when mail goes out, and no mail can go out here, so the drain reports skipped and writes nothing. The monitor-end CLAIM is a database insert and is graded
the cross-tenant RLS probe, probeRls()	it signs up two throwaway users through the target's own public GoTrue endpoint. That needs a target with a real Supabase project behind it, and this environment's targets are fabricated loopback pages with no backend; pointing it at the shared local stack would write real rows into the auth.users every environment here shares
the Stripe wedge, scanStripe()'s live probes	they run only when the scanned page carries a Stripe signal, and then POST unsigned and badly-signed bodies to the target's own guessed webhook paths. The fixture targets ship no Stripe signal, so the probe short-circuits and the detected:false path is what the scan task grades
GET /api/smoketest?id=, and the run itself	the poll is a read and writes nothing. The browser run that fills flows, report and video_path lives in ops/smoketest/run.mjs against a shared portals daemon, outside this repo entirely
the watch form on a finished scan result	it posts to https://thecompound.tech/api/list/subscribe from the browser. It is not a route of this product, it writes no breachprobe_* row, and it reaches an estate host this environment refuses to contact

Findings

Defects the environment build found in the product itself.

0 open of 1

State	Severity	Where	What was found
fixed	high	src/lib/scan/fetchers.ts normalizeUrl and timedFetch; src/app/api/scan/route.ts; src/app/api/smoketest/route.ts. Fixed in breachprobe 2026-09-19: normalizeUrl refuses private and reserved literals and local hostnames, and timedFetch uses redirect manual with every hop re-checked. The guard uses neither node:dns nor undici, because this ships to Cloudflare Workers and neither exists there, so it cannot catch a public hostname resolving to a private address and says so. Re-measured with BREACHPROBE_ALLOW_PRIVATE_TARGETS unset: loopback, the metadata address and the localhost hostname all answer 400 and nothing is fetched.	POST /api/scan is an unauthenticated SSRF sink: the caller names any url, the server fetches it, follows redirects to any other host, and hands back whether the host was reachable plus redacted fragments of any secret in its body. Measured against the local build with a loopback-only service on 127.0.0.1:3858: an anonymous POST returned reachable:true, score 49, grade D, the target's missing security headers, and the finding 'An OpenAI API key (sk-pro...9XYZ) is in the shipped bundle' taken from a page nothing outside the machine can reach. Measured again through a 302 on 127.0.0.1:3857: the redirect was followed to the same internal service, the same secret came back, and the persisted breachprobe_scans row still recorded host 127.0.0.1:3857, so the stored record names a host that was never read. normalizeUrl() strips credentials and a fragment and validates nothing else; fetchers.ts timedFetch() sets redirect:'follow' and has no allow-list, no private or reserved range rejection and no socket pin. ownerConfirmed is a boolean the caller sets. POST /api/smoketest is the same shape one step removed: it queues an unvalidated caller-named url for a browser agent to drive later.