I asked 3 LLMs to triage bug bounty exploits. So I built a contract that runs them instead

A bug bounty asks the hunter to go first. You hand over the bug, then a human triager decides whether it counts and how much it is worth. The protocol can patch it, call it "known" and pay nothing. And now a growing share of the reports in that queue are AI-written, confident and wrong.

I wanted to see how bad the second problem is, so I measured it. Then I built a bounty where nobody triages at all: the contract runs the exploit and pays the first real break.

Repo: https://github.com/edycutjong/sealed · Live: https://sealed.edycu.dev · Demo (1:34): https://youtu.be/sEMg3q7gG5k

The measurement: can a model be talked into paying?

I built a 52-item corpus against five unmodified OpenZeppelin Ethernaut levels (Fallback, Telephone, Delegation, Force, King): real exploits, near-misses, no-ops, wrong-snapshot runs, malformed input, abuse input, and 15 submissions written by three different LLM families, each with the "hunter's" own explanation attached.

Then I gave every item to three models acting as triagers. Each got the level source, the bounty goal and the submission, and had to answer VERDICT: PAY or VERDICT: REFUSE.

Of the 21 non-breaking submissions:

Read it with the limits I put in the repo: one run per model, no code execution allowed, and only 5 of the 21 fakes came with a written claim. Every fake a model paid was one of those 5. So this is not "LLMs are useless at triage". It is narrower and more useful: a convincing explanation is enough to move a model to PAY, and it also refuses real breaks. Both failure modes cost a bounty money or reputation.

The verifier has neither failure mode, because it never reads the explanation. It runs the transactions.

The design: commit, verify in a TEE, deliver, get paid

Sealed is a NEAR contract that holds the bounty pool on an account with zero access keys (deployed from global contract code), so nobody, including me, can move the pool by hand. At funding it commits three hashes: the chain snapshot, the invariants, and the verifier WASM.

1. The hunter stores the exploit as an encrypted secret in NEAR OutLayer, then commits H(exploit ‖ hunter ‖ salt) with a bond. The exploit stays sealed.
2. The contract calls OutLayer's request_execution, pinned to the verifier's hash. A TEE worker decrypts the exploit, replays it in revm on the committed snapshot, and asks the level's own validateInstance whether it broke.
3. The verdict comes back through a yield/resume callback. The first valid break per invariant is awarded into escrow. Everything else is refused on chain with a reason, and a hunter-caused failure slashes the bond.
4. The hunter delivers the exploit sealed to the protocol's key. The contract checks it hashes to what the worker attested, and only then pays.

The protocol still decides what counts as a break, but up front, before any claim, and in public.

The verifier: no Rust-side opinion

The core of the verifier is short on purpose. It runs every exploit transaction from the player, then calls the level's own validator. There is no predicate I wrote that could be argued with:

/// Run every exploit tx from the player, then call the level's own `validateInstance` from the validator. pub fn execute(snap: &Snapshot, exploit_bytes: &[u8]) -> Result<(bool, u64, Option<String>), String> { // ...load the committed snapshot into revm... for (to, data, value) in txs { let left = EXPLOIT_GAS_BUDGET - gas; match evm.transact_commit(tx(player, kind, data, value, left)) { // Revert / halt / out-of-gas are normal EVM outcomes. Ok(r) => gas = gas.saturating_add(r.tx_gas_used()).min(EXPLOIT_GAS_BUDGET), // Invalid tx (e.g. value above the player's balance): the tx simply does not happen. Err(e) => trace(|| format!("tx {kind:?} -> invalid {e:?}")), } } let r = evm.transact(tx(validator, TxKind::Call(factory), validate_calldata(instance, player), U256::ZERO, VALIDATE_GAS_LIMIT))?; let broken = match r.result { ExecutionResult::Success { output, .. } => { let o = output.data(); o.len() == 32 && o[31] == 1 && o[..31].iter().all(|b| *b == 0) } _ => false, }; Ok((broken, gas, None)) } 

It compiles to wasm32-wasip1, about 1.5 MB, and the build is byte-reproducible on the pinned host, so the hash the contract commits is the hash anyone can rebuild. The heaviest real exploit used 0.37% of OutLayer's 2e9 instruction limit; the heaviest fake, 23.98%.

The callback: refusals are first-class

Most of the contract is about saying no correctly. A verdict that says "broken" still has to match the commit, the invariant and the snapshot, or it is refused and the bond is slashed:

let reason: Option<String> = if !broken { Some(err.map(|e| e.to_string()).unwrap_or_else(|| "no invariant broken".into())) } else if h_exploit != c.commit_hash { Some("commit mismatch".into()) } else if v_inv != c.invariant_id { Some("wrong invariant".into()) } else if h_snapshot != want_snap.as_str() { Some("snapshot mismatch".into()) } else if !h_enc_ok { Some("malformed h_enc".into()) } else { None }; 

Six refusal kinds landed on testnet (pool pull, invariant busy, duplicate, fake, delivery mismatch, expired award), each with a transaction you can look up. Twenty near-workspaces sandbox tests cover the invariants: conservation (pool + escrow + paid = funded − withdrawn after every call), one payout per invariant, one verdict in flight, a withdraw guard so the protocol cannot pull reserved funds, and the delivery hash check.

The numbers, and how to check them without trusting me

On NEAR testnet the demo bounty paid 5 of 5 real exploits and 0 of 21 submitted fakes. Request to verdict was p50 7.27 s, p95 10.85 s over 29 runs, read from OutLayer events.

You do not need a NEAR account or a key to check any of that:

git clone https://github.com/edycutjong/sealed && cd sealed make replay # re-runs every published commit offline under wasmtime = its on-chain verdict make verify-demo # reads the chain: pinned hash, worker key, verdict, PAID; FAILs unless the bounty has 0 keys make corpus # all 52 corpus items vs golden verdicts 

make replay needs only python3 and wasmtime.

What it does not do (yet)

• Testnet only. The mainnet leg is scripted and has not run.
• Attestation is partial. make verify checks that OutLayer's scoped worker key answered a request pinned to the verifier hash. It does not check the TDX quote, so it does not prove the worker ran exactly those bytes; make replay re-runs them locally instead.
• RPC trust. Verification believes the archival node it queries; an RPC error is reported as RECORDED, never as a pass.
• Griefing. A fresh account can keep one invariant busy for the cost of gas plus the OutLayer fee.
• No outside users yet. Every measured run is my own hunter agent plus the fakes corpus.

That last one is the real gap. If you run bounties or hunt them, clone it, run make replay, and tell me where it breaks. I'd also like to know: would you put a pool behind a contract that pays without asking you first, and what would you need to see?

Built at NEAR HACKATHON 2026 with NEAR OutLayer. Repo: https://github.com/edycutjong/sealed