If you maintain an open source repo, you have seen this PR. The description is tidy, the diff is plausible, and the author cannot tell you what it does when the input is empty.

AI made opening a pull request free and reviewing one expensive. PullViva is my attempt to move some of that cost back to the person who opened it.

In one line: a required GitHub check that quizzes first-time contributors on their own PR. It is called PullViva, and it turns green only when the author passes.

How it works

  1. A first-time contributor opens a PR. Maintainers and bots are exempt by default.
  2. PullViva posts a required check on the PR that links to a quiz, so the merge waits.
  3. The author signs in with GitHub. A pass is bound to that GitHub account.
  4. Cloudflare Turnstile runs first, so bots never trigger a model call.
  5. An LLM reads the actual diff and writes five multiple-choice questions: what changed, why, an edge case, a risk, how it was verified.
  6. The author answers. Pass, and the check goes green. Fail, and they get a hint, a 30 minute cooldown and a fresh question set. There are three attempts.

Here is the required check after a contributor passes:

A pull request with the required PullViva check showing Verified

The check reads "PullViva - Verified" and is marked Required. This image comes from the repo README, which describes it as a 2x render of the same pages the app serves, rebuilt in HTML from a real fork pull request. Until the author passes, the same check shows "Answer a few questions about this PR (about 3 min)".

And this is the quiz the author sees. The questions come from the pull request's own diff. This one is from a demo PR that documents a retry policy:

PullViva quiz page: question 1 asks what retry behavior the new document describes, with four multiple-choice answers

Design choices

  • Questions come from the diff, not a generic bank. A quiz about "what is a pull request" proves nothing. A quiz about this change at least requires opening it.
  • Enforced by GitHub, not by the contributor's laptop. It is a required status check on your ruleset, pinned to the app. There is no --no-verify flag that skips it.
  • One account, one commit. A pass is bound to the PR author's GitHub account. Push new commits and the check is evaluated again.
  • The answer key stays on the server. Questions are written after sign-in and the human check. The key is never in the PR or on the page.
  • You can always override. Anyone with write access can click Mark as passed on the check.
  • Configurable. Who is quizzed, number of questions, pass mark, attempts, difficulty, and exemptions by label, path, size or user, from a dashboard or a YAML file. For PRs with several contributors you can optionally quiz everyone who committed.
  • Bring your own model. Anthropic's native API, or anything that speaks the OpenAI format. You set a base URL, a model name and one key.

PullViva feature grid: enforced by GitHub, one account and one commit, answers stay on the server, you can always override, configurable, open source

The honest limits

A quiz filters. It does not prove. Someone with a chatbot in another tab can still pass. What PullViva does is raise the cost of a drive-by pull request from zero to a few minutes of attention, tie each pass to one GitHub account, and leave a record you can read. For a maintainer that gap is the point: a lot of low-effort PRs stop at zero cost.

Open source and self-hosting

The core is Apache-2.0. The gate, quiz, config and dashboard are all in the repo. Self-host it on Cloudflare Workers with D1, your own GitHub App and your own model key. The repo README has the full walkthrough, including the wrangler steps and the GitHub App permissions.

There is also a hosted version that adds billing, quotas and plan limits:

  • Free: public repos, up to 3 active repos, 25 quizzes a month.
  • Pro: $9 a month for 1,000 quizzes, any repos for the owner, private repos, up to 10 questions, a 100% pass mark and custom prompts.

The Ar9av/pullviva repository page on GitHub: public, Apache-2.0, topics include github-app, cloudflare-workers and llm

Try it

I would like to hear from maintainers. What would a quiz have to ask before you trusted it as a signal? And which exemptions would you need?