AI & ML
On the Validity of Fisher Randomization Tests under the Weak Null
arXiv stat.TH0 views
arXiv:2611.24821v1 Announce Type: cross Abstract: Consider the problem of testing the Neyman weak null of no sample average treatment effect $H_{0N}:\tau = 0$ in randomized experiments with binary outcomes. Although the Fisher Randomization Test (FRT) is typically justified under Fisher's sharp null of no individual treatment effect, we show that it is finite-sample-valid for the weak null $H_{0N}$ in three common combinations of study design and test statistic: 1) balanced completely randomized experiments with the difference-in-means test statistic, 2) matched pairs experiments with the same test statistic, and 3) balanced Bernoulli randomized experiments with the centered Horvitz-Thompson test statistic. As an immediate corollary, Fisher's exact test in a balanced completely randomized experiment is valid under the weak null, and so is McNemar's test in a matched pairs experiment. Similar results hold for the one-sided null $H_0:\tau \leq 0$. Beyond balanced designs, we show that twice the one-sided FRT $p$-value with the same test statistics is valid under $H_{0N}$. In simulation experiments, the FRTs compare favorably to other exact procedures.
Source:https://arxiv.org/abs/2611.24821PreviousAnytime-valid goodness-of-fit test by $e$-spacingsNextAssessing model calibration with boosting trees
Related posts
Manuj Mittal on Why “Most Tokens Aren’t Securities” Leaves Crypto With More Questions Than Answers
0HackerNoon
I Found My Own Website Blocking ChatGPT. Then I Checked 640 Others.
0HackerNoon

Small Language Models, Not Giant LLMs, Should Power the Next Generation of AI Agents
0HackerNoon
Building a Browser-Based Image Editor With Web Workers and OffscreenCanvas
0HackerNoon

The TechBeat: 178 Organizations Served Malware Through Optimizely. What Have the Companies Said? (11/19/2026)
0HackerNoon

I Sent 12 AI Shoppers Through a Broken Checkout, but Only One Bought Anything
0HackerNoon