AI & ML
Squoosh Bench: Evaluating UX-1 on 318 Historical Website A/B Tests
arXiv cs.HC0 views
arXiv:2611.25200v1 Announce Type: new Abstract: Experimentation teams must decide which proposed website changes to test with live traffic. We evaluated whether forecasts from simulated shoppers could inform that choice. We screened 412 historical website A/B tests and evaluated four systems on the 318 that ran longer than 21 days. Each system predicted an original win, a variant win, or an inconclusive result. Page roles were masked and recorded outcomes were withheld during inference. Squoosh UX-1, a model fine-tuned on human browsing data, simulated 1,000 shoppers per arm and correctly classified 267 of 318 outcomes (84.0%, 95% Wilson interval 79.5-87.6%, treating experiments as independent). Qwen3 Next 80B, the strongest general-purpose configuration on outcome accuracy, correctly classified 196 of 318 (61.6%). UX-1 favored the variant in 126 experiments. In the live tests, 118 of those variants won (93.7%), four lost, and four had inconclusive results. The remaining 192 experiments included 19 additional variant wins.
Source:https://arxiv.org/abs/2611.25200PreviousOpen Response Explorer: a Structured Collaborative Writing System for Peer-Informed Open-Response AssignmentsNextTowards Restorative Urban Spaces: An Exploratory Field Study of Nature Videos on Public Multi-Display Signage
Related posts
VESTA: Visual Exploration with Statistical Tool Agents
0arXiv stat.CO
An interpretable semiparametric Bayesian machine learning model for high-dimensional circulating epigenetic signatures of multiple myeloma
0arXiv stat.CO
Quantum Nondecimated Wavelet Transform: Theory, Circuits, and Applications
0arXiv stat.CO
dapper: Data Augmentation for Private Posterior Estimation in R
0arXiv stat.CO
HAVAPlot: A Web-Based Scientific Visualization and Post-Processing Platform
0arXiv stat.CO
ISC Stormcast For Friday, November 20th, 2026 https://isc.sans.edu/podcastdetail/10188, (Fri, Nov 20th)
0SANS ISC