General
Two is better than one: optimizing binary rating scales for online platforms
arXiv cs.MA0 views
arXiv:2611.23626v1 Announce Type: new Abstract: Online platforms often struggle with inflated, uninformative user ratings. This issue can be mitigated by better design of the data collection mechanism, for example through changing the phrasing of user questions. Previous work optimizes a single rating scale to best distinguish the inherent qualities of items under binary feedback. In this paper, we demonstrate the power of utilizing different rating scales for different users. First, we show that a designer can always split a single rating scale into two to improve item rankings. Next, we formally characterize the optimal design of multiple scales as a series of 0-1 step functions whose threshold values are evenly spaced. Finally, we consider an adaptive model where subsequent rating scales can depend on the responses from previous users and items are ranked by their full sequence of responses, and show that adaptivity leads to exponentially greater distinguishing power. All in all, our theoretical results show that using multiple rating scales significantly improves item rankings on platforms.
Source:https://arxiv.org/abs/2611.23626PreviousMANGO Arena: Same Seeds, Different Chains Evaluating LLM Agents in a Disrupted Supply ChainNextResilient Federated LQR over Noisy and Lossy Uplinks via Scalar Communication
Related posts
If nobody writes it down, it did not get decided
0dev.to Productivity
Micrograph B-factor estimation via Wilson statistics
0arXiv eess.IV
Greedy Vector Approximation Based on Rotation for Multiplierless Complex Products
0arXiv eess.SP
Toward Massive MIMO HAPS Networks: Antenna Architectures and Design
0arXiv eess.SP
Deep-Learning-Assisted Full Waveform Inversion for Attenuation Reconstruction in Ultrasound Computed Tomography
0arXiv eess.SP
Sparse Fault Detection in Linear Systems: Equivalence of GLR Tests and $\ell_1$-regularized Least Squares
0arXiv stat.TH