General
Two is better than one: optimizing binary rating scales for online platforms
arXiv cs.MA0 views
arXiv:2611.23626v1 Announce Type: new Abstract: Online platforms often struggle with inflated, uninformative user ratings. This issue can be mitigated by better design of the data collection mechanism, for example through changing the phrasing of user questions. Previous work optimizes a single rating scale to best distinguish the inherent qualities of items under binary feedback. In this paper, we demonstrate the power of utilizing different rating scales for different users. First, we show that a designer can always split a single rating scale into two to improve item rankings. Next, we formally characterize the optimal design of multiple scales as a series of 0-1 step functions whose threshold values are evenly spaced. Finally, we consider an adaptive model where subsequent rating scales can depend on the responses from previous users and items are ranked by their full sequence of responses, and show that adaptivity leads to exponentially greater distinguishing power. All in all, our theoretical results show that using multiple rating scales significantly improves item rankings on platforms.
Source:https://arxiv.org/abs/2611.23626PreviousMANGO Arena: Same Seeds, Different Chains Evaluating LLM Agents in a Disrupted Supply ChainNextResilient Federated LQR over Noisy and Lossy Uplinks via Scalar Communication
Related posts
SakaQL: A new general-purpose relational programming language
0Hacker News New
Bram Stoker's Notes for Dracula
0Hacker News New
Show HN: Postdate – share a link now that opens at a time you choose
0Hacker News New
The New Era of Wikimedia Traffic: Thinking Beyond Pageviews
0Hacker News New
Artificial Intelligence and the Production of Scientific Knowledge
0Hacker News New
Ask HN: When do we stop switching and start resonating?
0Hacker News Ask