This is a Plain English Papers summary of a research paper called OpenDS-World: Benchmarking Multimodal Evidence Search on the Live Web. If you like these kinds of analyses, you can find more AI and machine-learning research on AIModels.fyi or follow us on Twitter.
OpenDS-World tests research on the live Web
OpenDS-World contains 200 tasks that test whether an agent can trace clues from an audiovisual video anchor through independent Web sources to a unique factual answer. Unlike a static retrieval benchmark, each task requires the agent to decide what to search next, access evidence in different forms, and connect it across a multi-step chain.
The benchmark targets three linked sources of difficulty: diverse modalities, heterogeneous resource formats, and interactive access. Its evidence chains cover four modalities, 42 formats, and seven access types; 82.5% of tasks involve at least two modalities, and 95.5% require more than one access type. The task set spans culture, daily life, science, economy, history, nature, society, and technology.
An OpenDS-World example connects audiovisual identification with virtual-tour navigation, document inspection, and structured-data retrieval across Web sources. It demonstrates why solving a task requires evidence acquisition across multiple formats and access mechanisms....
