AI & ML
Disagreement-Guided Agentic Data Curation for Multimodal Safety
arXiv cs.MA0 views
arXiv:2607.14256v2 Announce Type: replace-cross Abstract: Written safety policies in multimodal settings define categories whose visual boundaries span many contexts. We study disagreement as a signal for identifying useful synthetic boundary cases. Our pipeline generates policy-conditioned images, selects cases with target--rater or inter-rater disagreement, conditionally escalates unresolved cases, and retrieves the resulting rater-labeled examples as in-context demonstrations. On a failure-enriched multimodal safety benchmark, policy grounding reduces false negative rate (FNR) from 0.875 to 0.413 and raises F1 from 0.220 to 0.731; retrieval from 50 disagreement-selected examples further reduces FNR to 0.245, compared with 0.297 from a 350-example Regular pool. The selected memory also improves a different target-model family, with a smaller gain on a proprietary benchmark with a refined policy. In a blinded human study, experts disagree on 24% of disagreement-guided images and 17% of baseline-curated images. These findings support traceable, disagreement-guided curation for multimodal safety evaluation and example selection.
Source:https://arxiv.org/abs/2607.14256PreviousColosseum: Auditing Collusion in Cooperative Multi-Agent SystemsNextHow to Receive Focused Beams in Near-Field MIMO? Analytical Combiners and Array Size Optimization
Related posts
How to detect user frustration in chatbot conversations
0dev.to
Mình Đã Biến Gemini Thành Trợ Lý Cá Nhân Như Thế Nào (Lịch, Drive, Gmail và Những Lần Suýt Toang)
0dev.to
Im looking fr any free ai models / tools are there?, leaVE UR Lights💡
0dev.to
I checked 10 SEO agency websites for AI-search readiness. The real bugs were the boring ones.
0dev.to

How much GPU memory does an LLM need? The whole sum, worked out on real cards
0dev.to

Routing Upscaling Jobs by Output Size, Not Input Size
0dev.to