AI & ML
Disagreement-Guided Agentic Data Curation for Multimodal Safety
arXiv cs.MA0 views
arXiv:2607.14256v2 Announce Type: replace-cross Abstract: Written safety policies in multimodal settings define categories whose visual boundaries span many contexts. We study disagreement as a signal for identifying useful synthetic boundary cases. Our pipeline generates policy-conditioned images, selects cases with target--rater or inter-rater disagreement, conditionally escalates unresolved cases, and retrieves the resulting rater-labeled examples as in-context demonstrations. On a failure-enriched multimodal safety benchmark, policy grounding reduces false negative rate (FNR) from 0.875 to 0.413 and raises F1 from 0.220 to 0.731; retrieval from 50 disagreement-selected examples further reduces FNR to 0.245, compared with 0.297 from a 350-example Regular pool. The selected memory also improves a different target-model family, with a smaller gain on a proprietary benchmark with a refined policy. In a blinded human study, experts disagree on 24% of disagreement-guided images and 17% of baseline-curated images. These findings support traceable, disagreement-guided curation for multimodal safety evaluation and example selection.
Source:https://arxiv.org/abs/2607.14256PreviousColosseum: Auditing Collusion in Cooperative Multi-Agent SystemsNextHow to Receive Focused Beams in Near-Field MIMO? Analytical Combiners and Array Size Optimization
Related posts

Introducing policy-based routing for AWS Transit Gateway
0AWS Networking Blog

Inside Amazon Connect Talent: The science of structured AI interviews
0AWS Machine Learning Blog

Amazon SageMaker AI now detects latency drift, idle GPUs, and missed optimizations on your inference endpoints
0AWS Machine Learning Blog
Run your whole selling business in AWS Quick
0AWS Machine Learning Blog
How the BMW Group governs hundreds of agents across thousands of developers
0AWS Machine Learning Blog

Reducing Rollout Bottlenecks in Reinforcement Learning with NVIDIA Dynamo
0NVIDIA Developer Blog