AI & ML
MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents
arXiv cs.MA0 views
arXiv:2503.01935v2 Announce Type: replace Abstract: Large Language Models (LLMs) have shown remarkable capabilities as autonomous agents, yet existing benchmarks either focus on single-agent tasks or are confined to narrow domains, failing to capture the dynamics of multi-agent coordination and competition. In this paper, we introduce MultiAgentBench, a comprehensive benchmark designed to evaluate LLM-based multi-agent systems across diverse, interactive scenarios. Our framework measures not only task completion but also the quality of collaboration and competition using novel, milestone-based key performance indicators. Moreover, we evaluate various coordination protocols (including star, chain, tree, and graph topologies) and innovative strategies such as group discussion and cognitive planning. Notably, gpt-4o-mini reaches the average highest task score, graph structure performs the best among coordination protocols in the research scenario, and cognitive planning improves milestone achievement rates by 3%. Code and datasets are public available at https://github.com/MultiagentBench/MARBLE.
Source:https://arxiv.org/abs/2503.01935PreviousCBS with Continuous-Time RevisitNextUncertainty, bias and the institution bootstrapping problem
Related posts
Windows’ Baseline Protection Can Simplify Security, but There Could Be Pitfalls
0CNET News
Chase Hit by Third-Party Data Breach Unrelated to Recent Outage
0CNET News
OpenAI’s GPT-6.2 and Sol Are Getting Backlash From Early Testers
0CNET News

An engineer didn’t message his AI for 22 of 44 days. It built an Android app anyway.
0The New Stack
When hackers target grocery stores
0Phys.org Tech
Elastic and HoneyHive join forces to advance AI observability and evaluation
0Elastic Blog