Database
OfficePrism: Benchmarking Agents on Single-Domain Operations Across Enterprise Business Functions
arXiv cs.SE0 views
arXiv:2611.22262v1 Announce Type: cross Abstract: Broad workplace scores can mix competence within one business function with the extra cost of coordinating across functions and systems, making failures hard to localize. OfficePrism evaluates this prerequisite through 420 single-domain enterprise tasks spanning seven business functions under shared, benchmark-defined L1--L3 task-structure levels. Each task runs in a fresh environment combining a shared synthetic organization with one function-specific domain and yped state access. Our pipeline uses grounded records, business rules, and tool schemas to jointly construct a Chinese-language ticket, a hidden executable reference witness, and an expected response, followed by automated filtering and human curation. The same task set is evaluated through full-schema and search-and-dispatch packages. Across 124 model--harness--package configurations, the per-case joint natural-language-and-database pass rate ranges from 22.6% to 76.4%, with a median of 67.9%. L1 is highest in all 124 configurations, and L2 exceeds L3 in 122. Among 50 configurations sharing the same harness and model labels across both packages, search scores higher in 30, ties in one, and scores lower in 19.
Source:https://arxiv.org/abs/2611.22262PreviousDiscontinuity-Aware Solvers for Vapor-Compression CyclesNextWhen the World Is the Test Suite: Agentic Intelligence for the Physical World
Related posts
Designing for Chaos: Implementing Circuit Breakers and Bulkheads in Go
0dev.to Go
Dashboard or spreadsheet: the honest test for when you need software
0dev.to Database
Database and PostgreSQL
0dev.to Database
Copy only the missing rows from SQL Server to PostgreSQL from the command line
0dev.to Database
Your PostgreSQL UPDATE ... FROM Never Finishes: The Repeated Target Table That Turns It Into a Cross Join
0dev.to Database
Nikolay Samokhvalov: The hidden costs of CREATE TEMP TABLE: catalog bloat and LWLock:SInvalWrite
0Planet PostgreSQL