AI & ML![Cleaning or Recalling? Knowledge Dependence in LLM-based Data Cleaning [Experiment, Analysis & Benchmark]](/storage/uploads/covers/auto/p184942.svg)
Cleaning or Recalling? Knowledge Dependence in LLM-based Data Cleaning [Experiment, Analysis & Benchmark]
arXiv cs.DB0 views
arXiv:2611.08106v1 Announce Type: new Abstract: High scores on public data-cleaning benchmarks can reflect knowledge of familiar entities, making it difficult to assess how well an LLM repairs unfamiliar data. We present CoRE (Cleaning or Recalling), a protocol comparing public benchmarks with fresh real records and paired name surrogates built from English words or syllables. The surrogates retain error positions and non-identity values while transferring the benchmark's error mechanisms. Across 21 controlled tables and eight LLMs, repair performance varies substantially with entity names on Hospital and, for commercial models, Movies. Renaming removes some information needed for repair, such as a film's identifier mapping. It also affects repairs whose candidates remain available: replacing entity names with word-based surrogates lowers DeepSeek-V4-Pro's year accuracy from 91% to 18%. On Hospital and Movies, commercial models score higher on fresh records than on word-based surrogates. A screened continuation probe finds no positive evidence of benchmark-row memory in the tested models. Familiar facts can conflict with the task. Even with explicit table-evidence instructions, commercial models rewrite assigned directors at rates of 0.000-0.404 on known films. Reference scoring can also disagree with external records. On Rayyan, an ISSN correction matching a title-matched PubMed record receives no credit, whereas 94% of TabClean's credited repairs reproduce the reference date transformation. We provide controlled tables, scoring code and materials for offline replay.
Source:https://arxiv.org/abs/2611.08106PreviousPer-Query Recall Guarantees for Approximate Nearest Neighbor Search in Dynamic Vector DatabasesNextCascadeFilter: Cost-Efficient Semantic Filtering with Joint Threshold Selection
Related posts
‘Mathematics is more than a mechanical process’ — why I think the field will survive OpenAI’s disruption
0Nature News
Preprint platform offers to pay researchers to review papers, but questions remain
0Nature News

I’ve reviewed almost every Android foldable this year — here’s how the iPhone Duo compares
0The Verge
Physicl raises $10M to make the digital world real enough for robots
0TechCrunch

Kubernetes, mark II? Why Google’s giving away gVisor, a code sandbox used by Anthropic and OpenAI
0The New Stack
AI helps find better recipes for 3D-printing titanium parts
0Phys.org Tech