AI & ML![Cleaning or Recalling? Knowledge Dependence in LLM-based Data Cleaning [Experiment, Analysis & Benchmark]](/storage/uploads/covers/auto/p184942.svg)
Cleaning or Recalling? Knowledge Dependence in LLM-based Data Cleaning [Experiment, Analysis & Benchmark]
arXiv cs.DB0 views
arXiv:2611.08106v1 Announce Type: new Abstract: High scores on public data-cleaning benchmarks can reflect knowledge of familiar entities, making it difficult to assess how well an LLM repairs unfamiliar data. We present CoRE (Cleaning or Recalling), a protocol comparing public benchmarks with fresh real records and paired name surrogates built from English words or syllables. The surrogates retain error positions and non-identity values while transferring the benchmark's error mechanisms. Across 21 controlled tables and eight LLMs, repair performance varies substantially with entity names on Hospital and, for commercial models, Movies. Renaming removes some information needed for repair, such as a film's identifier mapping. It also affects repairs whose candidates remain available: replacing entity names with word-based surrogates lowers DeepSeek-V4-Pro's year accuracy from 91% to 18%. On Hospital and Movies, commercial models score higher on fresh records than on word-based surrogates. A screened continuation probe finds no positive evidence of benchmark-row memory in the tested models. Familiar facts can conflict with the task. Even with explicit table-evidence instructions, commercial models rewrite assigned directors at rates of 0.000-0.404 on known films. Reference scoring can also disagree with external records. On Rayyan, an ISSN correction matching a title-matched PubMed record receives no credit, whereas 94% of TabClean's credited repairs reproduce the reference date transformation. We provide controlled tables, scoring code and materials for offline replay.
Source:https://arxiv.org/abs/2611.08106PreviousPer-Query Recall Guarantees for Approximate Nearest Neighbor Search in Dynamic Vector DatabasesNextCascadeFilter: Cost-Efficient Semantic Filtering with Joint Threshold Selection
Related posts

A New Community For In-House UX People
0Smashing Magazine
A flaw in ChatGPT, a lesson for our front-ends
0dev.to
The Cheapest LLM API, by What You Actually Run
0dev.to
Mbodi AI (YC P25) Is Hiring Research and Robotic Engineers
0Hacker News

Why responsive stores need orchestration, not more screens
0TechRadar

The Audiolab 6000CDT MKII is a new CD transport — further proof that the physical media revival is in full swing
0TechRadar