AI & ML
LIC-Audit: How Comparable Are Reported Gains in Learned Image Compression?
arXiv eess.IV0 views
arXiv:2611.23419v1 Announce Type: new Abstract: Learned image compression (LIC) papers often report BD-rate improvements of only a few percent. They are measured against VTM reference software whose version keeps changing: the claims we audit name 7 versions, 5 in 2025 papers alone. But how comparable are these numbers across papers, and what evidence should future papers give for such small gains? We introduce LIC-Audit, an analysis of 353 released rate-distortion curves from 36 repositories, and a controlled re-evaluation of eight public codec releases with 28,193 encode-decode records. We identify three problems. First, different VTM reference runs change BD-rates by up to 3.56 percentage points, and on CLIC Pro published numbers order 10 of 55 method pairs differently from a common computation; comparing codecs directly over a fixed quality range removes this dependence. Second, releases use different and mostly undocumented conventions for counting bits, computing PSNR, and padding images. Under consistent measurement, the GLIC method's 1.00% lead over the HPCM method on Tecnick becomes a 1.43% deficit, changing only the padding reverses the comparison of HPCM and CMIC, and most gaps narrow when MS-SSIM replaces PSNR. Third, average curves hide how much results vary from image to image, so whether a small gain is statistically significant depends on the assumed variation. An optimistic assumption calls 12.0% of comparisons between equally good codecs significant on test sets as small as Kodak, instead of 5%; a cautious one leaves 18 of 45 differences between recent codecs undecided, yet per-image results show 12 of them to be significant. Beyond auditing past results, LIC-Audit provides a versioned evaluation protocol, a public leaderboard website with a scoring tool, source-linked benchmark data, per-image measurements, and reporting guidelines, to help future LIC papers make comparable and verifiable claims of progress.
Source:https://arxiv.org/abs/2611.23419PreviousStreaming Generative Video Compression with Embedding Alignment and In-Context PropagationNextGaussian Splatoms: Direct Atomic Structure Identification in Electron Tomography
Related posts
Im looking fr any free ai models / tools are there?, leaVE UR Lights💡
0dev.to
I checked 10 SEO agency websites for AI-search readiness. The real bugs were the boring ones.
0dev.to

How much GPU memory does an LLM need? The whole sum, worked out on real cards
0dev.to

Routing Upscaling Jobs by Output Size, Not Input Size
0dev.to
A ten-second Flutter pause with a visible Skip button
0dev.to UX

Thống kê lịch sử đối đầu Aston Villa - Sunderland: Đại diện Midlands áp đảo toàn diện
0dev.to Data