AI & ML
RaGCD: Reliability-Aware 2D-3D Multimodal Generalized Category Discovery
arXiv eess.SP0 views
arXiv:2611.13827v1 Announce Type: new Abstract: Generalized Category Discovery (GCD) aims to recognize known categories while discovering novel ones from partially labeled data. Existing GCD studies primarily focus on unimodal observations or image-text interactions, leaving the complementarity between visual appearance and explicit 3D geometry underexplored. To exploit complementary visual appearance and spatial geometry, we investigate GCD with paired images and point clouds. However, the relative reliability of these modalities varies across samples, so unreliable evidence can compromise both multimodal fusion and cross-modal knowledge transfer. This challenge is particularly pronounced for novel categories, where labeled supervision is unavailable. To address it, we propose RaGCD, a reliability-aware framework for 2D-3D multimodal GCD. Specifically, Reliability-aware Modality Allocation (RMA) combines prediction-derived reliability cues with modality-specific features to adaptively allocate image and point-cloud contributions at the channel level. Reliability-conditioned Alignment (RCA) further reweights feature-level and prediction-level alignment and selects reliable observations for prototype-level alignment, regulating cross-modal interaction according to sample reliability. Extensive experiments on ModelNet40, Washington RGB-D, and ShapeNet-ViPC demonstrate that RaGCD outperforms the compared methods in overall and novel-category clustering accuracy. Ablation studies support the complementary roles of RMA and RCA, while controlled corruption experiments show that modality allocation shifts toward the less corrupted modality. These results support the effectiveness of reliability-aware interaction for category discovery from paired appearance and geometry.
Source:https://arxiv.org/abs/2611.13827PreviousResidual-Closed-Loop SAGE-Brent Estimation for Unknown-Order Wideband Underwater Acoustic ChannelsNextA Faster and Efficient Indoor User Localization Using Bistatic MIMO Radar
Related posts
I wanted one screen that said whether a kids' app was worth installing
0dev.to

Before I Could Build GPT, I Had to Teach a Machine to Learn
0dev.to
Cogent and OpenAI build cyber defense factories for enterprises
0OpenAI Blog
ChatGPT Plus for veterans and servicemembers
0OpenAI Blog
Hitting 200 Strokes: What the Signature Maker Does at Its Stroke Limit
0dev.to
A Lightweight, Open-Source Network Digital Twin Blueprint for IoT-Edge Environments
0arXiv cs.NI