AI & ML
FUSEMOS: Perceptual Evaluation of Text-to-Music Generation with Dual-Encoder Fusion and Ranking-Aware Composite Loss
arXiv eess.AS0 views
arXiv:2611.07562v1 Announce Type: new Abstract: Perceptual evaluation of generated music is essential for developing text-to-music (TTM) generation systems. Existing automatic music perceptual evaluation methods rely solely on the contrastive language audio pretraining (CLAP) model to extract both audio and text features for predicting human-annotated mean opinion scores (MOS), particularly for overall musical impression (OMI) and textual alignment (TA). However, CLAP is limited in capturing fine-grained musical attributes such as timbre and expressiveness \emph{etc}. To address this, we propose FUSEMOS, a dual-encoder fusion framework that integrates CLAP with MERT, a self-supervised model pretrained on large scale music audio, to better capture both high level semantic alignment and detailed musical characteristics for more accurate OMI and TA estimation. Furthermore, to overcome the limitations of conventional regression losses in measuring human preferred relative ranking of music segments, we design a novel ranking-aware composite loss function. Experiments conducted on the MusicEval benchmark demonstrate that our approach outperforms the existing SOTA methods in both mean squared error and rank correlation metrics by a large margin.
Source:https://arxiv.org/abs/2611.07562PreviousA Unified Multi-Encoder Framework for Music Lyric Intelligibility Assessment: Bridging Invasive and Non-Invasive ParadigmsNextAuditory and Visual Virtualization in 6DoF XR: Effects on Plausibility and Dynamic Auditory Distance Perception
Related posts
AI for scientific discovery: How researchers use Claude
0Hacker News
FAA Begins Retiring Decades Old Floppy Disks from Air Traffic Control
0Hacker News
WARN Act and Child Support — What Happens to Support Payments After a Layoff Notice
0dev.to Data
1,084 Mercado Libre Cards Told Me How Many Were Sold. Between Them They Used Ten Numbers
0dev.to Data
Three traps when an AI assistant reads insider trades from Form 4s
0dev.to Data
OpenAI's Images 2.6 Brings GIF Creation and Faster Editing to ChatGPT
0dev.to News