AI & ML
FUSEMOS: Perceptual Evaluation of Text-to-Music Generation with Dual-Encoder Fusion and Ranking-Aware Composite Loss
arXiv eess.AS0 views
arXiv:2611.07562v1 Announce Type: new Abstract: Perceptual evaluation of generated music is essential for developing text-to-music (TTM) generation systems. Existing automatic music perceptual evaluation methods rely solely on the contrastive language audio pretraining (CLAP) model to extract both audio and text features for predicting human-annotated mean opinion scores (MOS), particularly for overall musical impression (OMI) and textual alignment (TA). However, CLAP is limited in capturing fine-grained musical attributes such as timbre and expressiveness \emph{etc}. To address this, we propose FUSEMOS, a dual-encoder fusion framework that integrates CLAP with MERT, a self-supervised model pretrained on large scale music audio, to better capture both high level semantic alignment and detailed musical characteristics for more accurate OMI and TA estimation. Furthermore, to overcome the limitations of conventional regression losses in measuring human preferred relative ranking of music segments, we design a novel ranking-aware composite loss function. Experiments conducted on the MusicEval benchmark demonstrate that our approach outperforms the existing SOTA methods in both mean squared error and rank correlation metrics by a large margin.
Source:https://arxiv.org/abs/2611.07562PreviousA Unified Multi-Encoder Framework for Music Lyric Intelligibility Assessment: Bridging Invasive and Non-Invasive ParadigmsNextAuditory and Visual Virtualization in 6DoF XR: Effects on Plausibility and Dynamic Auditory Distance Perception
Related posts
Where an e-commerce explainer video belongs, and what to show in it
0dev.to
How to explain a crypto product on video without sounding like a pitch
0dev.to
Why Europe's Freezer Hacks Spread So Far, So Fast
0Dark Reading

Protesters storm Microsoft AI data center construction site
0Tom's Hardware

The Download: a humanoid cleans house and Samsung builds for the AI boom
0MIT Technology Review

How Remedy designed Control Resonant’s wildest mission: ‘Anything kind of goes’
0The Verge