Open Source
Signature-Kernel Based Evaluation Metrics for Robust Probabilistic and Tail-Event Forecasting
arXiv stat.ML0 views
arXiv:2602.10182v2 Announce Type: replace-cross Abstract: Standard sample-based metrics like Continuous Ranked Probability Score (CRPS) and Quantile Loss (QL) are frequently used in the evaluation of probabilistic time-series forecasting. However, these metrics fail to capture multivariate correlations or assess the ability to capture distribution tails. These global metrics are insensitive to errors on high-utility events in distributional tails due to over-representation of the distribution's body. To accurately measure distributional fidelity on tail-regions and regions of high utility while still having a proper scoring rule, we introduce a family of censored signature kernel metrics. Our proposed metrics concentrate evaluation of forecasts to a focus region, representing high-utility or distributional tails, by collapsing the body of the distribution to a single pivot. Our benchmarks on time-series foundation models (TSFMs) reveal that while a clear ranking can be formed for distribution capture, there is no clear winner for tasks like systemic load prediction. This indicates that models with a strong ability to capture an overall distribution do not produce forecasts with high downstream utility. To encourage the use of the proposed metrics we open-source the efficient signature kernel (ESK) library. This library facilitates batch computation, with custom triton kernels, achieving a speed-up of up to 3.58$\times$ compared to implementations via the popular SigKernel library. Code for our metrics are available on GitHub.
Source:https://arxiv.org/abs/2602.10182PreviousPragmatic Curiosity: A Unified Framework for Hybrid Learning and Optimization via Active InferenceNextKinetic Langevin Splitting Schemes for Constrained Sampling
Related posts
ASUS Expands TUF Gaming Series in India With New F16 and A15 Laptops, T700 Mini Desktop
0FOSS Bytes

How to Set Up a Self-Service Support Portal for Your Community with SeaTicket
0dev.to Tutorial

Tantou (担当) v1.0.0 — Open-source (AGPL-3.0) Human Capacity OS & real-time request dispatch
0dev.to PHP
YUL: Bringing Software Supply Chain Security to Coding Agents
0arXiv cs.SE
Towards Pose-Robust Few-Shot Vision-Language-Action Models
0arXiv cs.RO
High-Precision Docking of Differential-Drive Mobile Robots Through LiDAR and Vision Collaboration
0arXiv cs.RO