AI & ML
Relative Entropy under Ordered Chi-Squared Constraints: Exact Bounds and Extremizers
arXiv math.IT0 views
arXiv:2611.24607v1 Announce Type: new Abstract: We derive a sharp upper bound on the relative entropy $D(P\|Q)$ in terms of the two ordered chi-squared divergences $\chi^2(P\|Q)$ and $\chi^2(Q\|P)$. For any $A,B>0$, the constraints $\chi^2(P\|Q)\leq A$ and $\chi^2(Q\|P)\leq B$ yield upper bounds on $D(P\|Q)$ and $D(Q\|P)$. Two pairs of Bernoulli distributions each attain both bounds simultaneously. As a consequence, for sequences of probability mass functions $\{P_n\}$ and $\{Q_n\}$ on a common finite or countably infinite alphabet, both $D(P_n\|Q_n)$ and $D(Q_n\|P_n)$ tend to zero whenever one of the two ordered chi-squared divergences tends to zero and the other remains uniformly bounded. Without this boundedness assumption, $D(P_n\|Q_n)$ can fail to converge to zero even when $\chi^2(Q_n\|P_n)\to 0$. Finally, in contrast to the attainable upper bounds, on any finite or countably infinite alphabet containing at least three elements, the infimum of $D(P\|Q)$ under the exact constraints $\chi^2(P\|Q)=A$ and $\chi^2(Q\|P)=B$ is zero and is not attained.
Source:https://arxiv.org/abs/2611.24607PreviousHull variation of algebraic geometry codes under divisor twistsNextThe Asymptotic Capacity of the Grabbag Channel With and Without Feedback
Related posts
![Physical AI Foundations and Definitions — Ch.00: Introduction [KR]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzz2y7pn0xi5qj511e402.png)
Physical AI Foundations and Definitions — Ch.00: Introduction [KR]
0dev.to
![Physical AI Foundations and Definitions — Ch.09: Standards and Definitions [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fw4mzb4ntt5lhj7vgf333.png)
Physical AI Foundations and Definitions — Ch.09: Standards and Definitions [EN]
0dev.to
![Physical AI Foundations and Definitions — Ch.06: Relation to Embodied AI and Robotics [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fshqcmgwjgqqejta2saya.png)
Physical AI Foundations and Definitions — Ch.06: Relation to Embodied AI and Robotics [EN]
0dev.to
![Physical AI Foundations and Definitions — Ch.03: History and Origins [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Frrm68fq83d8whnovwor0.png)
Physical AI Foundations and Definitions — Ch.03: History and Origins [EN]
0dev.to
![Physical AI Foundations and Definitions — Ch.02: Physical AI vs Digital AI [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fsgutmwrbqzyn5a22lilm.png)
Physical AI Foundations and Definitions — Ch.02: Physical AI vs Digital AI [EN]
0dev.to
![Physical AI Foundations and Definitions — Ch.00: Introduction [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzz2y7pn0xi5qj511e402.png)
Physical AI Foundations and Definitions — Ch.00: Introduction [EN]
0dev.to