Backend
On the Optimality of Implicit Feedback in Markov Decision Processes over Imperfect Links
arXiv cs.SY0 views
arXiv:2611.06365v1 Announce Type: new Abstract: Goal-oriented communication protocols are explicitly designed to support the objectives of target applications, such as the monitoring and control of remote processes, while minimizing data traffic. This study considers a transmitter node that observes the environment state and sends it to a remote agent, which then performs control actions. This problem can be modeled as a remote Partially Observable Markov Decision Process (POMDP), a subclass of decentralized POMDPs. Existing studies limit transmissions through communication costs, but neglect the effect of channel impairments and non-ideal or delayed feedback. In this work, we study the optimal joint transmission and control policies over lossy forward channels without explicit acknowledgments. We introduce \emph{revealing} policies, under which the receiver's action, observed directly by the transmitter, discloses the outcome of the forward transmission, keeping the two agents synchronized without a feedback channel. We derive sufficient conditions for the optimality of time-invariant policies in this implicit feedback regime for general POMDPs, along with strictly weaker sufficient conditions in the special case of contextual bandits. Our simulations over a family of remote POMDPs show that these conditions are conservative, and revealing policies are optimal over a substantially wider parameter range than the conditions warrant.
Source:https://arxiv.org/abs/2611.06365PreviousTopology Identification of Directed Signed Networks via Repelling LaplaciansNextSpatial Differences Analysis of Nodal Frequency Considering Frequency-Voltage Coupling
Related posts
TAGR: Temporally Adaptive Generative Recommendation for Industrial Live-Streaming Advertising
0arXiv cs.IR
ArcRAG: Agentic Compositional Retrieval over Knowledge Graphs
0arXiv cs.IR
From Waste to Discovery: Recycling High-Scored Unexposed Candidates in Taobao Feed Recommendation
0arXiv cs.IR
Toward a Compositional Knowledge Architecture for a Shared Organizational Cortex
0arXiv cs.IR
Turbocharging PiPNN for Proximity and $k$-NN Graph Building
0arXiv cs.IR
Combining Relevant Expert Knowledge Using Mixtures of Feedback Fuzzy Cognitive Maps
0arXiv cs.IR