Backend
On the Optimality of Implicit Feedback in Markov Decision Processes over Imperfect Links
arXiv cs.SY0 views
arXiv:2611.06365v1 Announce Type: new Abstract: Goal-oriented communication protocols are explicitly designed to support the objectives of target applications, such as the monitoring and control of remote processes, while minimizing data traffic. This study considers a transmitter node that observes the environment state and sends it to a remote agent, which then performs control actions. This problem can be modeled as a remote Partially Observable Markov Decision Process (POMDP), a subclass of decentralized POMDPs. Existing studies limit transmissions through communication costs, but neglect the effect of channel impairments and non-ideal or delayed feedback. In this work, we study the optimal joint transmission and control policies over lossy forward channels without explicit acknowledgments. We introduce \emph{revealing} policies, under which the receiver's action, observed directly by the transmitter, discloses the outcome of the forward transmission, keeping the two agents synchronized without a feedback channel. We derive sufficient conditions for the optimality of time-invariant policies in this implicit feedback regime for general POMDPs, along with strictly weaker sufficient conditions in the special case of contextual bandits. Our simulations over a family of remote POMDPs show that these conditions are conservative, and revealing policies are optimal over a substantially wider parameter range than the conditions warrant.
Source:https://arxiv.org/abs/2611.06365PreviousTopology Identification of Directed Signed Networks via Repelling LaplaciansNextSpatial Differences Analysis of Nodal Frequency Considering Frequency-Voltage Coupling
Related posts
Run LLMs Locally in 2026: Ollama vs LM Studio vs llama.cpp, Compared
0dev.to AI
La Gran Migración
0dev.to AWS
Can We Still Trust What We See Online? The Growing Importance of Digital Content Provenance
0dev.to WebDev
A Developer’s Guide to Preserving UTM Parameters from Landing Page to CRM
0dev.to WebDev
How to Launch a Branded OTT Platform Without Building It From Scratch
0dev.to WebDev
Are We Becoming Developers Who Can't Actually Code Without AI?
0dev.to WebDev