AI & ML
The Limit Is in the Input, Not the Model: State-Dependent Command Safety in Natural-Language Interfaces to Industrial Controllers
arXiv cs.CR0 views
arXiv:2611.24682v1 Announce Type: new Abstract: Language models judging commands to industrial controllers may not receive machine state, the equipment's current condition. The same command can require opposite safety decisions in different states, so an incorrect answer may reflect missing information rather than poor reasoning. We propose an answerability audit: does the model receive the information needed to decide? The audit records the model's inputs and pairs the same command with two states requiring opposite answers. The paired score counts a pair only when both answers are correct. We add rules, then state values, to the model's input. In an author-built test of 15 pairs from three systems with four language models, none reliably distinguished the paired cases without state, even with rules. With state, scores ranged from 0 to 13 correct pairs out of 15; three models improved. With text alone, one model always allowed commands and another always denied them. Their overall (raw) accuracies differed by 32 percentage points; both scored 50% balanced accuracy, which weights safe and unsafe cases equally, and no correct pairs. A rule evaluator matched all 15 pairs by applying the same rules that supplied the expected answers; this is not an independent safety test. The audit complements systems that check actions against safety rules: it checks whether the model receives the information needed for those decisions. This helps evaluators separate missing-input problems from model errors. These small-sample scores do not establish general model reliability.
Source:https://arxiv.org/abs/2611.24682PreviousChainIT Transaction Truth: Stablecoins, Web3 Wallets, and Multi-Rail Commerce. A Zero-Trust Control Plane for Verified Identity, Organizational Authority, Runtime Compliance, Secure Wallet Execution, NextLLM-Guided Cyber-Physical Attack Path Identification in Smart Grids
Related posts
Razorpay webhook not verified: anyone can fake a payment
0dev.to
From FAQPage Schema to Answerable AI Documentation
0dev.to
Wiring Android's MediaPipe LLM Inference API to a Multi-Turn Conversation Engine
0dev.to
What Is LLM Throughput? Tokens per Second, Concurrency, and Batching
0dev.to
Diese Woche haben mir zwei Tools etwas vorgeschlagen und es direkt abgeschickt.
0dev.to
AI Agent Observability: What to Log Before Production
0dev.to