In this episode of The New Stack podcast, recorded at AGNTCon + MCPCon in San Jose, Nick Lucchesi speaks with Mike Chambers, Senior AI Developer Experience Engineer at AWS, about how small decision models could make AI agents faster, more efficient, and cost-effective. Unlike large language models (LLMs) that generate responses through multiple processing steps, decider models quickly evaluate information and return probability-based decisions.

Say “good morning” to an AI agent, and it might search its long-term memory before saying hello back. Somewhere in that system, a service performs a semantic lookup to determine whether your greeting requires understanding your beverage preferences. Even if you take your coffee very seriously, this seems like a lot of infrastructure for two words.

Does this request need a more capable model than conventional options? Is this retrieved memory relevant? Should the workflow pause for a human?

This inefficiency and indecision is where Mike Chambers, a senior AI DevEx engineer at AWS, sees an opportunity. In a conversation last month at AGNTCon + MCPCon North America 2026 in San Jose, California, Chambers explained to The New Stack Editor-in-Chief Nick Lucchesi how developers can use decider models to handle the small judgments that add up inside an agent. 

Decider models (also called decision models or System One models) are specialized AI models that choose from a fixed set of options or rate inputs on a scale — returning calibrated probabilities without generating free-form text. Popularized by TypeSafe AI’s Jev, they give software and agentic workflows fast, low-latency judgments with explicit confidence scores. (AWS’s Strands Labs released its own open-source decision model, the 2-billion-parameter Strands Decider 2B, on October 1.)

An if statement with a feel for language

Chambers describes the approach as “a little bit like having an agentic-powered ‘if’ statement inside of your code.”

Developers have used classifiers for years. What makes this generation of decider models interesting, Chambers told The New Stack, is that general-purpose pretraining lets you try one on a new classification task without first training a model of your own. You supply context in natural language, ask a question about it, and get a numeric score your application can use to choose a branch.

In the models Chambers describes, that score comes back in a single forward pass. There’s no token sequence to generate. That can make a decider attractive for a choice that sits on the critical path of every request. A model router, for example, can use a score to send routine prompts to a smaller model and reserve a more capable one for harder jobs. The potential savings come from both the routing decision itself and the downstream work it saves.

Chambers tried the idea with the open-source Strands framework and Amazon Bedrock AgentCore Memory. He put Strands Decider in front of the memory lookup to judge whether an incoming message called for fetching anything from past conversations. A greeting could go straight through. A request that depends on something the user said earlier could still trigger a search.

He also tried using a decider after retrieval. A memory service can return the top matches to a query, but some may have little bearing on the task at hand. A decider can score each result for relevance before the agent includes it in the context it sends to the model. One decision can save a service call. The other can give the model less extraneous material to sort through.

A predictable output still needs testing

A number in a predictable JSON field makes a decider easy to wire into an application. Whether that number is reliable enough to act on takes testing. “Will the number be correct? That’s for your testing to be able to go and figure out whether that’s the case or not,” Chambers said.

Chambers calls his memory experiment experimental. The interview also gets into what happens when the stakes rise. A missed preference might mean a mediocre coffee recommendation. An agent about to spend money is a different matter, and Chambers argues for checks outside the agent’s loop and human approval for decisions that are hard to reverse.

Watch the full interview on The New Stack’s YouTube channel for Chambers’ walkthrough of the memory experiment and a look at where developers are taking open-source deciders next, from documents to robotics.

The post Does your AI agent need an LLM to decide what to do next? appeared first on The New Stack.