Beyond LLMs
Decision models for smarter AI agents
Bharadwaj Pendyala
Frisco, TX · Nov 19, 2026
Who has an agent that calls an LLM just to pick from a list?
Which team? Is it urgent? Yes or no? And it waits seconds for every answer.
Color every step: decide or generate
The rule: if you could write down every possible answer today, it’s a decision. Seven steps here, and only one needs prose.
Bharadwaj
Pendyala
Tech Lead at Salesforce. I work on enterprise workflow systems and the tooling around agents.
Slides and demo code are linked at the end.
- Slidesbharad.dev/talks
- LinkedInin/bharadwajpendyala
- GitHubbharadwaj-pendyala
- X@bharadwaj_py
What a decision costs on an LLM
claude -p on my laptop, CLI startup included.reply.match(/\{[\s\S]*\}/)Find the JSON in the prose. Hope the fields are right.
“97% sure.” The model writes that number as text. Nothing measures it.
If the answer comes from a short list, don’t pay for prose.
System 1 catches the ball. System 2 does your taxes.
| System 2: LLM | System 1: decision model | |
|---|---|---|
| Good at | Planning, writing, new problems | Picking from a short list |
| Output | Text, one token at a time | Answers from your list, all at once |
| Confidence | A number it writes | Probabilities it’s trained to get right |
| Your code | Parses | Branches |
From Daniel Kahneman, Thinking, Fast and Slow (2011). An analogy, not a spec.
A decision model is a function call
Questions run in parallel. Answers always come from your list. That guarantees the shape, not the truth.
Four question types
One of a list
Which team? What does the lead want?
Where on a scale
How urgent, from 0 to 2? Returns 1.7, not just “high”.
How likely true
Does the draft promise a discount? Jev: noul. OpenAI: predicate.
Where in an image
Jev only, experimental.
Jev, from TypeSafe
- Trained for calibrated probabilities (they call it RLCD).
- Choice, score, noul, box.
- Vendor claim: 70 to 500 ms end to end.
We’ll measure it live instead of trusting the slide.
{
model: 'jev-latest',
state: { ticket },
questions: {
department: {
type: 'choice',
instructions: 'Which team should handle this?',
criteria: DEPARTMENTS,
},
urgency: { type: 'score', ... },
wantsRefund: { type: 'noul', ... },
},
}RLCD: reinforcement learning for calibrated decisions
OpenAI Decisions API
- Generally available, one model:
gpt-6-luna. - Choice, score, predicate.
- OpenAI: about 10x faster than the Responses API for these tasks.
Two vendors shipped nearly the same shape.
{
model: 'gpt-6-luna',
input: ticket,
questions: [
{
type: 'choice',
name: 'department',
instructions: 'Which team should handle this?',
choices: [...],
},
{ type: 'score', name: 'urgency', ... },
{ type: 'predicate', name: 'wantsRefund', ... },
],
}LLM plans. Decision model branches. Code sets the line.
Act on it.
Ask the LLM to think it through.
A person decides.
The agent writes the email. Jev decides if it should.
A booth follow-up agent for an account executive, built on AWS.
One state machine. Two questions to Jev.
SES sends it.
Step Functions waits. The AE approves from email.
No email.
How a booth lead flows through AWS
Each AWS piece has one job
Runs the steps in order and waits for the AE. The console shows each lead’s path.
One function runs every step. The thresholds live here, in code.
The agent loop and two tools. Bedrock or the Anthropic API, picked by one setting.
A fake CRM, and a receipt for every run.
Sends the email. In sandbox mode, only to your verified inbox.
Holds the Jev and Anthropic API keys as SecureStrings.
One SAM template deploys all of it.
Decide before you generate. Check after.
What do they want?
Meeting, pricing, info, or unclear. One cheap call picks the path before the agent runs.
Does the draft promise anything?
A discount, a date, a feature. The fast model checks the slow model’s work.
The do-not-contact check never reaches Jev. It’s a rule, so it’s an if.
The whole policy is one function
Step Functions branches on the label. No numbers live in the state machine.
SURE_ENOUGH = 0.8 LIKELY_NOTHING = 0.3 LIKELY_PROMISE = 0.5 def route(receipt, sure_enough=SURE_ENOUGH): intent = receipt['intent'] unclear = intent['probabilities']['unclear'] if intent['choice'] == 'unclear' \ or unclear >= LIKELY_NOTHING: return 'skip' if 'promises' not in receipt: return 'draft' if intent['confidence'] < sure_enough \ or receipt['promises'] >= LIKELY_PROMISE: return 'approve' return 'send'
Demo: five booth leads
One state machine. Two Jev calls. One AE on a phone.
What you just saw
| Lead | The note | What decided it | Path |
|---|---|---|---|
| L-1 | Pricing before a Q1 budget review | Jev: clearly pricing, clean email | Sent |
| L-2 | Beat a vendor’s price, SSO date, maybe a call | Jev: pricing or a call? | AE approved |
| L-3 | Grabbed a sticker and a t-shirt | Jev: maybe nothing to answer | No email |
| L-4 | Wants a call, opted out in the CRM | Do-not-contact rule, no Jev | Blocked |
| L-5 | Call with an SE about Step Functions | Jev: clearly a meeting, clean email | Sent |
The agent ran three times. Jev ran seven times. A person looked once.
Pick a new threshold. Nothing calls a model.
Every receipt is in DynamoDB. Rerouting reruns the policy function and nothing else.
$ make reroute T=0.45 Same Jev answers, no new model calls. Threshold 0.8 → 0.45 L-1 pricing 100% promises 6% send → send L-2 pricing 49% promises 15% approve → send ← changed L-3 info 48% skip → skip L-4 - blocked L-5 meeting 100% promises 13% send → send
When not to use a decision model
Replies, code, summaries. Use the LLM.
Look up, think, look up again. That’s System 2.
“Don’t email people who opted out” is an if. Write it.
A fixed list guarantees the shape, not the truth.
Check the calibration on your own data
| Model said | Answers in bucket | Actually right | Verdict |
|---|---|---|---|
| 90 to 100% | 50 | 46 (92%) | Calibrated |
| 70 to 90% | 30 | 24 (80%) | Calibrated |
| Below 70% | 20 | 9 (45%) | Send to a person |
Example numbers. A hundred labeled examples and an afternoon gets you yours.
Three things
- Color your agent’s steps with
make color. Count the decisions. - Move them to a decision model. Let code own the threshold.
- Measure calibration on your data before you trust anyone’s.
My bet: in a year, agent harnesses ship a decision model next to the LLM by default.