Direct Model Calls and Decisions
Call a model once through the Framework's provider edge, or ask Decisions for a number instead of prose, without building an agent.
Some work is one prompt and one answer: classify a string, draft a summary, extract a field. That needs the provider edge — drivers, endpoints, credentials, retries, error classification — but none of the agent loop around it.
Model::complete is the whole API for that case:
use everruns::{Model, OpenAI};
let model = Model::new("gpt-6.2-sol", OpenAI::from_env()?);let answer = model.complete("Name the three primary colors.").await?;println!("{answer}");The model is the same value an agent takes, reached
through the same Provider, which can also
be asked which models it offers. Nothing is
persisted: a direct completion owns no session, no history, and no workspace.
Reach for an agent as soon as the work needs tools, multiple turns, durability,
or events.
When the answer should be a number rather than prose, such as a probability, a choice among options, or a graded score, use Decisions instead.
Testing without a provider
Section titled “Testing without a provider”Model::simulated returns canned responses from an in-process simulator
(everruns-llmsim). It is a test double, not a local model: it runs no
inference and is not a way to use Everruns without a model provider. It exists
so tests and examples can assert on agent behavior without a network call or an
API key.
Real work always goes through a provider — see Supported providers.
use everruns::Model;
let answer = Model::simulated("4").complete("What is 2 + 2?").await?;assert_eq!(answer, "4");See Testing and simulation for scripted multi-response simulators.
System messages, context, and controls
Section titled “System messages, context, and controls”Model::completion describes the call before sending it. Messages append in
call order; each control maps to one provider request field and stays unset
unless assigned, so the provider keeps its own defaults.
use everruns::{Model, ReasoningEffort};
let response = model .completion() .system("Answer with a single word.") .user("What is the capital of France?") .max_tokens(16) .reasoning_effort(ReasoningEffort::Low) .send() .await?;
println!("{}", response.text);println!("{:?} tokens", response.metadata.total_tokens);send returns the full LlmResponse — text, reasoning artifacts, tool calls,
and call metadata. text() returns only the answer text. Replay prior turns
with .assistant(...): the completion carries no history of its own, so
context is whatever the call passes.
When the model is a bare provider-visible id, attach the provider on the completion instead of the model:
use everruns::{Model, OpenAI};
let answer = Model::from("gpt-6.2-sol") .completion() .provider(OpenAI::from_env()?) .user("Summarize this in one line: ...") .text() .await?;Streaming
Section titled “Streaming”stream() returns the provider’s events as they arrive, ending with a Done
event carrying the call’s metadata:
use everruns::{LlmStreamEvent, Model};use futures::StreamExt;
let mut stream = model.completion().user("Write a haiku.").stream().await?;while let Some(event) = stream.next().await { if let LlmStreamEvent::TextDelta(delta) = event? { print!("{delta}"); }}What the stream guarantees:
- Every
TextDeltacarries text. Assistant text is their concatenation. ReasoningDeltastreams reasoning live; theReasoningItemthat follows repeats the whole block, plus any replay state. Show the deltas and store the item, or use the item alone; adding both duplicates the reasoning.- The stream ends with exactly one
DoneorError.Errorkeeps the provider’s message, error code, and HTTP status, including errors a gateway reports inside a200stream. Donecarriesfinish_reasonas the provider sent it, orNonewhen it sent none. It is never filled in asstop, so a caller can treat a missing reason as a failure.- Drivers retry
429and transient5xxresponses before the first event: up to 2 retries and 30 seconds by default. That time counts against any timeout you put around the call. To own retries yourself, build the driver with.with_retry_config(everruns::llm::LlmRetryConfig::no_retry()).
The full contract is on LlmStreamEvent in the API reference.
Errors
Section titled “Errors”CompletionError separates configuration mistakes from provider failures:
MissingProvider— the model names an id but nothing says how to reach it.NoMessages— the completion was sent empty.Call(..)— the provider call failed, carrying theAgentLoopErrorand its fullLlmErrordecision.
The first two are caught before any request leaves the process.
Going lower
Section titled “Going lower”Completion is a thin value-first layer over Provider, which is public.
Applications that already hold a Provider — or implement their own
ChatDriver — can call it directly with
everruns::llm’s Message and MessageRole, plus the crate-root
LlmCallConfig and LlmResponse re-exports:
use everruns::llm::{Message, MessageRole};use everruns::{LlmCallConfig, Provider};
let response = provider .chat_completion( vec![Message::text(MessageRole::User, "What is 2 + 2?")], &LlmCallConfig::new("gpt-6.2-sol"), ) .await?;That surface is the driver boundary itself: every field of LlmCallConfig,
including tool definitions, is available, and nothing is defaulted for you.
When all you have is a base URL, DriverId::for_base_url names the driver
whose vendor serves it (OpenAI, Azure OpenAI, OpenRouter, Anthropic, Gemini,
Fireworks). It returns None for a gateway or self-hosted server; those speak
the OpenAI-compatible wire, so use DriverId::OpenAICompletions.
The runnable version of this section is
direct_llm.rs,
which runs offline without an API key.
Decisions
Section titled “Decisions”Some questions have typed answers. Is this claim supported by the source? How severe is this complaint? Which queue does this ticket belong in? A chat model answers those in prose, so the call site ends up with a prompt asking for JSON, a parser, and a fallback for when the parse fails.
A decision service answers them as numbers instead, and the decision stays in your code.
This is the counterpart to a direct model call: the same shape, a different contract.
Model |
Decisions |
|
|---|---|---|
| you send | messages | state plus typed questions |
| you get back | text | calibrated numbers |
| decides the outcome | the model’s words | your threshold, in your code |
| streams | yes | no — one round trip |
Quick start
Section titled “Quick start”cargo add everruns --features typesafecargo add tokio --features macros,rt-multi-threadexport TYPESAFE_API_KEY=... # a key from typesafe.aiuse everruns::{Decisions, TypeSafeAI};
#[tokio::main]async fn main() -> Result<(), Box<dyn std::error::Error>> { let decisions = Decisions::new("jev-latest", TypeSafeAI::from_env()?);
let text = "CONGRATULATIONS! You've WON $1,000,000. Click here to claim your prize now!"; let spam = decisions.probability("Is this message spam?", text).await?;
println!("spam: {spam:.2}"); if spam > 0.9 { println!("quarantined"); } Ok(())}spam: 0.98quarantinedOne number, and your own > 0.9 decides — the model reports how likely, not what
to do. The same call answers 0.03 for “Standup moved to 10am.” and 0.74 for a
bare “Claim your prize now!”; the middling ones are what a threshold is for.
--features typesafe adds TypeSafeAI and the Jev capability. Without it
everruns names no vendor: Decisions::new takes any DecisionsService.
OpenAI’s Decisions API
Section titled “OpenAI’s Decisions API”OpenAI’s Decisions API
(public beta, model gpt-6-luna) answers the same questions. OpenAIDecisions
in everruns-integrations is a DecisionsService, so it drops into the same
Decisions:
cargo add everrunscargo add everruns-integrations --features openai-decisionsexport OPENAI_API_KEY=...use everruns::Decisions;use everruns_integrations::openai_decisions::OpenAIDecisions;
let service = OpenAIDecisions::new(std::env::var("OPENAI_API_KEY")?);let decisions = Decisions::new("gpt-6-luna", service);
let urgent = decisions .probability("Does this convey urgency?", "My card was charged twice, fix it today.") .await?;println!("urgent: {urgent:.2}");A runnable version that routes a support ticket with all three primitives is
openai_decisions_triage:
OPENAI_API_KEY=... cargo run -p everruns-integrations --features openai-decisions --example openai_decisions_triageAll three primitives are native and calibrated, and one request sends every
question in one call. Score questions take at most ten levels. On Everruns
Platform, set UTILITY_DECISION_DRIVER=openai with UTILITY_OPENAI_API_KEY to answer
guardrail jev checks with it (see
environment variables).
Three primitives
Section titled “Three primitives”A decision asks one or more questions about the same state. Each is one of three shapes:
use everruns::Decisions;
let answers = decisions .about("I've been on hold for two hours and my card was charged twice.") .noul("urgent", "Does this convey urgency?") .score( "severity", "How severe is the problem the writer describes?", [ "A minor annoyance", "A real problem with their account", "Serious harm requiring immediate action", ], ) .choice( "queue", "Which team should handle this message?", ["billing", "technical", "sales"], ) .send() .await?;
let urgent: f64 = answers.probability("urgent")?;let queue: &str = answers.selected("queue")?;noul— whether something holds, as the probability of yes. A value near 0.5 means yes and no are near-equally likely, not “medium”. (Noul)choice— exactly one option from your set, with the distribution behind it. Needs at least two options. (Choice)score— a position along levels you define, lowest first. Needs at least two levels. (Score)
The three are System One’s own, so TypeSafe’s
Primitives documents what each answer
means and how to choose between them, and
State covers what to put in the
about(...) value. Everruns names them the same way rather than inventing
synonyms.
Questions in one call are answered in parallel inside a single request, so asking five costs one round trip, not five. TypeSafe calls leaning on that speculative fan-out: ask the questions you might need, and let your code decide which ones mattered.
Ids are yours; instructions are the model’s
Section titled “Ids are yours; instructions are the model’s”The id labels the answer for your code and is never sent to the model. A question whose meaning lives in its id asks nothing:
// Wrong: the model never sees "is_the_joke_funny"..noul("is_the_joke_funny", "?")
// Right: the instructions carry the question..noul("funny", "Would a general audience laugh at this joke?")The same applies to score levels. Describe concrete situations — “A minor annoyance” reads on its own where “2 out of 5” does not.
Read the tail, not the average
Section titled “Read the tail, not the average”For “is there any serious hit here” rules, read the probability mass at or above a level rather than the weighted score. Something probably fine but possibly awful must not average into fine:
// Not: answers.score("severity")? > 1.5let serious = answers.tail("severity", 2)?;if serious > 0.3 { println!("escalated");}Errors
Section titled “Errors”DecisionsError separates configuration mistakes from service failures, the
same split CompletionError makes:
MissingService— the decisions was built without a service to reach.NoQuestions— the decision was sent with nothing to ask.Unconfigured— the service exists but the deployment never configured its credential, so it would answer nothing.NoSuchAnswer(id)— you read an id that was not asked, or read an answer as the wrong shape (ascoreas a probability).Call(..)— the service call failed, carrying theAgentLoopError.
The first three are caught before any request leaves the process.
Unconfigured is worth handling separately: a guardrail treats it as fail-open,
but a direct caller usually wants to know the number never arrived rather than
read a confident-looking default.
Going lower
Section titled “Going lower”Decision is a thin value-first layer over DecisionsService, which is
public. Applications that already hold a service — or implement their own, over
a different vendor or a local model — can call it directly with everruns’s
DecisionRequest, DecisionQuestion, and DecisionAnswer
re-exports:
use everruns::{DecisionQuestion, DecisionRequest, DecisionsService};
let outcome = service .evaluate( DecisionRequest::new("Claim your prize now!") .ask("spam", DecisionQuestion::noul("Is this message spam?")), ) .await?;That surface is the contract itself: every question type and the full
DecisionOutcome, including usage, with nothing defaulted for you.
Implementing DecisionsService is also how a different decision service (another
vendor, or a fine-tuned local model) plugs into the same Decisions,
guardrails included.
Choosing a model
Section titled “Choosing a model”The model is named up front, the way Model::new
names one: the service is transport, and the model is the thing that answers.
There is no default to inherit without noticing, because a threshold calibrated
against one version is not evidence about the next.
Ids are the provider’s own, so they are spelled the way the vendor spells them.
jev-latest is TypeSafe’s alias for the current Jev, so it tracks whatever the
current version is; an exact id like jev-1.14.0 pins one, so a vendor update
cannot move your thresholds under you. Bare jev is not an id the API knows —
nothing here rewrites what you pass.
Ask for the alias and read back what answered, which is the id to pin once a threshold is calibrated:
let version = answers.model(); // "jev-1.14.0" for a "jev-latest" requestA single call can name a different model with the same method on the request builder, and it wins for that call:
let answers = decisions .about("...") .noul("urgent", "Does this convey urgency?") .model("jev-1.14.0") .send() .await?;A deployment that must pin a model does so by never exposing the knob in the config an agent writes — not by the type being unable to carry one, because there will be other decision services and other models.
A deployment running the Everruns platform configures a separate
UTILITY_TYPESAFE_API_KEY for its guardrails — a
different account from the one an embedding application holds.
Giving an agent the decisions
Section titled “Giving an agent the decisions”Everything above is agentless: your code asks, your code decides. The other half is letting an agent classify as part of its own work — checking a claim against a source before citing it, rating a draft before sending it.
Jev is the same decisions as a capability, so an agent gets it as a tool:
use everruns::{Agent, Engine, Jev, Model, OpenAI};
let agent = Agent::builder() .name("reviewer") .instructions( "You review copy. When asked how something reads, measure it with \ jev_decision and report the numbers rather than judging by eye.", ) .model(Model::new("gpt-6.2-sol", OpenAI::from_env()?)) .capability(Jev::from_env()?) .build()?;
let session = Engine::new().create(agent);let turn = session .run("Rate this subject line for pushiness: 'Act now before it is too late'") .await?;The agent calls jev_decision, writing its own questions about whatever it is
looking at, and gets the same calibrated numbers back. It is the identical tool
the hosted TypeSafe integration gives platform
agents — same name, same schema — so behavior matches whether you embed the
Framework or run on Everruns.
Which one to reach for:
| you decide | the agent decides | |
|---|---|---|
| who asks | your code writes the questions | the model writes the questions |
| use | Decisions |
the Jev capability |
| good for | a policy check, a routing rule, a gate | verification inside a longer task |
What stays with an agent
Section titled “What stays with an agent”A decision owns no session, no history, and no workspace, and runs no tools. Reach for an agent as soon as the work needs any of those. Typed output guarantees the interface, not the truth: validate thresholds against your own data and consequences.
Testing without a credential
Section titled “Testing without a credential”Decisions::simulated returns a fixed number from an in-process stub. It is a
test double, not a local decisions: it runs no inference, reads nothing
from the state you pass it, and is not a way to classify without a provider. It
exists so tests and examples can assert on the code around a decision
without a network call or an API key.
Real work always goes through a decision service: TypeSafeAI above, or
your own DecisionsService.
use everruns::Decisions;
let p = Decisions::simulated(0.93) .probability("Does this convey urgency?", "Two hours on hold.") .await?;assert!(p > 0.9);Because the answer is fixed, a simulated decision proves your threshold logic runs — never that a real decision service would return that number.
The runnable version of this section is
direct_decisions.rs.
It uses the stub by default so it runs with no key; pass --live (with
--features typesafe and TYPESAFE_API_KEY set) to send the same questions to a
real decision service. agent_decisions.rs
does the same for the agent path.
Shared provider accounts for decisions
Section titled “Shared provider accounts for decisions”Decisions::from_registry accepts a ProviderRegistry and an explicit ModelSpec.
Select ModelSpec::on("typesafe", "jev-1.14.0") for direct TypeSafe, or
ModelSpec::on("openrouter", "jev-1.14.0") for OpenRouter. The OpenRouter driver maps that
explicit alias to typesafe/jev-1.14; other IDs remain opaque. Providers hold authentication
once, so their chat and decision drivers use the same account. Returned model IDs report
the snapshot that actually answered.
Jev::with_provider(provider, model_spec) supplies that same account to a Framework capability.
Use its exact provider key in the model spec. Existing Jev::new and Jev::with_client conveniences
remain available for applications holding a direct TypeSafe key.
Offline batches
Section titled “Offline batches”For independent requests that can wait hours, use Batch Model Requests to submit through Anthropic Message Batches and persist a ticket for later collection.
Model refusals
Section titled “Model refusals”The Responses driver reports explicit model refusals as LlmErrorKind::Refusal
for streamed calls, native JSON completions, and batch results. These are terminal
decisions and are not automatically retried. The diagnostic retains the provider’s
reason, bounded to 16 KiB; treat it as provider content when displaying or logging
it. Engine sessions apply their configured error disclosure policy and expose the
stable model_refusal category instead of treating refusal text as a successful
answer.