Jev vs Flash LLMs: 7x Cheaper on Workflows, Same Cost on One Label

Whether TypeSafe’s Jev is cheaper than a flash LLM depends on the shape of the job. Asking 12 questions about each support transcript, Jev cost 7 times less than GPT-5.6 Luna with thinking off and took 0.12 seconds after network time against 1.66. On a single 77-way label, it cost the same as Qwen3.8 Flash and GLM 5.3 Flash. Jev is a decision model: instead of writing text, it returns a choice, a score or a probability for each question you define. We ran it against five flash-tier chat models, at their cheapest and at their default thinking settings, on four tests built from public datasets. It was the fastest model on every test and the most accurate on none.

TL;DR

• With 12 questions per case, flash LLMs cost 4.8 to 37 times more than Jev and took 11 to 27 times longer, at similar accuracy.
• On one label per message, Jev cost the same as Qwen3.8 Flash and GLM 5.3 Flash with thinking off.
• At default thinking, one label cost 1.5 to 26 times more on flash models than on Jev.
• GPT-5.6 Luna was more accurate than Jev on every test except prompt injection.
• Jev’s answers at probability 0.99 or above were 98% correct.

What is Jev, and how is it different from an LLM?

Jev is TypeSafe’s “System One” model: it reads a piece of text or JSON (the state) and answers typed questions about it, with no generated text to parse. There are three question types:

A request asks its questions by name and gets one answer back per name. This one, in TypeSafe’s documented format (API reference), asks three yes/no questions about a short support chat:

curl https://api.typesafe.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "jev-latest",
    "state": {"transcript": "Customer: My card was declined twice today.\nAgent: One moment.\nCustomer: Also, how do I change my PIN?"},
    "questions": {
      "declined_card": {"type": "noul", "instructions": "In `transcript`, does the customer report a declined card payment?"},
      "change_pin":    {"type": "noul", "instructions": "In `transcript`, does the customer want to change a PIN?"},
      "lost_card":     {"type": "noul", "instructions": "In `transcript`, does the customer report a lost or stolen card?"}
    }
  }'

What came back when we ran it:

{
  "model": "jev-1.13.0",
  "answers": {
    "declined_card": {"type": "noul", "noul": 0.99},
    "change_pin":    {"type": "noul", "noul": 0.98},
    "lost_card":     {"type": "noul", "noul": 0.02}
  },
  "usage": {"input_tokens": 359, "output_tokens": 56}
}

Your code reads answers.<question>.noul and compares it with a threshold, the way it would compare any number. The model field names the version that answered; jev-latest moves when TypeSafe ships a new version, so pin jev-1.13.0 if you tune thresholds against it.

TypeSafe evaluates the questions in parallel. Jev 1.13 costs $0.042 per million input tokens and output is free (models), which is 3.6 times less per input token than Qwen3.8 Flash, 4.8 times less than GPT-5.6 Luna and 18 times less than Gemini 3.8 Flash. Input includes TypeSafe’s own template, about 300 tokens a call, plus every question and option you define: the call above billed 359 input tokens, $0.000015, and its 56 output tokens were free. TypeSafe lists what Jev is weak at in its jaggedness notes: counting, arithmetic, date comparison, multi-step questions and long states full of irrelevant text.

How did we test it?

Four tests, each run once on 2026-09-21 and 2026-09-22 through Jev and through five flash chat models: GPT-5.6 Luna, Qwen3.8 Flash, GLM 5.3 Flash, DeepSeek V4.1 Flash and Gemini 3.8 Flash. Each chat model ran at its cheapest working thinking setting (off, or low where off is not accepted) and, on the three classification tests, again at the vendor default.

• 12 questions per case: 150 customer support transcripts, each assembled from one to six messages of the Banking77 dataset (PolyAI, CC BY 4.0), so the right answers are known. Every model answers 12 yes/no questions per transcript (“does the customer report a lost or stolen card?”) with a probability each: Jev as 12 Noul questions in one call, the chat models as one JSON object. We reran 60 of them with a 3,000-token and a 12,000-token reference document (Wikipedia articles on banking) added in front of the transcript.
• One label: 308 Banking77 messages, 4 from each of 77 intents. Jev gets one Choice with 77 options; the chat models get the same list and reply with the intent.
• 28 labels: 200 GoEmotions comments (Google, Apache 2.0), each with 28 possible emotions.
• Prompt injection: the 116-row deepset prompt-injections test set (Apache 2.0), one yes/no question each.

Cost is each vendor’s list price times the tokens billed. Latency comes from a separate run of 40 calls per model on a kept-open connection, with the network round trip subtracted; the speed section explains how. We also ran GPT-5.6 Terra and Seed 2.0 Mini on the three classification tests; they are in the tables but not the chart.

When is Jev cheaper than a flash LLM?

When one text gets many questions, when the text is long, or when the chat model would otherwise think. On a single short label against a cheap model with thinking off, Jev costs the same.

Three things move the multiple. First, the number of questions: a chat model has to write an answer for every question, and output is its expensive side (GPT-5.6 Luna charges $1.20 per million output tokens against $0.20 for input), while Jev’s answers are free and each extra question adds about 17 input tokens. Second, thinking: at vendor defaults the chat models wrote a median of up to 161 output tokens per label where thinking off wrote 3 to 8, and the multiple against Jev rose with it. Third, input length: with a 12,000-token reference, the fixed template and the answers stop mattering, and the multiple settles near the ratio of input prices, 3.5 times for GLM 5.3 Flash up to 18.6 times for Gemini 3.8 Flash; DeepSeek V4.1 Flash sits at 7.1 times.

The one-label test is the case where Jev saves nothing against the cheapest models: 77 options add about 1,400 tokens to every call, so a one-line message bills about 1,690 input tokens on Jev against 440 on Qwen3.8 Flash.

How much faster is Jev?

About 7 to 28 times once the network is taken out: Jev’s median was 0.11 to 0.12 seconds and the chat models took 0.79 to 3.25 seconds.

We timed 40 calls per model and task, one at a time on a connection kept open, and subtracted the round trip measured on that same connection with a request that does no model work. What is left is the model’s own time plus the path from our proxy to its provider, the same route for every model.

The 95th percentile is where thinking hurts: at vendor defaults, one Qwen3.8 Flash call in twenty took over 21 seconds for a single label. Jev’s slowest calls stayed near 0.2 seconds.

Adding questions did not slow Jev down: going from 1 to 60 yes/no questions about one support ticket moved its median by less than 0.1 seconds, and the 60-question call billed 1,371 input tokens, $0.000058. Every answer on the 20-question calls was right in all five runs.

How do these results compare with TypeSafe’s own claims?

They agree on the direction and are smaller in size, because the comparisons differ. TypeSafe’s launch post quotes up to 444.6 times cheaper and 193.6 times faster, measured against frontier models with reasoning on multi-step workflows, and calls those numbers “the higher end of real world gains” (launch post).

Its published workflow evals include GPT-5.6 Luna: Jev at 67.8% accuracy, $0.0004 and 0.4 seconds per case against Luna at 66.8%, $0.0033 and 12.9 seconds, about 8 times cheaper and 32 times faster. Our 12-question test measured Luna with thinking off at 7 times Jev’s cost and 14 times its time after network, the same order of magnitude. Against the cheapest flash models on a single label, the cost gap closes to nothing.

Is Jev as accurate as a flash LLM?

Close on the workflow test, behind on the classification tests, and never first. On 12 questions per case, Jev’s micro F1 (every yes and no across all questions, where 1.0 is perfect) sat inside the range of the chat models, and its probabilities ranked right from wrong answers about as well as the best of them.

AUC measures how well the probabilities rank: 1.0 means every true “yes” got a higher probability than every “no”. With the 12,000-token reference in front, Jev held its accuracy (F1 0.914 against 0.922 on the same 60 transcripts without it).

On the classification tests the chat models pulled ahead, most of all at their default thinking settings:

GPT-5.6 Luna and Terra answered 115 of the 116 injection texts and 199 of the 200 comments; one request each returned an error, and accuracy is over the answers returned. A reply that was not a yes or no counts as wrong. On emotions Jev said yes too often: an emotion it gave 0.80 to 0.95 was in the human label only 15% of the time. The emotion labels are sparse (annotators picked one or two per comment), which lowers every model’s score, but every model got the same instruction. On prompt injection, Jev’s 0.5 default was the wrong cut-off, which the next section covers.

Can you trust Jev’s confidence?

Its probabilities separate right from wrong answers well, but they are not calibrated in the plain sense, so each task needs its own threshold. Calibrated means a 0.8 is right 80% of the time; on Banking77 Jev was right 98% of the time at 0.99 and above, and about half the time below 0.8.

Nearly half of the Banking77 messages (48%) got a top probability of 0.99 or more, so a pipeline that accepts those and sends the rest to a larger model keeps 98% accuracy on that share of its traffic. Prompt injection shows the other side. At the default 0.5 cut-off Jev flagged only half the attacks: the other half got probabilities between 0.03 and 0.5, low in absolute terms but mostly above the benign texts, whose median was 0.02. Ranked by probability, it separated attacks from benign text at an AUC of 0.984.

Those thresholds were chosen on the same 116 texts, so treat them as an upper bound. A chat model can give a probability too when its API returns token log-probabilities, the model’s own probability for each token it writes; among the chat models here, Qwen3.8 Flash and Seed 2.0 Mini returned them in our runs, and Qwen3.8 Flash’s probability of “yes” ranked the injection texts at an AUC of 0.977.

Should you use Jev or a flash LLM?

The pattern that got the most out of Jev in our runs: decompose the decision into many narrow questions about the same text, act on answers above a threshold tuned on your own labeled data, and send the rest to a chat model.

FAQ

What is TypeSafe Jev? Jev is a decision model from TypeSafe that answers typed questions (yes/no, pick one, rate on a scale) about a piece of text and returns probabilities instead of generated text. Jev 1.13 is the current version, served under the jev-latest alias, at $0.042 per million input tokens with free output.

Is Jev cheaper than an LLM? It depends on the job. Asking 12 questions about each support transcript, GPT-5.6 Luna with thinking off cost 7 times more than Jev and Gemini 3.8 Flash 37 times more; on a single 77-way label, Qwen3.8 Flash and GLM 5.3 Flash with thinking off cost the same as Jev.

Is Jev more accurate than GPT-5.6 Luna? Not in our tests: GPT-5.6 Luna with thinking off scored a micro F1 of 0.933 against Jev’s 0.909 on 12 questions per case, and 85.1% against 80.2% on Banking77. Jev caught more prompt injections than Luna with thinking off at its default 0.5 cut-off.

Why does TypeSafe say Jev is 444 times cheaper? That figure compares Jev with frontier models running reasoning on multi-step workflows, and TypeSafe calls it the high end. Against flash models we measured 4.8 to 37 times cheaper on a 12-question workflow and about the same cost on a single label.

Are Jev’s probabilities calibrated? Not in the plain sense. On Banking77 its answers were right 98% of the time at a probability of 0.99 or more but only about half the time below 0.8, and on prompt injections it gave half the attacks a probability under 0.5. Tune a threshold per task on your own labeled data.

Related: best flash LLM API in 2026, LLM structured outputs compared, LLM thinking controls, best LLM by use case.

← Back to blog