Jev workflow patterns from working reports

Start with the question the application needs answered

TypeSafe’s API uses state and typed questions. Choice selects among supplied options, Score evaluates a rubric, and Noul gives a probability for a statement. Choice and Score have a confidence field. Noul has no separate confidence field. A probability about the answer and a confidence value on a distribution are different fields.

The documentation recommends selecting thresholds according to the domain and the consequence of an action. A result that fits the output type can still be the wrong decision.

Sources:[1] TypeSafe AI[2] TypeSafe AI

A confidence-based handoff to Kimi K3

In Hassan’s email demo, Jev makes a first pass over 100 examples. The application sends 31 cases below a 95% confidence threshold to Kimi K3. The reported combined result is 96 correct decisions in 16 seconds, compared with 1.42 seconds for Jev’s stage alone.

The demo gives a handoff rule and a full-pipeline time. The 95% cutoff belongs to this demo. TypeSafe’s threshold guidance depends on the domain, so copying that boundary into another setting would go past the post.

Sources:[19] Hassan[2] TypeSafe AI

Pruning Claude’s tool history

fast-jev-compaction asks whether old tool calls and their results still matter. Its output keeps user and assistant text verbatim while pruning or shortening tool material. Calls remain paired with results, and recent messages are protected.

This use of Jev changes the context supplied to Claude by pruning tool history. The user question stays with Claude. The plugin falls back to Claude Code’s built-in compaction on errors or insufficient reduction. The authors caution that a probability can be high on a result that is unsafe to remove.

Sources:[8] tamaratran

Selecting browser actions before generating text

Jev Ultrafast builds an indexed table of observed browser elements. Jev chooses an operation and a compatible target; the agent asks a separate language model for text only when it needs to type. The project documents configurable text helpers, including DeepSeek and Gemini.

The executor checks page freshness and whether the selected target is still usable. The project requires independent outcome verification even when Jev selects DONE. Its README also lists unsupported features, including frames, uploads and pop-up tabs.

Sources:[24] Browser Use

Picking the next Codex model

jev-codex-router asks Jev four questions on each Codex call: whether Astra is mandatory, which of Luna, Terra, Sol or Astra to use, how much thinking to allow, and how long to keep the route. The executing model still gets the full request. An error keeps the turn on Astra at medium effort, and a shadow file can log the choice while Astra still answers.

The README’s figure of about 60% less than full Astra covers 237 turns under an older policy. The file says the figure comes from that simulation, and that the current policy has no matching figure.

Sources:[40] 0xNatoshi

Jev Router against GPT-6 Astra on DeepSWE

OpenRouter’s typesafe/jev-router asks Jev to pick a model and a reasoning effort for each request. Theo’s objection is that the prompt does not carry the codebase, the tools, or the size of the change. Jev can categorize the sentence. It cannot tell a 100-line port from a million-line one.

On DeepSWE he reports roughly the same performance as GPT-6 Astra on low, a slightly higher cost, and almost 5 times the runtime, after spending $1,000 on the bench. The posts give no score and no task count. The follow-up says to use Astra on low. This is a different arrangement from jev-codex-router, which asks four questions about a Codex call and can force Astra, and from the Pi extension that only changes a DeepSeek thinking level and falls back to high.

Sources:[79] OpenRouter[80] Theo[82] Theo[83] Theo[40] 0xNatoshi

Agent Arena’s measurement of a Jev Router

Arena’s 7 October 2026 thread evaluates a product it calls Jev Router by @typesafeai on more than 4,700 sessions. DeepSeek V4.1 Flash received 28.1% of model calls. GPT-6 Astra received 23.5%, Sol 22.8%, and Luna 11.1%. The opening post names GPT-6.1 Sol rather than Astra. End-to-end medians are 6.18 seconds for the router and 3.64 seconds for DeepSeek V4.1 Flash (Max). Arena says calling that DeepSeek model directly gets similar performance at lower cost and latency, and that the router costs 38% more. Steerability is +10.0%, against +10.48 for Claude Opus 5.5 (High).

The thread prints no API model id. It does not say the calls used OpenRouter’s typesafe/jev-router, so this measurement stays separate from Theo’s DeepSWE note, which publishes no score, and from Görn’s 16-prompt hook log.

Sources:[127] Arena.ai[79] OpenRouter[82] Theo

A confidence bar in front of Claude Sonnet 4.6

Grounded AI’s CascadeEvaluator asks jev/jev-latest whether the next turn should call a tool or reply. At confidence 0.7, 73 of 200 items go to bedrock/us.anthropic.claude-sonnet-4-6 and the other 127 stay with Jev. Mean time is 1.28 seconds against 2.59 for Sonnet on every item. Agreement with Kimi K2-thinking’s next action is 0.695 against 0.675. At the write-up’s default of 0.9, 112 of 200 items go to Sonnet and agreement is 0.685.

The X post says the accuracy was the same. The table is agreement with K2-thinking, and the write-up says some disagreements with that label are ones it accepts. bench.json at commit 523194a stores the unrounded fields. The file does not record a resolved Jev version. maslul’s 0.7 floor is a different hook: it withholds a tier, and the example then calls Haiku 4.5 or Sonnet 4.6 for the whole turn.

Sources:[129] Josh Longenecker[130] Josh Longenecker[128] Josh Longenecker[121] Ilia Tankelevich

Freeze a 1% error budget before measuring coverage

Vals fits a confidence threshold on the first 30% of 400 claim-verification items so that automated error stays at or under 1%, then applies that threshold to the other 70%. Jev 1.13.0 automates 95% of the held-out items at 1.6% error. Claude Sonnet 5.5 automates 100% at 0.4%. Gemini 4 Argon automates 100% at 0.8%. DeepSeek V4.1 Flash automates 96% at 0.8%. Six of twelve systems exceed the budget they were tuned for, including Jev, GPT-6 Astra, GPT-6.1 Sol, and GPT-6 Luna.

Sweeping the threshold over all 400 items would have let Jev automate 91% within the budget. That figure uses the labels it is scored on. On the same roster Jev ties Astra, Sol, and Luna at 390 of 400, then ranks last on a 396-question LegalBench slice. The 0.9 cascade in the Carnegie Mellon paper is a different freeze, on preference pairs.

Sources:[133] Connor Frank[132] Vals AI[102] Yubo Li, Yidi Miao, Ramayya Krishnan and Rema Padman

Triage stalled sales threads, then review the unsure 7%

Salesgear sends each of 203 hand-written threads to Jev as one request with six typed questions. Code computes warmth and whether a come-back window has passed. Gemini 3.8 Flash answers the same questions in JSON mode. Jev scores 193 of 203, Gemini 199 of 203. Both find 60 of 61 real deferrals. Answers under 75% confidence, 15 threads, go to a person and hold 6 of Jev’s 10 mistakes.

The threads are synthetic. Claude helped draft them, so Gemini is the baseline. results.json at commit 926e72e stores every answer. Jev ran with 8 workers, Gemini with 2. Machine mail still produced the only wrong pick in both top tens, a travel out-of-office reply.

Sources:[135] Premsanth Rajamani[136] Salesgear

Replace a Gemini screen on a live candidate pool

TypeSafe’s Jack & Jill case study says the initial scoring stage, 6,000 retrieved candidates down to a 2,000-person shortlist, moved from Gemini 3.1 Flash Lite to Jev. An offline test on 150 live roles kept 94.6% of hiring-manager winners on Jev’s shortlist against 93.9% for Gemini. The 0.7 point gap has a 90% interval of −0.4 to +1.8. Median time to screen a role is 10.3 seconds against 20.3. Cost per 1,000 candidates is $0.092 against $0.755.

$265,000 is current-volume savings. $500,000 is a projection that includes expected growth. Extra workflows sit outside those figures. The page names no Jev version and links no public log. This is TypeSafe’s write-up of a customer test.

Sources:[137] TypeSafe AI

Rank the passages, then let the language model read two

David Arias’s 10-K pipeline keeps the filing lookup in code. Keyword and vector search produce a shortlist. One Jev request picks the best passage and says whether the answer is among the 30. If that probability is under 0.5, the code asks again with the next 30. The language model then reads two passages and writes the answer. On the DeepSeek V4 Flash pair, that is one model call and 840 tokens, against 4.7 calls and 82,908 tokens for an agent that opens whole filings.

The agent scored 46 of 50 and Jev RAG scored 50 of 50. Code had already chosen the latest filing, and the four agent misses opened the previous year. With GPT-5 nano the same split is 48 of 50 against 34 of 50. The article treats 50 questions on four large-cap 10-Ks as one pattern, and says the set is too small to settle the comparison for other documents.

Sources:[89] David Arias[90] David Arias

A Claude Code proxy that only raises the tier

Engin Diri’s jev-router, at commit f909310, asks Jev when a person writes a new message. The default policy can raise the session’s tier and does not lower it. Tool steps stay on the tier that message received. The Claude-only config maps the Choice labels to Haiku 4.5, Sonnet 5 and Opus 5.5. Accept bars in that config are 0.85, 0.6 and 0.3. A sensitive Noul at or above 0.7 sends the message to the top tier. The TypeSafe channel pins jev-1.13.0.

The README says those bars have not been tuned on real Jev answers. A 24 September check mocked Jev. Full Codex sessions were not in the live checks. The Codex profile in the same repository maps the frontier tier to gpt-6-astra. This proxy is separate from the compaction plugin.

Sources:[95] Engin Diri[96] Engin Diri[97] Engin Diri

A difficulty tier before Claude Haiku 4.5 or Sonnet 4.6

maslul 0.5.0 can ask Jev for a simple, medium, or hard tier and then call the model configured for that tier. The published example uses Claude Haiku 4.5 for simple and medium and Claude Sonnet 4.6 for hard. The classifier calls jev-latest. Confidence below 0.7, or a failed call, returns no tier, and the router continues on its own strategy. The example’s default level is hard.

The hook is off unless the application passes it in. The example file leaves the Jev extra off. A comment in jev.py says abstract level descriptions returned medium at 0.48 to 0.78 for a moving plan, a savings split, and an insurance comparison, in English and in Hebrew. The concrete defaults returned hard at 0.97 to 0.99, and the easy requests stayed simple. The post’s 0.3 second figure is not recorded in the client, which sets a 5 second timeout. The README names Gemini, Grok, and OpenAI as other providers. The shipped example and the wording check use the Claude config.

Sources:[119] Ilia Tankelevich[120] Ilia Tankelevich[121] Ilia Tankelevich[122] Ilia Tankelevich

EosHQ falls back to Gemini 3 Flash Preview

The EosHQ article times agent selection and automation detection with Gemini 3 Flash Preview still in the path. On 18 decisions each, the Jev path including fallbacks takes 1,616 ms against 3,380 ms, and 925 ms against 3,432 ms. Six agent decisions and three automation decisions still called Gemini. One invoice-trigger example needed Gemini every time and was slightly slower.

The article does not name a Jev version. Label selection is one observed request, 3.07 seconds to 0.455 seconds, and the article says the Gemini side of that pair also changed the prompt. The samples come from EosHQ’s synthetic harness, collected on 28 September 2026.

Sources:[100] Kishore Mandyam[101] EosHQ

A confidence handoff to Claude Opus 5

On the Banking77 test split, OpenRouter’s write-up keeps Jev 1.13 when confidence is at least 0.90 and sends the rest to Claude Opus 5 with reasoning off. That leaves 75.9% of the 3,080 utterances on Jev, at 84.0% accuracy and $0.69 per 1,000 requests, against 84.4% and $2.42 for Opus alone. Jev alone is 81.0% at $0.11 per 1,000.

The cascade table uses the same 3,080 examples as the accuracy measurement, so Rogers calls the result an upper bound. Confidence ranked the errors and overestimated accuracy in the middle of the range. The 0.90 cutoff belongs to this write-up.

Sources:[87] Kenny Rogers

An experimental Stagehand act path

An open Stagehand pull request runs Jev before the existing act model and falls back to Gemini 3.8 Flash when Jev is not confident. On 40 tasks the baseline scores 39/40 and three trials of the Jev arm score 118/120, with median latency from 1.97 seconds to 0.46 seconds. The path stays off unless experimentalJevAct is set. Thresholds were tuned on the suites, and turning the language model off drops the act suite to 27/40.

This is a different project from Jev Ultrafast. A companion observe pull request is also open and off by default. Kyle Jeong’s $0.001 figure is one remote-browser task, not the act-suite median.

Sources:[43] miguelg719[44] miguelg719

Accept a confident judge verdict

The Carnegie Mellon paper freezes a rule before scoring 510 later preference pairs. It keeps Jev when the maximum label probability clears a threshold fit on 96 earlier pairs, and sends the rest to the fallback. With GPT-6 Astra at 0.9, the cascade accepts 53.7%, scores 92.5% against Astra’s 93.1%, and uses 56.8% of Astra’s fee on reported usage. The paper’s closing line rounds that pair to 99% of Astra’s accuracy at 57% of the fee.

At 0.7, the GPT-5.6 policy accepts 81.0% of pairs and loses 2.35 points, past the two-point tolerance used to pick the threshold. The single-order curve reads the threshold off the items it scores. The paper keeps that curve separate from the frozen rules. The paper says live sequential latency for the cascade is still an open measurement. The Claude and Gemini fallback rows are on the benchmark page.

Sources:[102] Yubo Li, Yidi Miao, Ramayya Krishnan and Rema Padman

Filter scraped chunks before gpt-5.4 writes

GPT Researcher’s eval note, at commit 2906e632, scores each scraped 1,000-character chunk with Jev and keeps up to 10 chunks at or above 1.5. gpt-5.4 then writes the report. On 28 saved tasks, an independent gpt-5.4-mini call marks 73% of those chunks relevant, against 46% for an embedding filter. Blind comparisons, counted only when gpt-5.4 agrees in both orders, are 15 wins, 10 ties and 3 losses. Cost per report is $0.115 and $0.117.

The client sends jev-latest unless JEV_MODEL is set, and the note does not record the resolved version. With no TypeSafe key, select.py uses a BM25 keyword fallback: precision 0.51, and 14 wins, 8 ties and 6 losses against embeddings. On the eight open-ended questions, the unfiltered report wins 6 of the decided comparisons with Jev. The note says SimpleQA is saturated, and that the writer and the pairwise judge are the same model family.

Sources:[103] Assaf Elovic[104] Assaf Elovic

Routing needs information about the candidate models

Jing Wang’s router experiment changed its selected model after another candidate was added. Wang points out that the context supplied no direct price or performance information. The test includes no measured cheapest-model policy and no stable quality ranking.

Implementation details are in the linked project documentation. These reports describe setups the authors say they ran.

Sources:[21] Jing Wang

Read Decision Index scores as four separate tasks

Artificial Analysis’s Decision Index v1.0 mixes knowledge, reasoning, tool calling and safety classification at 25% each. Jev 1.13 leads the decision-model column at 58. Claude Haiku 5.5 (Low) scores 61 at 1.4 seconds per task and $184 per million tasks, against 0.13 seconds and $42. GPT-6 Luna (Low) scores 55 at 1.5 seconds and $131. Jev’s knowledge accuracy is 35 against 52 for Haiku and 66 for Luna. On When2Call Jev scores 81 against 73 for Haiku and 77 for Luna.

Language models answer with structured JSON and give no probability for the other options. A decision model answers every question in one call. These figures are the 13 October 2026 launch snapshot. The live leaderboard can change.

Sources:[139] Artificial Analysis[138] Artificial Analysis[140] Artificial Analysis

Compare Jev with Claude 5.5 on three Japanese sets

driller runs jev-latest and Claude Haiku, Sonnet and Opus 5.5 through the same Choice interface on MASSIVE, chABSA and JNLI. After rewriting the option text to match the annotation guides, McNemar finds Opus ahead of Jev on JNLI. Jev’s median is 0.16 to 0.17 seconds. A cascade that sends low-confidence Jev items to Sonnet or Opus is a replay of those labels, and the author calls the resulting accuracies optimistic.

The first chABSA pass used a neutral description that did not match the guide, and rewriting that one description moved every model up 6 to 10 points. The article treats that rewrite as a check on the question, not as a second independent test of the models.

Sources:[142] driller[141] driller

Keep Luna for recovery when Jev picks the next action

Ling, Yu, Chen and Fang replace gpt-5.6-luna action selection with jev-1.13.0 in five agents. Decision cost and time fall in every family. ALFWorld success stays within about two points of the original agents. Two AndroidWorld agents fall by 13.22 and 18.10 points. Auxiliary reasoning, extra environment steps, failed recovery and early “done” claims absorb part of the decision saving. T3A’s cost per delivered success rises even though spending per attempt falls.

Reflexion-J wins more first trials than the Luna agent, then recovers 11 of 19 initial failures against 25 of 26. The paper’s practical test is successful delivery, recovery, and resources per delivered outcome.

Sources:[143] Yuchen Ling, Shengcheng Yu, Zhenyu Chen and Chunrong Fang

Ask the yes/no judge both ways

Arize’s eight-model bake-off scores Jev 1.13 at 85.5% balanced accuracy on RAGTruth against 81.5% for OpenAI Decisions on gpt-6-luna, with overlapping intervals. Reversing the hallucination question, from “Is every claim supported?” to “Does it contain an unsupported claim?”, leaves Jev, Clef 27B and OpenAI near their original ROC. Liquid d1 drops from 0.92 to 0.17. The blog treats the score as belonging to the model and the question together.

OpenAI received labelled text sections, not Jev’s named-field layout. Arize says a different layout could score differently.

Sources:[145] Jim Bennett[144] Arize AI

Route a company agent roster with Jev

Sanity’s @s harness treats each agent as a Studio document and asks Jev which published agent should handle a Slack message. At the time of writing, Jev chose among 66 agents plus one option for questions about @s itself. Confidence below 0.4 returns the three most likely agents for the user to pick. The state is the message plus up to the last 20 messages in the thread. Follow-ups stay with the current agent unless the new question belongs elsewhere. Watched channels get a yes-or-no on whether the agent should reply.

About 42 of 61 hackathon agents worked fully or partly. Most failures were missing access, not routing. HAL, a meeting-hygiene agent, answered everyone when its description said “workstreams,” and the team tightened that description. The post names no Jev version and prints no router accuracy. An earlier docs-agent post times the same classifier idea against Haiku: entry choice 282 ms against 1,328 ms, with openai/gpt-oss-120b writing the answer.

Sources:[197] knut[198] TypeSafe AI[199] Even Westvang, Rostislav Melkumyan and Knut Melvær[200] John Siciliano and Knut Melvær

Judge a loaded page before sending it to an LLM

Lightpanda put Jev inside the browser binary and built page.judge() on top of it. One call asks whether the page has real content and what kind: article, listing, product, or docs. The launch post times that call at about 0.2 seconds, and a video of 30 sites at about 14 seconds total. HTTP 200 pages can still be cookie walls, CAPTCHAs, logins, paywalls, error pages, empty searches, or pages still loading.

The page-classify Python example calls judge() once per URL with a rate-limit Noul and a kind Choice, then backs off, accepts a cookie wall, or waits for a page that is still loading. The file is on a branch. A comment says judge() is not on PyPI yet. The example records median and p90 Jev times at runtime and does not print those numbers in the file.

Sources:[201] Lightpanda[202] TypeSafe AI[203] lightpanda-io

Call Jev from a UiPath workflow

UiPath’s TypeSafe connector is in preview in Integration Service. A workflow uses the customer’s own TypeSafe API key. Requests do not go through the UiPath AI Trust Layer. Search terms in Maestro are Jev, TypeSafe, or System One. Choice, Score, and Noul are the three question types.

UiPath’s internal evaluation reports 94% agreement with an LLM-as-a-judge system, 7.7 times lower cost, and 4 to 20 times faster than that judge. Those figures are labelled early internal experiments. The post names no Jev version, no public log, and no dataset size. TypeSafe’s quote-tweet called the pairing about an order of magnitude cheaper and faster.

Sources:[204] UiPath[205] TypeSafe AI[206] Sandeep Panda

Front TypeSafe with a gateway

Zuplo’s 20 October 2026 changelog adds Jev as an AI Gateway provider. The catalog lists jev/jev-latest, jev/jev-1.13.0, and jev/jev-preview. Requests go to /v1/systemone under the app URL, priced at TypeSafe’s $0.042 per million input tokens, with output free. Data Loss Prevention, Akamai AI Firewall, and Prompt Injection Protection block System One requests by default as uninspectable. Setting onUnknownShape to skip forwards them uninspected.

Sources:[209] Zuplo[34] TypeSafe AI

Keep combined state tracking on a model that writes steps

Yang and coauthors find that jev-1.13.0 retrieves and selects records without error up to 20k tokens and 64 options, then loses accuracy once the answer depends on two or more updates of one user. Final-state accuracy at sixteen updates is 53 to 80%. A stale summary is followed even when the log contradicts it. On 738 real items, Jev answers 97.5% of the questions that one forward pass of Qwen3.8-27B solves and 74% of the others.

Confidence flags Jev’s own errors and not errors inherited from a stale input. An offered hand-off option is taken on 1.6% of those real-task items. The paper’s practical split is: compute and check the state with a model that writes intermediate steps, then let Jev select from that state.

Sources:[210] Wang Yang, Xinpeng Li, Yanyan Zhang, Chuang Ma, Shouren Wang, Jianzhe Wei, Liqin Ye, Xiaoran Cai, Mohsen Hariri, Vipin Chaudhary and Xiaotian Han

Rank warehouse tables, and keep the full schema in the prompt

ClickHouse’s 27 October 2026 Agentic Analytics post asks Jev one yes/no question per warehouse table. On a 136-question development set over 18 tables, mean nDCG@10 is 0.951, recall@10 is 0.991, and the top five tables already hold every required table for 84% of questions, in about 0.4 seconds. A 250-case pilot on catalogs with 75 to 710 tables reaches 87% recall@10 and finds every required table in 73% of cases.

The same post labels 277 failures from a four-model development run in one System One call per trajectory. Wrong data selection is the largest bucket at 87 of 277. Feeding the ranking back as an in-loop hint leaves accuracy flat across 20 models: pooled net +12 on fired cells, with a 95% interval from −18 to +42. The ranking stays; the hint is the negative result. Code is in ClickHouse/data-agent-mnist at commit 403f3e7.

Sources:[225] Eduardo Vellasques, Lionel Palacin and Al Brown[224] ClickHouse[226] ClickHouse

Install Jev as a skill in Muse or Muse Code

TypeSafe’s coding-agents page lists Muse Spark among coding-agent LLMs that Jev does not replace. The agent still writes and calls tools. Jev answers typed questions the agent sends. aaddrick’s unofficial skill copies into Muse’s ~/workspace/skills/ folder and installs in Muse Code with muse skills install. TypeSafe’s own skill page names Claude Code and npx skills add. theglitcharchitect/muse-skills ships a shadow-mode router that logs a verdict before expensive steps and still escalates spending, sending, deleting, and publishing to a person.

@0xCodila’s playbook and @jkpelaez’s CRM note describe that split in Muse between 22 and 30 September 2026. Both posts name Muse.

Sources:[162] TypeSafe AI[163] TypeSafe AI[169] aaddrick[170] theglitcharchitect[165] codila[168] jkpelaez

Pictures stay outside the Jev call unless the images header is on

Screenshot text, filenames, and per-pixel color questions are collected in the image report. The models table still lists Input as Text. On 14 October 2026 TypeSafe documented image files behind TypeSafe-Danger-Zone-Images: true, for jev-1.13.0 and jev-1.14.0. The Jev Day images article covers that header, the assets map, and the size limits.

Sources:[34] TypeSafe AI[47] TypeSafe AI[155] TypeSafe AI