Sources behind the Jev reports
We read the original X posts, follow-ups, and linked project docs before publishing. The cards paraphrase those sources. A source check records what the author reported. We have not rerun the experiments. Each measurement keeps its task, model, and sample.
Source notes (234)
TypeSafe AI
Source checked:
[1] Jev and the Choice, Score and Noul primitives
The documentation describes Jev as a System One model that evaluates state and typed questions. Choice selects an option, Score evaluates a rubric, and Noul returns a probability for a statement.
API behavior documented by the provider. The page describes the three question types and includes no latency, accuracy, or cost comparison.
TypeSafe AI
Source checked:
[2] Confidence, probabilities and fallback decisions
Choice and Score include confidence derived from their probability distributions. Noul has no separate confidence field. TypeSafe recommends choosing thresholds for the domain and consequences of an action.
Implementation reference. A high confidence value can still sit on a wrong decision.
paulwei (@coolish)
Published:
Source checked:
[3] Trying Jev after GPT-6 Astra in Slay the Spire 2
The author describes GPT-6 Astra as capable but slow in an earlier game run, then reports roughly 0.7 seconds of action deliberation with Jev.
Personal game experiment with a video. The 0.7 seconds comes from that one session.
paulwei (@coolish)
Published:
Source checked:
[4] Astra revises Jev prompts after a failed game run
The first Ascension 10 attempt reached floor 6. The author says Astra then iterated on Jev prompts and the pair reached floor 17, while describing Jev as weaker at the game than Astra.
Follow-up by the same author. The reported run stopped at floor 17.
Sac (@Saccc_c)
Published:
Source checked:
[5] Codex and Jev for adding a Mac calendar event
Sac describes a Codex + Jev computer-use experiment and compares adding the same Mac calendar event. The reported experience is smoother, with similar token consumption.
Qualitative report and comparison video. The GPT version is unnamed, and the post has no repeated latency measurement.
tamara (@tamarajtran)
Published:
Source checked:
[6] Scoring tool calls for context compaction
The project author proposes using Jev to score tool calls and discard irrelevant material during compaction.
Project announcement. The repository says its native demo app is scripted and makes no API calls, so the animation has no timing to cite.
Alex Volkov (@altryne)
Published:
Source checked:
[7] A user report of Claude context reduction
Volkov reports reducing a Claude session from nearly one million tokens to about 86,000 in roughly one second using fast-jev-compaction.
One session. The post includes no downstream task score. The timestamp is the edited-post time shown by X.
tamaratran
Source checked:
[8] fast-jev-compaction implementation and limitations
The plugin scores tool calls and results, preserves user and assistant text in its output, and keeps calls paired with their results. Its hook falls back to built-in compaction on errors or insufficient reduction.
The README documents early-access Claude Code function hooks, estimated token sizes, and a scripted native demo. The author warns that a probability can be high on a tool result that is unsafe to delete.
nikhil mudholkar (@nikhilmudholkar)
Published:
Source checked:
[9] Business-email classification against Gemini
Mudholkar compares Jev with Gemini on 1,565 German and English business emails in 10 categories. Gemini performs better on overall accuracy; the author is interested in Jev for its uncertainty signal.
An author-reported benchmark. Each model runs on its own. The thread has no combined Jev and Gemini pipeline.
nikhil mudholkar (@nikhilmudholkar)
Published:
Source checked:
[10] Email dataset composition and shared instructions
The dataset contains 1,201 real emails and 364 AI-written examples for rare categories, mostly in German. The three models receive the same email text and category instructions.
Methodology from the benchmark thread. 364 of the emails were written to fill rare categories.
nikhil mudholkar (@nikhilmudholkar)
Published:
Source checked:
[11] Email accuracy results for Jev and two Gemini models
Reported overall accuracy is 96.4% for Jev, 97.5% for Gemini 3.5 Flash-Lite and 98.5% for Gemini 3.8 Flash. Equal weighting of categories yields 92.0%, 94.6% and 96.9%, respectively.
Scores against the author’s reference labels on this dataset.
nikhil mudholkar (@nikhilmudholkar)
Published:
Source checked:
[12] Where the email-classification mistakes occurred
The author reports that 737 predictions at 99% confidence or above matched the reference labels. Below 70% confidence, nearly half were wrong.
The confidence split comes from this dataset alone.
nikhil mudholkar (@nikhilmudholkar)
Published:
Source checked:
[13] Reported cost per 1,000 business emails
The author reports $0.08 for Jev, $0.80 for Gemini 3.5 Flash-Lite and $1.79 for Gemini 3.8 Flash per 1,000 emails in these runs.
Costs for this workload, counted per 1,000 emails. They are separate from the provider’s list price and from the bill for all 1,565 emails. The author notes that savings are small at ordinary inbox volumes.
nikhil mudholkar (@nikhilmudholkar)
Published:
Source checked:
[14] Attachments and synthetic examples in the email test
Attachments were excluded, and 364 emails were generated to match chosen labels. The author describes the lack of non-text input as a barrier to production use.
Limitations supplied by the experiment’s author. Attachments were left out of the run.
Rahul Kumar (@rahulbuildsmore)
Published:
Source checked:
[15] Jev and Gemini labeling 1,000 app reviews
Kumar compares sentiment, topic, bug and churn labels on 1,000 app reviews. He reports Jev at 4.6 seconds and $0.023, versus Gemini 3.8 Flash at 18.8 seconds and $0.158.
The post links to a public project with recorded runs and measurement notes. The two models are compared side by side, and Kumar does not chain them into one workflow.
goodrahstar
Source checked:
[16] Column Race recorded runs and evaluation methodology
Both lanes process 20 reviews per request at concurrency 8. The recorded pair uses Jev 1.13.0 and Gemini 3.8 Flash. Sentiment correlation with star ratings is 0.803 and 0.822; topic agreement is 84.4% and bug agreement is 92.6%.
One recorded pair over 1,000 Android reviews. Stars are a weak sentiment proxy, and topic and bug labels have no human ground truth, so the agreement rates only say the models matched each other. Costs use token counts and list prices. The repository has no invoices.
Dan McAteer (@daniel_mac8)
Published:
Source checked:
[17] Ranking X posts with Jev and Grok Bot
McAteer describes retrieving 1,000 agent-related posts with the X API, using Jev for pairwise selection, and having Grok Bot surface five tips.
A first-hand workflow description. No model version, comparison count, latency or cost breakdown is supplied.
codila (@0xCodila)
Published:
Source checked:
[18] A Jev router in front of GrokBot actions
The author describes consulting Jev before browser, research and retry actions, starting in shadow mode, inspecting logs, and retaining a bypass and human control over irreversible actions.
A personal setup report with promotional language. The broad performance claims and the seven-minute setup claim have no measurements attached.
Hassan (@nutlope)
Published:
Source checked:
[19] Jev first-pass email screening with Kimi K3 fallback
Hassan tests 50 legitimate and 50 fraudulent emails. Jev classifies them in 1.42 seconds; 31 cases below a 95% confidence threshold go to Kimi K3. The full pipeline takes 16 seconds, gets 96 of 100 correct and costs about $0.07.
Author-reported demonstration, not independently rerun here. The stated cost split is $0.003 for Jev and $0.068 for Kimi. The 1.42-second figure excludes the Kimi stage.
yonsakhan (@yonsakhan)
Published:
Source checked:
[20] Using Kimi K3 to develop a Jev-powered comment filter
The author says they developed an X comment-filtering userscript with Kimi K3 and use Jev for semantic spam decisions, including obfuscated text. They describe reversible collapsing and a local cache.
A development report. Kimi helped write the software. Runtime classification is attributed to Jev.
Jing Wang (@jingwangtalk)
Published:
Source checked:
[21] A model-routing experiment sensitive to the candidate list
Wang experiments with a Jev router over DeepSeek, GLM, Kimi, GPT and Claude candidates. Adding Claude Opus 5 changes the selected model from DeepSeek Flash to Claude Sonnet 5. The author says price and performance information was missing from the supplied context.
A small exploratory test. A name on the candidate list shows what the router was offered. The test includes no finished integration for each name and no measured routing policy.
LimboAI (@limbopeng)
Published:
Source checked:
[22] Refactoring intent recognition with Jev and DeepSeek
LimboAI reports replacing intent-recognition work in a project with Jev alongside DeepSeek V4 Flash and describes a positive experience with speed and cost.
Qualitative personal feedback. No dataset, baseline, elapsed time or billing totals are given.
Yanhua (@yanhua1010)
Published:
Source checked:
[23] Searching train services with Pi, DeepSeek and Jev Ultrafast
Yanhua describes running Browser Use’s Jev Ultrafast with Pi and DeepSeek to look up train services on 12306 and organize the results.
Personal browser-automation demonstration. The post leaves out the DeepSeek version and any numerical cost, and it stops at the search.
Browser Use
Source checked:
[24] Jev Ultrafast action selection and text generation
The agent builds an indexed element table, asks Jev for an operation and compatible target, and calls a text model for TYPE_TEXT. The README describes an OpenAI-compatible helper configurable for Gemini, GLM and DeepSeek.
Project README. We did not rerun Yanhua’s setup. The checked README defaults to Mercury 2.5, requires independent outcome checks, and lists unsupported browser features.
Diogo Almeida (TypeSafe AI) (@typesafeai)
Published:
Source checked:
[25] Introducing System One models and Jev
The launch post describes Jev as a function-call style model: unstructured state in, typed probabilistic decisions out. It lists input tokens at $0.042 per million ($42 per billion), free output tokens, and a 70-500ms response time from request to answer.
Provider announcement covering System One models, RLCD, parallel sampling, and workflow-eval claims of 193.6× faster and 444.6× cheaper versus an LLM baseline. The figures come from TypeSafe’s evals. This site has not rerun them.
TypeSafe AI (@typesafeai)
Published:
Source checked:
[26] Jev is available without a waitlist
TypeSafe announced that Jev is available to everyone, with no waitlist, and pointed new users to the console login.
Public-availability post five days after early access. A follow-up stated that every account starts with $5 in credit, described as about 120 million tokens.
TypeSafe AI (@typesafeai)
Published:
Source checked:
[27] Starter credit for new TypeSafe accounts
TypeSafe stated that all users start with $5 in credit, described as about 120 million tokens.
Posted alongside the no-waitlist announcement. At the listed $0.042 per million input tokens, that credit covers a large volume of Choice, Score and Noul calls. The token equivalent is the author’s figure.
Vercel (@vercel)
Published:
Source checked:
[28] Jev adoption on Vercel AI Gateway
Vercel reported that Jev was adopted faster than any other model in AI Gateway history, reaching about 13% of teams on the first day, twice the GPT-5.6 family and six times Fable 5.1.
First-day Gateway adoption statistics from Vercel. The post describes Jev as typesafe-ai/jev through AI SDK 7 experimental_evaluate. The percentages are shares of Gateway usage.
Vercel Developers (@vercel_dev)
Published:
Source checked:
[29] Promotional Jev pricing on Vercel AI Gateway
Vercel Developers announced that Jev was free on Vercel AI Gateway until 25 September.
Promotional Gateway pricing. Native TypeSafe list pricing remains $0.042 per million input tokens with free output, as documented on the model card.
Venice (@AskVenice)
Published:
Source checked:
[30] Jev listed on the Venice API
Venice announced a beta listing of Jev on the Venice API and described it as a model that answers typed questions from application state.
Third-party API listing restating the System One contract: a probability, a chosen option, or a score on a rubric. The post includes no latency or accuracy numbers.
Sasha Sheng (TypeSafe) (@hackgoofer)
Published:
Source checked:
[31] Known limits and jaggedness in Jev 1.13
TypeSafe cofounder Sasha Sheng wrote that Jev is still too slow, too expensive and too limited, and asked the community to post issues in the Discord model-jaggedness channel.
First-party note on known unevenness in Jev 1.13. Official docs also list accuracy drops as state grows and weaker non-English performance.
小码哥 (@xmglab)
Published:
Source checked:
[32] Using Jev with Claude Code and Codex
小码哥 describes wiring Jev into Claude Code and Codex: create an API key, set TYPESAFE_API_KEY, and install the official skill so the coding agent executes while Jev decides.
Field note on the official TypeSafe agent skill. The post describes the workflow and includes no timings or invoice.
Beyond (@BeyondWinWinWin)
Published:
Source checked:
[33] Cockpit scoring run with 15,000 Jev decisions
Beyond reports 15,000 Jev decisions on cockpit data for about $1, with high scores on gauge-limit, option-count and tower-instruction items, and coin-flip performance on a rate-times-time task. A 36/36 landing case is described as Jev reading state while other code flies the aircraft.
Independent high-cardinality scoring run. The author treats cheap bulk decisions and calibrated uncertainty as the point, and notes that surrounding code still owns the action.
TypeSafe AI
Source checked:
[34] Jev 1.14 and 1.13 model card: price, context and aliases
The models page lists jev-1.14.0 and jev-1.13.0 at $42 per billion / $0.042 per million input tokens, with output free. Default context is 64k tokens per request and 32k per question. The table lists Input as Text for both. Jev 1.14 is dated as available from 14 October 2026. Both jev-latest and jev-preview point at jev-1.14.0. A warning on the page says there is no preview build. jev-1.13.0 stays available as a versioned id.
Canonical model card as accessed 26 October 2026. On 15 October 2026 the same page pointed jev-latest at jev-1.13.0 and jev-preview at jev-1.14.0. Audio and video are excluded here; convert those to text or structured fields first. Image files and 262k context are documented under danger-zone headers on this page and on the Images page. Rate limits are per organization in the developer console.
躺在废墟里 Lieflat (@lieflat_3)
Published:
Source checked:
[35] Labeling awkward metaphors in three AI-written articles
The author asked Jev, Claude Opus 5, Claude Opus 4.8, GPT-5.6 Terra, GPT-5.6 Luna and Gemini to find awkward metaphors in three AI-written articles. On one 202-sentence article, Jev labeled every sentence in 2.6 seconds. Opus 5 in non-reasoning mode took 8.2 seconds, Gemini 88.3 seconds, and Terra and Luna each took more than 160 seconds. Of 10 sentences with explicit comparison markers, Jev marked none and Opus 5 marked 5. The sentence Jev scored as the worst metaphor was a definition of the g factor. The author counts that sentence as a definition.
The labels are the author’s judgment. The post leaves Gemini’s version unnamed. Jev scored every sentence. The other models returned only the sentences they flagged. A repeated ruler metaphor received Jev scores of 0.32, 0.31 and 0.30.
Melvyn (@melvynx)
Published:
Source checked:
[36] Switching email classification back to Gemini
Melvyn says Jev did poorly on many of his tasks, that email classification was bad, and that he switched that work back to Gemini.
A reply to Nevo David. No mailbox size, label set or accuracy figure is included.
Nevo David (@wickedguro)
Published:
Source checked:
[37] Hand-written posts marked as AI by Jev classifiers
Nevo David says he does not use AI to write his posts, and that Jev classifiers he tried still tagged them as AI-written. He asks for a comparison with Fable and Astra and says Jev may fit simpler classification.
Personal experience in a reply thread. The post states that Jev is 100x faster and cheaper without attaching that ratio to a test. No sample size is given.
Serg (@karakhanyanS)
Published:
Source checked:
[38] A report that 70% of one author’s posts were flagged
Serg replies that the same kind of classifier marked 70% of his posts as AI slop.
A reply in Nevo David’s thread. The reply leaves the number of posts and the classifier setup undescribed.
Jamie Watters
Source checked:
[39] 42-claim test of whether a cited passage supports a sentence
Watters scores 42 claim-and-passage pairs: 24 constructed controls and 18 real published sentences. The schema-only run reports 85.7% for Jev, GPT-5.4 and Claude Sonnet 5, and 81.0% for Gemini 3.1 Pro. On the hard 18, the README shows 77.8%, 66.7%, 66.7% and 55.6%. A scratchpad rerun of the frontier models leaves those hard-set scores no higher. Unsupported-claim recall for Jev is 78.9%, short of the 80% bar set before the run.
One run per model, labels from the author’s own pipeline, frontier calls through OpenRouter. The table names jev-1.13.0, and the known-weaknesses section says the calls used the jev-latest alias. Costs come from the repo’s price file. McNemar p-values on the hard-18 scratchpad change are 1.000, 0.219 and 1.000. At those p-values the README stops short of calling the drops significant. For GPT-5.4 and Sonnet 5, the verdict call transcribes a short scratchpad.
0xNatoshi
Source checked:
[40] Per-turn Codex model routing with Jev
The router asks Jev, on each Codex model call, whether Astra is mandatory, which of Luna, Terra, Sol or Astra to use, how much thinking to allow, and how long to keep the route. Architecture, independent final review and risk-focused review can force Astra. Errors fail open to Astra at medium effort. A shadow file logs the choice and Astra still answers.
Repository documentation read at commit 8701ef7. The figure of about 60% less than full Astra is a simulation of 237 turns under an older policy. The README says it was not read from a Codex quota, and that the current policy has no matching figure. Later credit estimates use published rates, with no account debit in the file. The author says outcome labels are still missing.
Kyle Jeong (@kylejeong)
Published:
Source checked:
[41] One remote-browser task with Stagehand and Jev
Jeong describes a Stagehand loop that sends an accessibility tree as state and candidate actions as questions, lets Jev pick the next action, and reports about $0.001 for one remote-browser task. A follow-up shows a jev-latest request with operation, click, input and completion questions.
One demonstrated task. The thread leaves the site unnamed, shows a single run, and leaves the typing model unnamed. The later act-suite timings are a separate measurement.
Kyle Jeong
Source checked:
[42] How Stagehand’s act path uses Jev
Jeong describes classifying a Stagehand instruction, asking Jev which accessibility-tree candidate to use, accepting a pick at a 0.7 threshold, and falling back to a language model otherwise. Early testing is reported as a drop in Act median latency from 1.97 seconds to 0.46 seconds. He also writes that Jev-only agents made poor production computer-use systems in Browserbase’s trials.
Engineering post. The latency sentence points at the open Stagehand pull-request stack for the underlying runs. The 0.7 figure is this post’s threshold, and the pull request describes a more detailed acceptance rule.
miguelg719
Source checked:
[43] Experimental Jev decision tree for Stagehand act()
The pull request compares an experimental Jev act path plus Gemini 3.8 Flash fallback with the existing Gemini 3.8 Flash act path. On 40 tasks, the baseline is 39/40 and the Jev arm is 118/120 across three trials, with median latency 1.97 seconds to 0.46 seconds and 4 of 147 acts calling the language model. A 40-task breadth set outside the pull request moves from 35/40 to 39/40 and from 3.19 seconds to 0.77 seconds. With the language model disabled, act scores 27/40.
Open pull request on 22 September 2026. The path is off unless experimentalJevAct or STAGEHAND_EXPERIMENTAL_JEV_ACT is set. Thresholds were tuned on these suites. Jev token use is logged but not yet included in result metadata. Breadth at three trials on two models is reported as 204/240 to 229/240.
miguelg719
Source checked:
[44] Experimental Jev observe() and cache check
The observe pull request reports a suite move from 9/12 to 10/12 against Gemini 3.8 Flash. Eleven of 16 observes were answered by Jev in 0.18 seconds, against a 1.99 second baseline, and 5 of 16 went to the language model. With the language model disabled the suite scores 7/12.
Open pull request on 22 September 2026, off by default. The cache-check threshold of 0.35 comes from API probes, and no eval runs that path end to end.
Eastwood (@Tsj_estwld)
Published:
Source checked:
[45] SemEval-2026 DimABSA scores for Jev, Kimi-K2 Thinking and Qwen
Eastwood reports Jev 1.13.0 on all ten SemEval-2026 DimABSA subtask 1 test sets. Equal-weight mean RMSE_VA is 2.4750 for untuned Jev, 2.1889 for a Qwen3-14B QLoRA fine-tune, and 1.8731 for the official Kimi-K2 Thinking one-shot result. That is 0 wins and 10 losses against Kimi, and 3 wins and 7 losses against Qwen. Mean error is 32.1% higher than Kimi’s.
Lower RMSE_VA is better. The aspect is given, and the model scores polarity and activation. The chart caption says results use the official test metric without normalization, and that training and example conditions differ across methods. Kimi-K2 Thinking and the Kimi K3 in Hassan’s email demo are different models. The per-dataset chart shows Jev higher than Kimi on every set.
matyas (@Matyas_Jirsa)
Published:
Source checked:
[46] Sixty-email comparison with DeepSeek V4.1 Flash
The attached table compares DeepSeek V4.1 Flash and Jev on 60 emails. Both score 60/60. Median latency is 3.361 seconds versus 0.313 seconds, average latency 6.600 versus 0.348, and the 95th percentile 19.105 versus 0.465. Cost for the 60 is $0.01912 versus $0.00278, with zero API or schema errors on both sides.
The post text scales the same rates to 10,000 emails, about $3.19 versus $0.46 and about 9 hours 20 minutes versus 52 minutes. Those durations equal the median latency multiplied by 10,000. The post shows no log of that batch. The corpus, label rules, and Jev version are unnamed. DeepSeek V4.1 Flash is the image title. LimboAI’s note names V4 Flash, a different label.
TypeSafe AI
Source checked:
[47] Jev state is a string, object, or array
State is a string, a JSON object, or an array. The page says audio and video are not supported. English is the primary training language; other languages, including CJK scripts, are accepted at lower accuracy.
Input contract for the hosted API as accessed 15 October 2026. This page does not document image files. Image assets are on the Images page.
TypeSafe AI
Source checked:
[48] Jev 1.13 jaggedness: math, dates, color, and generation
The jaggedness page for jev-1.13 says Jev is not a calculator, reads dates as text, and is not trained to generate text. Color questions do better with English names than with hex or RGB triples. Accuracy falls as the state grows with content unrelated to the decision. Last reviewed 2026-10-02.
Official limits for jev-1.13. Arithmetic, date order, and counting belong in code. Passing a named color is a step in the calling code. The page includes no case of Jev inspecting an image.
keno (@kenonews)
Published:
Source checked:
[49] A guide to extracting evidence before Jev decides
keno writes that the API takes text, so an application can pass extracted text, objects, or other details. The post walks through OCR, hosted vision APIs, local vision-language models, detectors, embeddings, color summaries, and existing metadata. He says the visual description comes from that earlier step.
A design guide. The sample evidence record is an example he wrote. The labeled comparison of rules, OCR plus Jev, and a vision model has no results in the post.
keno (@kenonews)
Published:
Source checked:
[50] Screenshot OCR, then a support-queue choice
The post shows a screenshot going through local OCR, after which Jev classifies the issue and chooses a support queue. It points back to the longer guide for local tools, free-tier APIs, and where each approach falls short.
A short demonstration of the same author's guide. The video publishes no labeled accuracy figure.
rezoch340
Source checked:
[51] WeChat window capture, local OCR, and Jev judgments
The Windows companion screenshots the WeChat window, reads it with local RapidOCR, asks Jev seven judgment questions, has a language model draft three replies, and asks Jev to rank them. Pasting into the input box is manual. Text on photos and stickers is dropped when the background is not flat.
Checked at commit 033d6ef. Default judgment models are typesafe/jev-1.13 on OpenRouter and jev-latest on the TypeSafe API. The README's 250 to 800 ms figure is one OCR pass during capture.
Jeremy Person (@jeremyperson)
Published:
Source checked:
[52] A personal mockup that uses image filenames
Jeremy Person says a personal mockup uses image filenames and no metadata. The post names Stefan's designer video as the inspiration.
A learning demo on the author's machine. The text never says the image files were opened. Stefan's caption leaves the inputs undescribed too.
Dragos Dunica (@dragos_dydy)
Published:
Source checked:
[53] Jev decides when a dinosaur toy should show a picture
Dragos describes an ESP32 toy for his nephew. Gemini Flash handles transcription and the conversation, Gemini 3.1 Flash TTS speaks, FLUX.2 Klein 4B draws, and Jev 1.13 decides whether a picture would help and whether the proposed scene is suitable for the child.
A prototype on a Waveshare ESP32-S3 Touch AMOLED 1.8, built with Astra, with the other models called through OpenRouter. One device and one child, with no latency distribution or count of rejected scenes.
Dragos Dunica (@dragos_dydy)
Published:
Source checked:
[54] Image generation runs beside the spoken answer
Dragos says image creation was parallelized. After the question is transcribed, Jev decides whether an image would help and calls an OpenRouter image model, flux.2-klein-4b. He says the picture is usually ready before the answer finishes, and that transcription and reply speed are the weak point.
A reply on the same dinosaur-toy thread. He mentions considering Gemini 3.8 Live in place of transcription plus TTS. The reply includes no timing table.
Anshu (@anshuc)
Published:
Source checked:
[55] Jevinci paints from per-pixel color choices
Anshu presents Jevinci and says the paintings are JavaScript, with no image-generation model. Jev predicts each pixel's color in parallel, and a more confident prediction paints a wider brush. The video is marked sped up 4x.
A creative demo. The 4x note means the clip is sped up. The linked repository was later renamed.
Anshu (@anshuc)
Published:
Source checked:
[56] The painting is derived from the pixel grid
Anshu says the state representation and questions took trial and error with Codex, and that Jev still answers the pixels in one shot. The painting is derived from the grid and the probabilities, while the grid remains.
Same-day follow-up by the same author. Codex here is the coding assistant used to build the demo.
achimala
Source checked:
[57] jev-paint renders Jev probability grids in the browser
The app turns Jev probability distributions into paintings. Question construction and painting run in the browser. A loopback Python helper forwards requests because the API disallows browser CORS origins. HSL and RGB channels are combined as if independent, and the renderer is described as an artistic transform.
Checked at commit ecf9c48, the repository GitHub serves for the old jevinci URL. The README says rendering time and model latency are separate, and that larger grids make more requests.
achimala
Source checked:
[58] Palette, HSL, RGB, and silhouette questions
The browser code calls jev-latest. Palette mode asks one Choice per pixel over 16 named colors. Other modes ask HSL questions, a Noul per RGB channel, or a silhouette Noul. Grid sizes are 8, 12, 16, 24, and 32, and the questions are split into batches.
Implementation at the same commit as the README. The 16-color list and the batch sizes come from this file. It includes no accuracy result.
Shridhar (@shridharathi)
Published:
Source checked:
[59] A video captioned as fast image search with Jev
Shridhar posted a 22-second video. The only text calls the clip fast image search with Jev, and the caption never says whether that used pixels, filenames, or descriptions.
The post text is the caption. Replies ask how the images were represented. The video itself has no written method.
Adam Holter (@AdamHoltererer)
Published:
Source checked:
[60] A reply that Jev would need text descriptions first
Replying under the image-search video, Adam Holter writes that Jev has no image input, so another model would have to turn the images into text descriptions first.
A reply stating that constraint. Holter gives no account of Shridhar's code, and no account of what the video did.
Xenova (@xenovacom)
Published:
Source checked:
[61] CLIP search offered in place of a multimodal Jev
Quoting the image-search video, Xenova writes that Jev has no image input and shows CLIP searching 25,000 images in under 50 ms, with no labeling step, locally in the browser on WebGPU through Transformers.js.
The 50 ms figure belongs to that CLIP demo. Xenova gives no comparison against Shridhar's video on the same images.
村山毅 Tim (@timtarou)
Published:
Source checked:
[62] An OCR follow-on compared with Gemini
Tim reports trying Jev for OCR-related work. Latency and cost were about half of Gemini, and accuracy about 80 to 90 percent of Gemini.
The post leaves out the page count, the Gemini version, and the Jev version. The follow-up narrows what the accuracy figure refers to.
村山毅 Tim (@timtarou)
Published:
Source checked:
[63] The Gemini comparison is field extraction after OCR
Tim clarifies that the comparison is the accuracy of field extraction after OCR. Character recognition on its own is a different measurement.
Same-author clarification a minute later. The sample size and the model versions are still absent.
さよ☆マギカ (@MajoSayo)
Published:
Source checked:
[64] LogJev reads logprobs from a vision-capable chat model
The author says LogJev sends an image or audio clip plus a bounded question to an OpenAI-compatible model, and turns that model's first-token logprobs into a choice. Local endpoints work when they return logprobs.
A reply on keno's extraction guide. The bridge is a separate server. The post leaves hosted Jev as it is.
DumoeDss
Source checked:
[65] LogJev image input stays on the chat provider
The server can accept image and audio parts when the selected chat model supports them and returns top logprobs. A provider configured as official Jev forwards the TypeSafe payload. The README says the bridge probabilities are uncalibrated, and its accuracy table is a 144-question text set.
Checked at commit 3e75f809, the 0.1.1 release commit. The text-set numbers are historical Python measurements of text questions.
Ed Plese (@edplese)
Published:
Source checked:
[66] A local image model presented as Jev for pictures
Ed Plese says he built a local model and reports 63 ms for a 512 by 384 image, fast enough in his words for 15 fps video. On the cake photo he lists Birthday 94%, Golden retriever 78%, Sprinkles 93%, Candles lit 67%, and the dog about to eat the cake at 52%.
Scores for the author's local model on one image. The post leaves the training set undescribed.
Ed Plese (@edplese)
Published:
Source checked:
[67] A follow-up label on a shapes image
On a follow-up image, Ed reports Left at 98% for the question of where the blue circle is relative to the red square.
Same local model, one additional image.
Nikhil Mudholkar (@nikhilmudholkar)
Published:
Source checked:
[68] Three hundred configurator requests for Jev and Gemini
Mudholkar says he tested Jev and Gemini on 300 customer requests. Asking more questions in one call cut the agent from about 7 model calls to 3 and raised accuracy for both. He says Jev had a better trade-off when a later check filtered wrong answers, and that Gemini then hit a structured-output limit.
Opening post of a seven-post thread. The model versions, the catalogue, and the counts are in the following posts.
Nikhil Mudholkar (@nikhilmudholkar)
Published:
Source checked:
[69] A fictional catalogue with 578,143 valid products
The agent chooses among eight linked dropdowns in a fictional industrial catalogue. Mudholkar counts about 120 million combinations on paper and 578,143 valid configurations, 0.48 percent.
Synthetic catalogue built for this test.
Nikhil Mudholkar (@nikhilmudholkar)
Published:
Source checked:
[70] Fan-out accuracy and Jev’s median time
On 270 solvable requests, fan-out moves accuracy from 47.0 percent to 55.6 percent for Jev and from 52.2 percent to 62.6 percent for Gemini. Jev’s median time moves from 4.60 seconds to 1.89 seconds. Model calls fall from about 7 to 3.
Gemini’s median time and both model versions are absent from the post. Accuracy here means agreement with the author’s solvable-request labels.
Nikhil Mudholkar (@nikhilmudholkar)
Published:
Source checked:
[71] A requirement check that also drops some correct answers
After fan-out, some answers satisfied the catalogue while dropping a requirement the customer had stated. The added check stops 83 wrong Jev answers and 6 correct ones, and 72 wrong Gemini answers and 20 correct ones. Mudholkar calls that about 14 mistakes stopped per correct answer lost for Jev, and 3.6 for Gemini.
Counts are on the solvable requests. The check compares explicit requirements with the final product in code.
Nikhil Mudholkar (@nikhilmudholkar)
Published:
Source checked:
[72] Gemini rejects a wide structured batch before generation
At about 300 questions per call, Jev answered and Gemini rejected all 300 test requests before generation. The input was about 55,000 tokens. Configurator questions worked at 126 and failed at 127. Simpler yes/no questions reached 329. He says Jev also answered 801 short questions in a separate email test.
He attributes the rejection to Gemini’s structured-output constraints. The post treats the million-token window as a different limit. The 801-question figure is a different email test.
Nikhil Mudholkar (@nikhilmudholkar)
Published:
Source checked:
[73] The configurator test is synthetic, and Gemini still leads on accuracy
Across the 300 requests, Gemini still leads on accuracy. With the requirement check, Mudholkar reports about one eleventh the cost per correct configuration for Jev, and says Jev wrongly rejected 70 percent fewer correct answers. He says the test is synthetic and that the check was developed on these same cases.
The one-eleventh figure has no price table in the post. A postscript says the thread leaves out further detail.
hev mind (@hevmind)
Published:
Source checked:
[74] A reranker author reports a separate batch-size limit
hev says their reranker sizing hit a batch-size limit, and that the thing to debug is structured-output enforcement. He sets the million-token window aside. The reply links their repository.
A reply to Mudholkar’s thread. The 126/127 configurator split is Mudholkar’s. The repository’s own limit is about 30 passages per Jev call.
hev
Source checked:
[75] jev-1.13.0 reranking BEIR shortlists
The file records jev-1.13.0 on BM25 top-30 shortlists. On SciFact’s 300 test queries, 30 Nouls in one call score nDCG@10 of 0.768 at $0.18 list price. Claude Opus 5 at low effort scores 0.756, with 12 of 300 queries refused and scored as zero; substituting BM25 order on those 12 gives 0.778. GPT-5.6 Luna scores 0.747, 0.355 and 0.363 on SciFact, NFCorpus and FiQA.
Generated 2026-09-17. Costs are token usage at list price. Haiku 4.5 and Opus 5 appear on SciFact only. NFCorpus has 323 queries. We have not rerun the queries.
hev
Source checked:
[76] How the Jev reranker batches documents
The reranker sends one Noul per document, up to about 30 documents, inside a request budget of about 32,000 tokens. Lists longer than that are split. The README says a probability at or above 0.9 was relevant 76 percent of the time on the SciFact run, and a probability under 0.1 was relevant 0.5 percent of the time.
Checked at commit 1eb4726. The calibration sentence matches the pair-Noul check in RESULTS.md. Hosted calls send the documents to TypeSafe.
Himanshu Kumar (@himanshu231204)
Published:
Source checked:
[77] A per-turn Claude Code router announced on X
Himanshu says Claude Code otherwise keeps one model for a session, while his router asks Jev on each new message and then keeps the tool loop on that choice. He says a failure leaves the session usable within 3 seconds, and links the repository.
Announcement post. The README at the linked commit is the source for which models are offered and for the screenshot setup.
Himanshu Kumar
Source checked:
[78] jev-model-router README at commit 8d754fa
The proxy asks Jev which of the account’s Haiku, Sonnet or Opus models should handle a new Claude Code turn. Tool calls in that turn reuse the choice. A missing key, a timeout or an error continues on a safe model within about three seconds. The README marks jev-codex experimental and says the project is not affiliated with Anthropic or TypeSafe.
Checked at commit 8d754fa on 25 September 2026. The README says the path was exercised on Claude Code 2.1.282 with 94 tests. The screenshots were captured with scripts/fake_jev.py, a local stand-in. The README presents them as captures from that stand-in.
OpenRouter (@OpenRouter)
Published:
Source checked:
[79] OpenRouter announces typesafe/jev-router
OpenRouter introduced typesafe/jev-router as a cache-aware router that uses Jev to pick a model and a reasoning effort for each request, with the stated aim of balancing quality, speed and cost.
Product announcement. The post describes the router. It does not include a score, a task count, or a price for the routed models.
Theo (@theo)
Published:
Source checked:
[80] Why a prompt is a weak input for routing a coding agent
Theo writes that Jev categorizes and does not reason, and that a prompt alone does not show the codebase, the tools, or how large the change is. His example is a request to port a project to Rust, which could be a 100-line file or a million-line application.
Reply in the thread under OpenRouter’s Jev Router announcement. The argument is about what the router can see. It is not a benchmark score.
Theo (@theo)
Published:
Source checked:
[81] An in-progress look at Jev Router’s model mix
While the DeepSWE bench was still running, Theo wrote that Jev Router was so far basically a DeepSeek 4.1 Flash router.
Interim note, posted hours before the final DeepSWE comparison. The post does not give a percentage split across models.
Theo (@theo)
Published:
Source checked:
[82] DeepSWE run of Jev Router against GPT-6 Astra on low
Theo says he spent $1,000 and stayed up until 2am benchmarking Jev Router. On DeepSWE, performance was roughly the same as GPT-6 Astra on low, the router cost slightly more, and the run took almost 5 times as long.
Author-reported comparison with the OpenRouter router. The post gives no DeepSWE score, no task count, and no dollar gap beyond “slightly more” and the $1,000 spent on the bench.
Theo (@theo)
Published:
Source checked:
[83] Follow-up after the Jev Router bench
Immediately after the DeepSWE post, Theo’s update is to use Astra on low.
One-line follow-up in the same thread. It names the setup he would use instead of the router. It does not add a second measurement.
Ali Alshehri (@AliCantCode)
Published:
Source checked:
[84] A public-field check that moved from gpt-4o-mini to a Luna model
Ali says a service that used gpt-4o-mini could switch to the model he calls gpt luna 6 after Jev was added, with responses 55% faster and cost 73% lower.
The post names the two models and those two percentages. It does not give the request count, the time window, or an invoice.
Ali Alshehri (@AliCantCode)
Published:
Source checked:
[85] Escalating a content check when Jev’s confidence is under 80%
In the same thread, Ali says Jev answers within 200ms whether public user content is appropriate, and that confidence under 80% is sent on to the language model.
Reply describing the gate. It does not repeat the model id from the first post, and it does not publish the sample behind the 55% and 73% figures.
Delip Rao and Chris Callison-Burch
Published:
Source checked:
[86] Jev Choice against three flash rubric judges
The paper compares Jev Choice with GPT-5.6 Luna, Gemini 3.8 Flash and DeepSeek V4.1 Flash on nine panels and 5,003 pairs, using the same criterion texts. Accuracy separates in 8 of 27 paired comparisons. Across the panels, Jev Choice cost $0.063 and finished in 29 seconds. Luna, DeepSeek and Gemini cost 29, 66 and 325 times as much, and took 859, 6,298 and 950 seconds.
arXiv:2609.29769, submitted 24 September 2026. Requests used the jev-latest alias. A same-day stability check logged jev-1.13.0. The main runs did not log the version. LLM judges are called once per criterion. On Jev’s most confident errors, 96.0% of the LLM verdicts repeat the wrong answer. A cascade replayed on the recorded verdicts gains at most 1.5 points over the best single judge with cross-fitted thresholds, and at most 2.0 with oracle thresholds. Jev’s concurrency was lower than the LLM judges’, so the wall-time ratios understate a same-concurrency comparison. DeepSeek’s three failed calls on the binary panels were recorded as UNMET.
Kenny Rogers
Published:
Source checked:
[87] Banking77 intents: Jev 1.13 and Claude Opus 5
OpenRouter ran the 3,080-utterance Banking77 test split through Jev 1.13 and Claude Opus 5. Accuracy is 81.0% and 84.4%, macro-F1 is 80.5% and 83.6%, median round trip is 175 ms and 2,266 ms, and the billed totals are $0.34 and $7.44.
Kenny Rogers, OpenRouter, 22 September 2026. Both models received the same one-line criteria, written from label names without looking at the test data. Opus ran at temperature 0 with reasoning off and a strict JSON enum. Eight requests were in flight. Latency is the client round trip through OpenRouter. The Opus system prompt was cached. The write-up says the uncached list rate would have been about $19 per 1,000 requests. Confidence at or above 0.99 covered 58% of utterances at 96.3% accuracy. Below 0.5 covered 3.5% at 29.6% accuracy. Sending confidence under 0.90 to Opus left 75.9% on Jev, at 84.0% accuracy and $0.69 per 1,000. That cascade table reuses the same 3,080 examples, and the write-up calls it an upper bound. Both models scored 0 of 40 on get_physical_card. One dataset, one afternoon. Opus was not run with reasoning on.
pcparts001
Source checked:
[88] Pi extension that sets DeepSeek reasoning effort with Jev
The extension asks Jev whether the next Pi prompt needs none, low or high reasoning, then changes only that thinking level. The README’s example model is deepseek-v4.1-flash. On 30 tasks against always-high, the corrected per-task cost is $0.2844 versus $0.3352. Against always-low, routing costs $0.2845 versus $0.2112, with equal answer quality on that run.
Read at commit 2215f7c. The default route calls jev-latest on TypeSafe’s API, which the README equates with jev-1.13.0. The 30-task figures are from a pre-port benchmark on a direct DeepSeek route, and the README limits those quality and cost figures to the direct route. Spearman rho against difficulty is +0.504 (p = 0.0040, n = 30). The measured-results table lists decision latency of 409 to 856 ms. A missing key, a timeout, or a bad answer falls back to high. The README says none was never returned in that set, so the observed router was low versus high.
David Arias (@Davidariasfin)
Published:
Source checked:
[89] Jev ranks 10-K passages before a language model answers
David Arias describes 50 questions about four annual reports. Jev ranks retrieved passages and a language model reads the top two. With DeepSeek Flash, that pipeline scores 50 of 50 against 46 of 50 for an agent that reads whole filings. The post says jev-latest resolved to Jev 1.13, and that search put the right passage first 40% of the time against 90% after Jev ranked the same candidates.
The X post is the write-up. The linked article has the per-model tables. The 40% figure matches the article’s keyword-plus-vector row, not every search method on that page.
David Arias
Published:
Source checked:
[90] 50 questions on four 10-Ks: Jev RAG and an agent loop
The article scores 50 analyst-style questions on the Apple, Microsoft, Nvidia and Amazon 10-Ks. Jev RAG with DeepSeek V4 Flash, thinking on, scores 50 of 50. The agent scores 46 of 50. The same passages sent to Qwen 3.7 Flash and GPT-5 nano score 49 and 48 of 50, against agent scores of 41 and 34. Code looks up the latest filing before Jev runs. A hand reading changed 13 automatic grades.
Read 28 September 2026. The page dates the tests to 27 and 28 September 2026. Jev is called as jev-latest through OpenRouter. The X post says that alias resolved to Jev 1.13. DeepSeek Flash runs on DeepSeek’s API. Qwen 3.7 Flash is qwen/qwen3.7-flash and GPT-5 nano is openai/gpt-5-nano, both through OpenRouter at provider defaults. For DeepSeek, the agent uses 4.7 calls, 82,908 tokens, 11.1 seconds and $0.0031 cached or $0.0133 uncached per question. Jev RAG uses 1 call, 840 tokens, 2.0 seconds and $0.0007. Keyword plus vector search is first on 20 of 50. Jev Choice is first on 45 of 50, and two shuffles score 46 and 44. One Jev call over about 12,000 tokens had a median of 0.5 seconds and cost about $0.0005.
Zefan Cai (@Zefan_Cai)
Published:
Source checked:
[91] Open-Jev 27B v1.1 on the public JevBench tasks
Zefan Cai reports Open-Jev 27B v1.1 at 197 of 231 public JevBench tasks, and 80 of 111 on the hard tier. Jev is 200 of 231 and 81 of 111. The post says the 231 tasks are a public subset of 534.
Announcement post. The GPT-5.6 Luna and GPT-6 Astra rows, timings and scoring rules are in the repository note at commit 3308a15, not in this post.
Zefan Cai
Published:
Source checked:
[92] Public JevBench note at Open-Jev commit 3308a15
The note scores six streams on all 231 available public tasks. Jev 1.13.0 is correct on 200. GPT-5.6 Luna, reasoning effort none, is correct on 206. GPT-6 Astra, reasoning effort low, is correct on 231. Easy tasks are 48 of 48 for each of those three. On the original 72 the counts are 71, 69 and 72. On the hard 111 they are 81, 89 and 111.
Read at commit 3308a15. The other 303 of 534 tasks were unavailable, and the note reports no 534-task score. Open-Jev and Jev return native probabilities. GPT returns verbalized probabilities in JSON. Candidate order differs on 119 of 139 Choice tasks. One Jev vector was renormalized. P50 full-response times, one observation per task, are 291.3 ms for Jev, 953.8 ms for Luna and 2,206.4 ms for Astra. The note says that sample is not a repeated latency test. Tariff estimates are $0.03842, $0.17914 and $8.81978 per 1,000 decisions, and the note says they are not billing receipts. Brier means are 0.1811, 0.2074 and 0.0085. Top-label ECE is 0.0318, 0.0932 and 0.0149. Open-Jev 27B v1.1 is 197 of 231, and its direct-execution timings are left out of the latency table. Upstream tasks are pinned at fstandhartinger/jevbench commit f8ce713.
wang yiming (@wangyiming34110)
Published:
Source checked:
[93] A five-model snapshot of 231 planning tasks
The post points to chapter 6 of the Jev Cookbook and says that, on one snapshot of 231 planning tasks, Jev produced 120 valid answers at a p50 of 0.77 seconds, while Kimi produced 139 valid answers, the most of the five models.
The post gives valid-answer counts and one latency figure. Stop reasons, correct-answer counts and version strings are in the cookbook chapter at commit 546050a.
Datawhale
Published:
Source checked:
[94] Jev Cookbook chapter 6 snapshot for 26 September 2026
The chapter records one run of five aliases on 231 planned public tasks. Kimi K3 has 139 valid answers and 139 correct. Jev has 120 valid and 116 correct, then leaves 110 tasks unattempted after a reservation stop. DeepSeek Flash has 110 valid and 110 correct, then stops after consecutive errors. Conditional accuracy uses only scorable answers.
Read at commit 546050a. snapshot.json in the same commit matches the counts. Recorded spend is $0.001986 for Jev, $0.011435 for Kimi K3 and $0.020312 for DeepSeek Flash. p50 latency is 0.77, 9.63 and 1.40 seconds. The aliases do not include a full version string. Hard-tier valid answers are 1, 29 and 2 of 111. The stored composite is highest for Jev, and the chapter says that score describes this incomplete run. Raw prompts and provider responses are not in the repository. This run is separate from the completed 231-task audit in the Open-Jev note.
Engin Diri (@_ediri)
Published:
Source checked:
[95] A Claude Code proxy that asks Jev per message
Engin Diri says he built a Jev router for Claude, with thresholds on the prompt and a dashboard for the decisions. The thread links the repository and a Pulumi blog post.
Announcement post. Model names, the ratchet, and the accept bars are in the repository at commit f909310 and in the blog, not in this post.
Engin Diri
Published:
Source checked:
[96] Route every Claude Code message with Jev
The post describes a local proxy that asks Jev on each new Claude Code message. Mechanical work is aimed at Haiku 4.5, routine work at Sonnet 5, and complex or deep work at Opus 5.5, with an optional Fable 5.1 tier. A session’s tier can rise and does not fall. The author estimates a routing call at about $0.00003.
Read on 28 September 2026. The live-view caption in the post is a picture of the UI. The repository at commit f909310 is the source for the shipped bars, the model pin jev-1.13.0, and which checks were still open.
Engin Diri
Source checked:
[97] jev-router README and config at commit f909310
The proxy asks Jev once per new human message and, by default, only moves a session’s tier up. The Claude-only config maps the four Choice labels onto Haiku 4.5, Sonnet 5 and Opus 5.5. Codex’s frontier tier is gpt-6-astra and its trusted tier is gpt-6-sol. The package name is @ediri/jev-router.
Checked at commit f909310, release 1.6.0. config/anthropic-only.json at that commit sets policy.accept to 0.85, 0.6 and 0.3, sensitiveOverride to 0.7, claimGuard to 0.5, and idleResetMinutes to 10. The TypeSafe channel pins jev-1.13.0. src/jev.mjs sends one Choice plus two Noul guards. The README says the 58-prompt set has not been used to tune the bars, full Codex sessions were not in the live checks, and the offline suite has 176 tests against mocks. A 24 September 2026 check with Jev mocked used Claude Code 2.1.281. The unscoped npm name jev-router is a different project.
Christoph Görn (@goern)
Published:
Source checked:
[98] A Claude Code router timed against local Kev
Christoph Görn says a local Kev-4B router on an RTX 4090 was 10 to 15 times faster than the cloud sizer he called Jev, 100 to 180 ms against 700 to 1400 ms, and that Kev’s accuracy was 14%. The post links his write-up.
The 14% figure is Kev’s score in the write-up, on 8 prompts. The cloud side of that table is OpenRouter’s typesafe/jev-router alias, and the write-up says the log named other models. The 16-prompt latency average is a different sample.
Christoph Görn
Published:
Source checked:
[99] What answered the Claude Code sizing hook
The hook calls OpenRouter’s typesafe/jev-router and asks for a JSON size label. On the first 16 prompts the log counts 3 tiny, 1 everyday, 1 large, 4 left on the main session, and 7 errors. The answered_by field names gpt-6-luna five times, deepseek-v4.1-flash three times, and kimi-k3 once.
Read on 28 September 2026. The 9 successful calls average 2.99 seconds, from 1.26 to 4.41, and sizing cost about $0.0032. A separate 8-prompt table scores Kev-4B at 1 of 7 and the alias at 1 of 5, with latencies of 100 to 180 ms and 700 to 1400 ms. The author says the router code is in a private infra repository and that the router log has no successful Kev decision yet.
Kishore Mandyam (@nmandyam)
Published:
Source checked:
[100] EosHQ timings for agent selection and automation detection
Kishore Mandyam says EosHQ uses Jev for fast decisions. In synthetic tests, agent selection took 52% less time and automation detection 73% less, including Gemini fallbacks. The post links the article for the numbers and the exceptions.
The percentages match the article’s agent-selection and automation rows. The article is the source for sample size, the Gemini version, and the cases that were slower.
EosHQ
Published:
Source checked:
[101] EosHQ decision timings against Gemini 3 Flash Preview
The article reports a Jev path, including Gemini 3 Flash Preview fallbacks, at 1,616 ms for agent selection and 925 ms for automation detection, against 3,380 ms and 3,432 ms. Label selection is one observed request, 3.07 seconds to 0.455 seconds. The article does not name a Jev version.
Measurements dated 28 September 2026. Agent selection and automation detection are 18 Jev-enabled decisions each, from six scenarios repeated three times. Fallbacks were 6 of 18 and 3 of 18. Jev-only averages are 365 ms and 350 ms. One invoice-trigger example needed Gemini on all three repetitions and was slightly slower. Costs per 1,000 are estimates from token counts and list prices: Gemini 3 Flash Preview at $0.50 per million input tokens and $3 per million output tokens, including thinking, and Jev at $0.042 per million input tokens with output free. The article says the samples are not a production latency guarantee.
Yubo Li, Yidi Miao, Ramayya Krishnan and Rema Padman
Published:
Source checked:
[102] Jev 1.13.0 as a judge next to GPT-6, Claude and Gemini
The paper scores jev-1.13.0 against hosted GPT, Claude and Gemini judges on the same Choice contract. On a 400-pair RewardBench sample Jev scores 92.2% and GPT-6 Astra 93.5%. Claude Sonnet 5’s primary scores are 91.2%, 88.9% and 85.8%, and four contract failures stay in that denominator. Gemini 3 Flash scores 92.8%, 76.0% and 83.3%. Gemini 3.1 Pro scores 94.5%, 87.4% and 87.1%.
arXiv:2609.26550v1, dated 22 September 2026. Jev is jev-1.13.0. GPT-6 Astra is gpt-6-astra at low effort with a JSON schema. Claude Sonnet 5 is claude-sonnet-5 and Gemini 3 Flash and 3.1 Pro are gemini-3-flash-preview and gemini-3.1-pro-preview, also at low effort. RewardBench here is 100 pairs from each of four categories, not the official subset weights. JudgeBench is the GPT-4o split. HaluEval is 240 evidence-grounded judgments. Existing-label accuracy is a fourth column on 150 replies: 94.0% for Jev, 94.7% for Sonnet 5, 95.3% for Flash, 94.7% for Pro and 96.7% for Astra. Invalid outputs count as errors in the primary accuracy. Sonnet 5 is valid on 1,308 of 1,312 items, all four failures HTTP 200 contract breaks. Table 6’s valid-only Sonnet 5 rates are 91.7%, 89.4% and 85.8%. Flash and Pro are valid on all 1,312. The isolated latency panel is 120 judgments, and the printed medians name Jev, GPT-4.1 mini and GPT-6. Fees use reported usage at collection-time prices and are not invoices. The single-order cascades read the threshold off the same items. The two-order policies were frozen earlier, on 510 extension pairs: Astra at 0.9 scores 92.5% against 93.1% at 56.8% of the fee; Sonnet 5 at 0.6 scores 88.0% against 89.2%; Flash at 0.6 scores 87.1% against 82.9%; Pro at 0.9 scores 89.6% against 90.4%. A human adjudicated disagreements with GPT-6. The second pass was an LLM, and an author resolved the 39 items where the two passes differed.
Assaf Elovic (@assaf_elovic)
Published:
Source checked:
[103] GPT Researcher replaces its embedding filter with Jev
Assaf Elovic says GPT Researcher replaced its embedding filter with Jev on 28 research tasks, that 73% of kept chunks were relevant against 46%, and that blind comparisons preferred the Jev reports 15 to 3 at the same cost. He says the project now runs on Jev by default.
Announcement post. The 15-to-3 figure omits the ties in the repository table. The task split, the writer model, the keyword fallback and the Jev alias are in the eval note at commit 2906e632.
Assaf Elovic
Published:
Source checked:
[104] GPT Researcher context-filter note at commit 2906e632
The note replays 28 saved research tasks, 20 from SimpleQA and 8 open-ended, and writes every report with gpt-5.4. Jev keeps 1,000-character chunks scoring at least 1.5 on a four-level usefulness rubric. Context precision is 0.73 against 0.46 for embeddings, and the blind comparison is 15 wins, 10 ties and 3 losses. Cost per report is $0.115 and $0.117.
Read at commit 2906e632. The precision judge is gpt-5.4-mini. A pairwise win counts only when gpt-5.4 prefers the same side in both orders. Jev’s filter time is 2.4 seconds at the median with 32 calls in flight, and 1.7 seconds at the default of 64. The client calls jev-latest unless JEV_MODEL is set, and the note does not record which version answered. select.py at the same commit uses Jev when a TypeSafe key is present and a BM25 keyword fallback otherwise. That keyword row is 19 of 20 on SimpleQA, precision 0.51, and 14 wins, 8 ties and 6 losses against embeddings, at $0.116. Its sign test is about p = 0.12. SimpleQA is 19, 18 and 20 of 20 for Jev, embeddings and no filter. The note says a one-question difference is noise. On the 8 open-ended questions, no filter wins 6 of the decided comparisons with Jev. Skipping the filter costs $0.192 per report.
Tony Gentilcore (@tonygentilcore)
Published:
Source checked:
[105] Four Glean checks, including a citation judge against GPT-5.6 Luna
Tony Gentilcore’s article names Aviral Singh, Manav Agrawal, and the accounts @eddiedzhou, @mr_cheu and @MatZhao. On one 1,448-word answer, Jev’s citation-coverage verdict changed on 1 of 28 paragraphs across three runs. GPT-5.6 Luna with no reasoning changed 7, and Luna at xhigh changed 5.
X article posted 28 September 2026. The citation timing is 6.6 seconds and $0.014 for Jev, against 81.3 to 85.5 seconds and $0.030 to $0.047 for Luna with no reasoning, and 227.4 to 259.7 seconds and $0.047 to $0.060 at xhigh. Jev’s time is sequential client wall time. The Luna times sum model-call durations. The article says the two systems used different support criteria and that there were no independent human labels. Query classification reports agreement with a production LLM, not a human label: 66.8% for Jev, 35.9% for base Laya and 74.5% for a fine-tuned Laya. The request count for that table is not in the text. A 751-entry routing replay uses three experts. The article says Jev was more accurate than the existing LLM router. The chart percentages are not transcribed here. The median speedup on 40 transfer calls is 8.1 times. Choice reranking on 4,855 queries gave identical scores to about 37 of 41 candidates. Using production order to break ties raised Recall@6 from 40.2% to 44.3%.
Daniel Liao (@Daniel_Bernath_)
Published:
Source checked:
[106] Jev judges aircraft pairs over the San Francisco Bay
Daniel Liao says a Jev judge watched about 900 live flights from an E2B sandbox, flagged risky pairs in 103 ms, and missed none. He says gpt-4o missed 56 in the same sky and Claude Opus 5.5 took 12 seconds to miss none.
The post links 7datumba/jev-tower. The 103 ms, 56 misses and 12 second figures match files in that repository. The false-positive counts and the fetch error are in those files, not in this post. The post’s 900 flights are not the 60-tick window used for the ladder.
Daniel Liao
Published:
Source checked:
[107] jev-tower logs at commit 20dcbbb
tower.mjs replays one 60-tick ADS-B window. A pair is a hazard when the closest approach is inside 3 nautical miles and 1,000 feet within 120 seconds. Jev, called as jev-latest, and the chat model both flag a pair when their probability is above 0.5. Beside gpt-4o, Jev averages 99 ms with 0 misses and 335 false positives. gpt-4o averages 2,356 ms with 56 misses and 32 false positives.
Checked at commit 20dcbbb. data/tower-gpt4o.json, data/tower-gpt4o-mini.json and data/tower-claude-opus-5-5.json are the three logs. Jev’s counts stay at 385 true positives, 335 false positives and 0 misses. Average latency on the three runs is 99, 103 and 119 ms. gpt-4o-mini averages 2,242 ms, with 160 misses and 22 false positives. claude-opus-5-5 averages 12,318 ms, with 0 misses and 174 false positives on the pairs it scored, and 378 true positives against Jev’s 385. The Opus log prints one fetch failure. The README table lists Jev at 103 ms and the OpenAI false positives as 1 and 0, which these files do not show. The resolved Jev version is not in the logs. The ground truth is the script’s geometry rule.
OpenAI Developers (@OpenAIDevs)
Published:
Source checked:
[108] Decisions API preview, powered by GPT-6 Luna
OpenAI’s post says the Decisions API, powered by GPT-6 Luna, takes a question and a fixed set of answers. The caller uses the returned selection to classify content, route a request, or choose an agent’s next action. The post says the API is in limited preview.
Announcement post on 29 September 2026. It does not state a price, a latency, or a comparison with Jev. Image input and the preview’s release plan are in the follow-up post.
OpenAI Developers (@OpenAIDevs)
Published:
Source checked:
[109] Decisions API accepts text or images in the preview
The follow-up says a Decisions API call can send text or images as context, for example a support request and the teams that could take it. The API returns a selection. Preview access is limited to selected API customers, and the post says a broader release is planned in the coming days.
Second post in the announcement thread. It links to OpenAI’s DevDay 2026 recap and does not give a price or a measured latency. The model name GPT-6 Luna is in the first post.
Dan Shipper
Published:
Source checked:
[110] Every’s early Decisions API checks against Jev
Dan Shipper reports two early checks of OpenAI’s Decisions API against Jev. On a text-only replay of computer tasks, Jack Cheng found 76 of 78 correct controls for Decisions and 73 of 78 for Jev, at typical times of 230 and 500 milliseconds. On conversation classification, Kieran Klaassen’s test was tied on accuracy, with medians of 161 and 309 milliseconds.
Every, 29 September 2026. The article says Decisions accepts images as well as text, and that this computer replay was text only. It says Decisions trailed Jev on Every’s broader evals and does not publish those scores, the Jev version, the thread count, or the task list. Pricing is unannounced. Shipper expects a launch in a few weeks. The post that names GPT-6 Luna is OpenAI’s, not this article.
Entendre (@EntendreAI)
Published:
Source checked:
[111] Jev picks a ledger account from two, three, or five options
Entendre tested Jev on Crypto Accounting Bench by asking it to choose among 2, 3, or 5 real accounts. Reported accuracy is 51.7%, 47.5%, and 48.3%. Successful calls averaged 635 ms. On the five-account list, confidence of at least 0.75 raised accuracy to 61.5% and covered 39 of 118 tasks.
First post in the 28 September 2026 thread. The post says the highest percentages in the confidence charts come from the smallest samples. It does not name a Jev version or say how many calls failed. The shortlist rules and the language-model comparison are in the next two posts.
Entendre (@EntendreAI)
Published:
Source checked:
[112] Every Crypto Accounting Bench shortlist contained the right account
Entendre says each option was a real ledger account, and the incorrect choices were plausible alternatives for that transaction. Every shortlist contained the correct account. The score is agreement with the journal entry the company recorded.
Second post in the thread. It points to the Crypto Accounting Bench write-up, the research paper, and a public dataset. The paper and dataset describe full journal entries. They do not contain this Jev shortlist run.
Entendre (@EntendreAI)
Published:
Source checked:
[113] Shortlist accuracy set beside full journal-entry rates
With five candidates, Entendre reports 48.3% for Jev, 46.3% for GPT-5.6 Luna, and 46.9% for DeepSeek V4 Flash. With two candidates, Jev’s 51.7% matches the recorded Claude Opus 5 rate in the post, above GPT-5.6 Sol at 51.4% and Claude Sonnet 5 at 49.4%.
Third post in the thread. Entendre says the language-model figures are account correctness when those models wrote full journal entries from the complete chart, while Jev received a shortlist that already contained the correct account. The paper’s Pass@3 table is a different metric.
Kareem Khattab, Omar Khattab and Mohamed Ibrahem
Published:
Source checked:
[114] Crypto Accounting Bench paper, arXiv:2609.14811
The paper evaluates full journal-entry reconstruction on 118 tasks from 7 organizations. Pass@3, a perfect rubric and every required gate on one of three attempts, is 47.46% for GPT-5.6 Luna, 50.00% for GPT-5.6 Sol, 45.76% for Claude Opus 5, 47.46% for Claude Sonnet 5, and 46.61% for deepseek-v4-flash-0731.
arXiv:2609.14811v1, submitted 13 September 2026. Mean Score is the paper’s primary metric. Pass@3 is the third column of Table 2 and matches Figure 3. The median chart is 569 accounts. The paper does not evaluate Jev. Entendre’s later shortlist percentages are not these Pass@3 figures.
Entendre Finance
Source checked:
[115] Public Crypto Accounting Bench dataset, 118 full-entry tasks
The dataset card describes 118 tasks whose target is the complete journal entry an organization posted, chosen from that organization’s chart of accounts. It repeats the paper’s best Mean Score of 77.43% and best Pass@3 of 56.78%. It does not include Entendre’s later Jev shortlist run.
Read 30 September 2026. The card says the public files are a transformed derivative and that the judge pipeline behind the paper’s headline numbers is not released. The deterministic scorer in the repository is described as a lower bound. No dataset revision hash is cited here.
Lucas Biagetti (@LucasBiagettiA)
Published:
Source checked:
[116] Banking77 text-classification benchmark updated with Jev
Lucas Biagetti says he updated his text-classification benchmark, which he now frames as a decision-model comparison, to include Jev and to fill some gaps. He writes that Jev wins zero-shot classification in his exploratory analysis.
Post on 1 October 2026. It does not give the table, the class counts, or the OpenAI model identifier. The reply in the thread attaches a chart. The numbers and the model pins are in the repository brief and manifest at commit efbfb42.
Lucas Biagetti
Published:
Source checked:
[117] Banking77 v1.2 brief, 56 quality conditions
The v1.2 brief reports accuracy for 14 method and budget variants at 5, 10, 15, and 20 Banking77 classes. Zero-shot Jev scores 94.00%, 92.25%, 89.83%, and 86.12%. The OpenAI zero-shot row scores 92.00%, 88.25%, 87.33%, and 85.00%.
Checked at commit efbfb42. The brief says each width uses 40 held-out examples per class, one seed, and that the cohorts overlap, so the widths are not independent samples. It reports 1, 2, and 3 missing Jev probability vectors at 10, 15, and 20 classes. It makes no cross-provider cost or latency claim. MiniLM plus logistic regression at 100 examples per class scores 98.00%, 96.50%, 95.83%, and 92.38%. The model identifiers are in the manifest, not in this brief.
Lucas Biagetti
Published:
Source checked:
[118] Banking77 v1.2 manifest: jev-1.13.0 and gpt-5-nano-2025-08-07
The manifest pins Jev to jev-1.13.0 and the OpenAI zero-shot row to gpt-5-nano-2025-08-07 with reasoning effort minimal. Both use zero labeled examples. The campaign seed is 42, the class counts are 5, 10, 15, and 20, and the test set has 40 examples per class.
Same commit as the v1.2 brief. The MiniLM row uses sentence-transformers/all-MiniLM-L6-v2 at revision 1110a243, frozen, with logistic regression. BERT is google-bert/bert-base-uncased for two epochs. The manifest does not store latency or a billed cost for the zero-shot rows.
Ilia Tankelevich (@ilia_tankelevic)
Published:
Source checked:
[119] maslul 0.5.0 lets Jev pick the tier
Ilia Tankelevich says maslul 0.5.0 uses TypeSafe’s Jev model to pick the tier in his open-source Python LLM router. He reports about 0.3 seconds per decision and points to pip install "maslul[jev]".
Post on 1 October 2026. It does not name the Claude, Gemini, Grok, or GPT model that receives the tier, and it does not include a log of the 0.3 second figure. The hook’s defaults and the example Claude models are in the repository at commit de093d3. The follow-up post is about the level descriptions.
Ilia Tankelevich (@ilia_tankelevic)
Published:
Source checked:
[120] Level descriptions changed the Hebrew tier check
The follow-up says a check on real Hebrew conversations found that the wording of the level descriptions mattered more than the language. Abstract descriptions never let hard win, in English or Hebrew. Concrete examples put hard requests at 0.97 to 0.99 confidence, and those descriptions are now the defaults.
Reply on 1 October 2026 in the maslul thread. It does not list the requests or the confidence of the abstract run. Those details are in a comment in jev.py at commit de093d3. The post does not name a Jev version.
Ilia Tankelevich
Published:
Source checked:
[121] maslul Jev classifier and verifier at commit de093d3
jev.py adds an optional Jev classifier and verifier. The classifier asks one Choice question for simple, medium, or hard, calls jev-latest, and returns no tier below confidence 0.7 or when the call fails. The verifier asks a Noul and, on failure, accepts the reply by default.
Commit de093d3, 1 October 2026, the 0.5.0 hook. The HTTP timeout in this file is 5 seconds. The state is the last 6 text messages, cut to 20,000 characters, and media is dropped. A comment dated 2026-10-01 says abstract descriptions returned medium at 0.48-0.78 for a moving plan, a savings split, and an insurance comparison, in English and Hebrew, and the concrete defaults returned hard at 0.97-0.99 for the same requests. The file does not record a 0.3 second measurement. The example model names are in examples/maslul.toml.
Ilia Tankelevich
Published:
Source checked:
[122] maslul example tiers: Claude Haiku 4.5 and Sonnet 4.6
The example config at commit de093d3 sets simple and medium to anthropic:claude-haiku-4-5 and hard to anthropic:claude-sonnet-4-6. The default strategy is route_default and the default level is hard. The file configures only the Anthropic provider.
Same commit as the Jev hook. The comment says the file is a minimal example so the quickstart runs with an Anthropic key. It does not enable the Jev extra, and it does not name a Gemini, Grok, or OpenAI model. The README of the package documents those providers separately.
Tobias Deußer, Lorenz Sparrenberg and Rafet Sifa
Published:
Source checked:
[123] 37-dataset Jev evaluation, arXiv:2609.37647
The paper evaluates jev-1.13.0 zero-shot on 37 datasets, 346,009 requests. Against Qwen3.8-27B and Gemma-4-E4B scored on the same requests, Jev is ahead on 27 datasets, tied on one, and behind on 9, and ahead of Gemma on all 37. The evaluation split cost US$9.15.
arXiv:2609.37647v1, submitted 29 September 2026. Banking77 accuracy is 79.7%, 77.4%, and 67.0% on 3,076 examples. MMLU is 91.8%, 81.7%, and 67.4% on 14,042. Eleven of Jev’s leads over Qwen have non-overlapping bootstrap intervals, and none of Qwen’s nine leads do. Mean latency is 0.36 seconds at 32 concurrent requests. The whole study, including the pilot and extra analyses, is US$9.40. The reference models are not GPT, Claude, Gemini, Grok, Kimi, or DeepSeek. The harness and response archive are in the repository.
Tobias Deußer
Published:
Source checked:
[124] jev-benchmarking harness at commit 6bbdeb3
The README pins the evaluation to jev-1.13.0 and compares it with Qwen3.8-27B and Gemma-4-E4B on identical requests. It links the paper and the raw responses at doi.org/10.5281/zenodo.23039006. Offline scoring rebuilds each request and looks up the cached response.
Commit 6bbdeb3, 30 September 2026. The README says a dataset that changes on Hugging Face would no longer match the cache. LLM-AggreFact is gated. The open-model path needs a GPU. The response license allows research and evaluation use of the Jev responses and bars distillation and training a model to imitate Jev.
Koray Kavukcuoglu
Published:
Source checked:
[125] Gemini 4 Argon announcement
Google announces Gemini 4 Argon on 30 September 2026 and says it is rolling out to trusted cyber defenders through the Fairwind Program. The introductory price is $2 per million input tokens and $10 per million output tokens, cached input is 95% off the input price, and the later price is $4 and $20. The output limit is 1 million tokens, up from 64,000. Reported scores are 77.9% on DeepSWE v1.1, 51.3% on AutomationBench, 91.7% on LVBench, and 68% tied for first on CWE-bench v1 remediation.
The announcement does not mention Jev, an API model id, or a date for paid API or Google AI Ultra access. It says Argon leads the Vals Index and does not print that score. The DeepSWE, AutomationBench, LVBench, and CWE-bench figures are Google’s. The libgav1 note says agents replaced 32,000 lines of SIMD and the decoder ran 2.7 times faster than the earlier Rust port, with the same video output. A healthcare vulnerability found with Wiz is described without a package name or CVE. Jev is listed here because the note is cited from the Jev and Gemini report.
Zorepad (@zorepad)
Published:
Source checked:
[126] One terminal session with Opus 5.5, Jev and Gemini 4 Argon
The author describes one terminal session: Opus 5.5 on high effort writes the code, Jev scores file, tool, retry, and stop decisions, and Gemini 4 Argon is called three times and writes no lines. The reported /search p99 moves from 2.4 seconds to 180 milliseconds, with 18 of 18 tests passing and a pull request in 26 seconds for $0.47.
One post and a 30-second video on 1 October 2026. The post gives no repository, log, invoice, or model identifier. It says Jev forked the task 2,395 times, the last 28 decisions were 15 running in code and 13 sent back to Opus, and the edit was 4 lines after 581,000 tokens. It does not say whether the Argon calls were inside the Fairwind Program.
Arena.ai (@arena)
Published:
Source checked:
[127] Agent Arena thread on Jev Router
Arena evaluates a product it calls Jev Router by @typesafeai on more than 4,700 Agent Arena sessions. It says the router does not improve on the Pareto frontier: similar performance to DeepSeek V4.1 Flash (Max), 38% more cost, and 1.7 times the median model-request latency. A later post in the thread gives end-to-end medians of 6.18 seconds and 3.64 seconds.
Thread on 7 October 2026. The opening post names DeepSeek V4.1 Flash as the most picked model, with GPT-6.1 Sol and GPT-6 Luna also frequent, and steerability of +10% against +10.48 for Claude Opus 5.5 (High). The next posts give call shares of 28.1% DeepSeek V4.1 Flash, 23.5% GPT-6 Astra, 22.8% Sol and 11.1% Luna, a hypothetical leaderboard row of +3.6% net improvement at $0.155 per task, and P90 latency of 30.59 seconds against 14.26. The thread prints no API model id, no code, and no session export. It does not say the calls used OpenRouter’s typesafe/jev-router. Theo’s DeepSWE note and Christoph Görn’s 16-prompt log stay separate.
Josh Longenecker (@jldevtech)
Published:
Source checked:
[128] Jev in front of Claude Sonnet 4.6 on tool-call checks
The post says a cascade put TypeSafe’s Jev in front of Claude Sonnet 4.6, so Sonnet sees only the cases Jev is unsure about, and reports 51% faster, 61% cheaper, and the same accuracy.
Post on 8 October 2026 linking the Grounded AI write-up. The shorter wording says the accuracy was the same. The write-up and bench.json give 0.695, 0.685 and 0.675, and say the labels are Kimi K2-thinking’s next action.
Josh Longenecker
Published:
Source checked:
[129] CascadeEvaluator on 200 tool-or-reply decisions
The write-up measures CascadeEvaluator on 200 conversations from GeneralFunctionCall-Test. At a confidence of 0.7, 73 of 200 decisions go to Claude Sonnet 4.6. Mean time is 1.28 seconds against 2.59 for Sonnet alone, and the listed cost per million decisions is $2,997 against $7,751. Agreement with the reference is 0.695 against 0.675.
The page has no separate byline. The X post links it, and commit 523194a on grounded-ai/grounded_ai is by jlonge4 on 8 October 2026. The judge string is bedrock/us.anthropic.claude-sonnet-4-6. Jev is called as jev/jev-latest. Labels record what Kimi K2-thinking did next. List prices in the caveats are Jev at $0.042 per million input tokens, output free, and Sonnet 4.6 at $3 and $15. At 0.9, 112 of 200 decisions go to Sonnet, mean time is 1.84 seconds, cost is $4,572 per million, and agreement is 0.685. The write-up calls 0.9 the default threshold.
Josh Longenecker
Published:
Source checked:
[130] tool-call bench.json at commit 523194a
The JSON stores the 200-item run: Sonnet alone at 0.675 agreement, 2.591 seconds mean, 2.355 seconds median and $7,750.875 per million decisions. The 0.7 cascade stores sent_to_sonnet 0.365, agreement 0.695, mean 1.276 seconds and $2,996.74. The 0.9 cascade stores 0.56, 0.685, 1.838 seconds and $4,571.86.
Commit 523194a, 8 October 2026, message "Tool-call benchmark: CascadeEvaluator vs Sonnet 4.6 alone". The file names jev as jev/jev-latest and the judge as bedrock/us.anthropic.claude-sonnet-4-6. The write-up rounds these fields to 2.59 seconds, 2.35 seconds, $7,751, 1.28 seconds, $2,997, 1.84 seconds and $4,572. sent_to_sonnet of 0.365 and 0.56 is 73 and 112 of 200.
David Chiu
Published:
Source checked:
[131] 100-turn routing bench at commit ed4a0ab
The README scores a 100-turn enterprise-agent routing bench. All four decisions are correct on 77.0% of turns for a self-hosted Gemma 4 31B JSON call and 61.4% for hosted Jev 1.13.0. Route is 90.0% and 90.6%. Restriction is 95.0% and 84.4%. Jev’s p50 is 749 ms and its p95 is 818 ms.
Commit ed4a0ab, 4 October 2026, on github.com/ywchiu/jev_benchmark. The account name on that commit is ywchiu and the author name is David Chiu. The bench is 20 conversations of 5 turns and does not retrieve documents or run tools. The original five systems were run 5 times. Of the October rows, Clef-flash was run 5 times, Clef twice, and each Cygnet row once. Clef and Clef-flash repeated the same answers. The 31B server also served live traffic, so that latency is only indicative. Clef was the open-weight systemone() path. Cloudflare’s hosted API quota ran out partway through. Cygnet reads gemma-4-12B-it one token per question. The same readout on the 31B checkpoint scores 69.0%. Ask-first accuracy is 43% for Gemma and Clef, 31% for Jev, and 0% for Cygnet on 12B. Laya scores 0.0% and SemIf 24.0% on the all-four metric. This Gemma 4 31B row is not Gemma-4-E4B in arXiv:2609.37647, and it is not a Gemini API model.
Vals AI (@ValsAI)
Published:
Source checked:
[132] Vals thread on a 400-item Jev claim check
The opening post says Vals tested TypeSafe’s Jev against 11 other models and matched GPT-6 Astra’s 97.5% accuracy on 400 claim-verification questions at about 1/500th the cost. A follow-up gives about $0.025 per 1,000 cases against $12.29 for Astra, and 194 times lower median latency at 32 judgments about one document.
Thread on 6 October 2026 linking the write-up. A later post in the thread says Jev ranked last on a 12-task LegalBench slice, and that a 1% error-budget threshold automated 95% of held-out claim-verification cases at 1.6% error. The write-up’s probability-priced table lists Jev at $0.02 per 1,000 cases and Astra at $18.00. The $12.29 figure is the thread’s Astra cost without that extra probability charge.
Connor Frank
Published:
Source checked:
[133] Independent evaluation of Jev on claim verification and LegalBench
Vals scores twelve systems on 400 claim-verification items built from 10-Ks and on a 396-question LegalBench slice. Jev 1.13.0 scores 0.975 on the claims, 390 of 400, tying GPT-6 Astra, GPT-6.1 Sol and GPT-6 Luna at $0.02 per 1,000 cases. On LegalBench it ranks last, with class-balanced accuracy 0.730 against 0.732 for Claude Haiku 4.5 and 0.932 for GPT-5.6 Terra.
Write-up dated 6 October 2026. Gemini 3.8 Flash wrote 160 paraphrases and contradictions; the labels come from the construction rule, and a person audited every item. LegalBench is 12 subtasks with 33 questions each, sampled from 1,808 available items, and the page says that slice is not representative of the full benchmark. At 32 judgments about one document, Jev’s median latency is 194 times lower than GPT-6 Astra’s and 241 times lower than Gemini 4 Argon’s. A 1% error budget, fit on the first 30% of claim items, automates 95% of the held-out 70% at 1.6% error. Native structured output, no JSON prompting. Intervals bootstrap whole filings and Bonferroni-correct eleven comparisons against Jev.
Steven Denney and Matthew DiGiuseppe
Published:
Source checked:
[134] Seven political-science replications, arXiv:2610.06625
The paper re-runs seven published political-science annotation and scaling tasks with Jev 1.13 against GPT-6 Luna and Qwen3.8-27B. Jev matches or comes close on the human-coded scores. Luna at OpenAI batch prices costs less on six of the seven primary tasks. One-at-a-time medians are 0.09 to 0.27 seconds for Jev and 0.75 to 1.04 seconds for Luna.
arXiv:2610.06625v2, 6 October 2026, 71 pages. v2 corrects author order in the metadata. Jev ran through OpenRouter except on the V-Dem recall task, which used TypeSafe’s endpoint. Qwen ran through OpenRouter on Parasail at FP8. Speed rows are 200 timed items per application, sent one at a time from the Netherlands on 29 and 30 September 2026. On Gilardi’s 2021 relevance tweets Jev scores 89.0% against Luna’s 91.1%; on the 2023 sample, 76.2% against 84.2%. Mean V-Dem correlation is 0.68 for Jev and 0.63 for Luna. On party-name comparisons, 74% of Luna’s probabilities sit below 0.01 or above 0.99, against 18% for Jev. The comments list no public code archive.
Premsanth Rajamani
Published:
Source checked:
[135] Jev against Gemini 3.8 Flash on 203 sales threads
Salesgear scores jev-1.13.0 and Gemini 3.8 Flash on 203 hand-written sales threads. Category accuracy is 193 of 203 for Jev and 199 of 203 for Gemini. Both find 60 of 61 real deferrals and put the same wrong thread in their top ten. Jev’s median is 0.35 seconds at $0.45 per 10,000 threads; Gemini’s is 2.2 seconds at $12.34.
Page dated 30 September 2026. The threads are synthetic, written one by one for a fictional seller, with labels fixed before either model saw them. Claude helped draft, so Gemini is the baseline. Gemini ran in JSON classification mode with low thinking. Warmth and date arithmetic run in code. The page links github.com/developer943/pipeline-revival-jev; the public repository is developer943/sales-pipeline-revival-jev.
Salesgear
Published:
Source checked:
[136] sales-pipeline-revival-jev at commit 926e72e
Commit 926e72e stores the 203-thread dataset, the six typed questions, the Gemini baseline, and results.json. That file records Jev accuracy 0.9507, Gemini 0.9803, medians 0.346 seconds and 2.188 seconds, and run costs $0.00906 and $0.25055. Jev ran with 8 workers; Gemini ran with 2.
Commit message dated 30 September 2026, “Jev vs Gemini on 203 dead sales deals: app, dataset, results”. results.json was generated 2026-09-30 16:36. Jev is jev-1.13.0. Gemini is gemini-3.8-flash. Deferral recall is 0.9836 for both. Come-back window accuracy on real deferrals is 0.9180 for Jev and 0.7705 for Gemini. ECE is 0.0199 and 0.0230. The README’s 8-minute and 48-minute 10,000-thread estimates assume eight workers on both sides; the Gemini wall time in the JSON uses two workers.
TypeSafe AI
Published:
Source checked:
[137] Jack & Jill candidate screen, TypeSafe case study
TypeSafe writes that Jack & Jill replaced Gemini 3.1 Flash Lite on the initial scoring stage of a candidate-matching pipeline. An offline test on 150 live roles kept 94.6% of hiring-manager winners on Jev’s shortlist against 93.9% for Gemini. Cost per 1,000 candidates is $0.092 against $0.755. Median time to screen all candidates for a role is 10.3 seconds against 20.3.
Case study dated 12 October 2026. The quality test used every past winner plus a sample of the remaining candidates, weighted to represent the original 6,000 retrieved for each role. The 0.7 point gap has a 90% confidence interval of −0.4 to +1.8, which TypeSafe calls not statistically significant. AUC is 0.933 against 0.924. List prices in the table are Gemini $0.25 input and $1.50 output per million tokens, and Jev $0.042 input with output free. $265,000 is current-volume savings; $500,000 is a projection that includes expected growth. Extra workflows, including a 3.6 second to 0.2 second profile check, sit outside those figures. The page names no Jev version and links no public log.
Artificial Analysis (@ArtificialAnlys)
Published:
Source checked:
[138] Artificial Analysis Decision Index launch thread
The opening post says Jev leads among decision models on Decision Index v1.0, and that language models such as Claude Haiku 5.5 still score higher at much higher cost and latency. The key-results post gives Jev 1.13 a score of 58, ahead of GLiDE and Microsoft-Decision-1 at 54, with Haiku 5.5 (Low) at 61, 1.4 seconds per task and $184 per million tasks against 0.13 seconds and $42 for Jev.
Thread on 13 October 2026 linking the launch article, methodology, and live leaderboard. Follow-up posts repeat the Pareto cost figures: Decider v1.1 27B at $17 per million tasks, Microsoft-Decision-1 at $31, and Jev 1.13 at $42. Celeris-1 Decision is 0.06 seconds per task. The thread says decision models are poorly calibrated on knowledge and reasoning, and relatively well calibrated on WildGuardTest and When2Call. Live scores can move after this launch snapshot.
Artificial Analysis
Published:
Source checked:
[139] Artificial Analysis Decision Index v1.0 launch article
The 13 October 2026 launch article scores Jev 1.13 at 58 on Decision Index v1.0, ahead of GLiDE and Microsoft-Decision-1 at 54. Claude Haiku 5.5 (Low) scores 61 at 1.4 seconds per task and $184 per million tasks. GPT-6 Luna (Low) scores 55 at 1.5 seconds and $131. Jev’s median is 0.13 seconds and $42 per million tasks.
The index is an equal mix of AA-Omniscience-Decision (500 private five-option questions, accuracy and non-hallucination at 12.5% each), HLE-MultiChoice (513 text-only multiple-choice items), When2Call (3,423 NVIDIA tool-calling tasks after dropping 229 ambiguous ask-for-information items), and WildGuardTest (835 agreed rows, average of three F1 scores). Language models answer with structured JSON that names one option and supplies no probability. Jev scores 21 on HLE-MultiChoice, 35 accuracy and 35 non-hallucination on AA-Omniscience-Decision, 81 on When2Call, and 94 on WildGuardTest. Haiku 5.5 (Low) scores 24, 52, 73 and 93 on those four. Luna (Low) scores 22, 66, 77 and 89. The article says these are launch figures; the live leaderboard can change.
Artificial Analysis
Published:
Source checked:
[140] Artificial Analysis Decision Index methodology
Version 1.0 weights four evaluations at 25% each. A decision model answers every question in one call. For a choice, Artificial Analysis scores the option with the highest probability. For a yes/no question it takes yes at probability 0.5 or more. Language models receive a fixed system prompt and must return JSON through structured outputs.
AA-Omniscience-Decision non-hallucination is 100 × (1 − hallucination rate), where hallucination is a wrong answer instead of “I don’t know”. HLE-MultiChoice uses the 513 text-only multiple-choice questions from cais/hle, scored as an exact letter match. When2Call uses NVIDIA’s multiple-choice test set after filtering 229 tasks. WildGuardTest keeps rows where all three annotators agreed on prompt harm, response harm and refusal, and drops questions that consistently trigger model refusals. Cost per million tasks is one pass through the index, divided by the combined task count, times one million.
driller (@patraqushe)
Published:
Source checked:
[141] Zenn post on Claude Max credits used like Jev
The post links a Zenn article that tries Claude Max plan API credits as a Jev-style classifier and compares jev-latest with Haiku 5.5, Sonnet 5.5 and Opus 5.5 on three Japanese tasks.
Posted 10 October 2026, the same day the article went up. The article was edited on 11 October. The post itself carries no scores.
driller
Published:
Source checked:
[142] Claude Max API credits used like Jev on three Japanese sets
The article scores jev-latest against Claude Haiku 5.5, Sonnet 5.5 and Opus 5.5 on MASSIVE ja-JP scenario labels (216 items), chABSA sentiment (210) and JGLUE JNLI v1.3 (210). After rewriting the chABSA and JNLI option text to match the annotation guides, Jev accuracy is 0.884, 0.910 and 0.838. Opus probabilities mode is 0.926, 0.881 and 0.914. Jev’s median latency is 0.16 to 0.17 seconds against 1.3 to 3.5 seconds for the Claude rows.
Published 10 October 2026, edited 11 October. Claude calls go through typesafe-ai/system-one-adapter-python with structured outputs. Haiku ran in discrete and probabilities modes; Sonnet and Opus ran in probabilities mode. Effort is each model’s default. Each class has the same count, seed 42, one run per setup. McNemar finds Opus ahead of Jev on JNLI (4 Jev-only correct, 20 Opus-only, p=0.002). The MASSIVE Opus gap is p=0.035 and the article says it is not significant after comparing ten pairs on one set. Cost per 1,000 items is $0.02 to $0.04 for Jev. The Claude API bill for the check is $14.41; Jev is about $0.03 on a separate bill. A cascade that sends low-confidence Jev items to Sonnet or Opus is a replay of these labels, and the author calls those accuracies optimistic.
Yuchen Ling, Shengcheng Yu, Zhenyu Chen and Chunrong Fang
Published:
Source checked:
[143] Jev action selection in ALFWorld and AndroidWorld, arXiv:2610.13092
The paper swaps gpt-5.6-luna action selection for jev-1.13.0 across five agent families, 4,500 executions, ALFWorld’s 134 valid_unseen tasks and AndroidWorld’s 116 task types, three runs each. ReAct-J succeeds on 96.02% of ALFWorld tasks, 1.49 points under the original agent. Reflexion-J succeeds on 98.01%, 1.74 points under. On AndroidWorld, Jev trails T3A by 13.22 points, AutoDroid by 2.30 and FastAgent by 18.10.
arXiv:2610.13092v1, 9 October 2026. All LLM decisions and auxiliary generation use gpt-5.6-luna. Jev accepts text-only observations. ALFWorld also has an LLM-Choice control that uses Luna with Jev’s candidate list; those gaps shrink to 1.00 and 0.00 points. Decision cost falls 66.1% to 78.4% and decision time 73.9% to 81.5%, from 4.13 to 8.13 seconds to 1.04 to 1.75 seconds. Mean task cost falls in every family. T3A’s cost per delivered success rises 7.7%, from 68.70 to 73.97 mUSD, with a 95% interval that includes zero. FastAgent’s time per success rises from 239.4 to 377.8 seconds. Experimental code is at anonymous.4open.science/r/jev-emp-5BF4/.
Arize AI (@arizeai)
Published:
Source checked:
[144] Arize thread on eight decision models
The opening post says Arize benchmarked eight decision models on hallucination detection, 13 Phoenix evaluator suites, and agent routing. It writes that Jev, Clef 27B and Liquid sit within a point of each other on hallucination, and within the noise of an LLM judge that costs 60 to 400 times more.
Thread on 7 October 2026 linking the blog. A follow-up names Jev for a hosted API, Kev for self-hosting, and Clef 27B for open weights with a hosted option, and says to test the question as hard as the model. The thread itself does not print the RAGTruth percentages.
Jim Bennett
Published:
Source checked:
[145] Arize decision-model benchmark: Jev, OpenAI Decisions and six others
On 2,675 RAGTruth rows, Jev 1.13 scores 85.5% balanced accuracy at a tuned cutoff, against 81.5% for OpenAI Decisions on gpt-6-luna. Median latency is 0.08 seconds against 0.16, and cost per 1,000 is $0.045 against $0.090. The 95% intervals on accuracy touch, so Arize records a draw there and sends Jev through on speed and cost. On 655 Phoenix cases Jev scores 91.1% and OpenAI 90.0%. On 52 routing cases Jev is 49 of 52 and OpenAI 48 of 52.
Blog dated October 2026, posted on X on 7 October. OpenAI is the only model that does not accept Jev’s /v1/systemone layout; Arize wrote each case as labelled text sections and says a different layout could score differently. Accuracy wins require non-overlapping 95% intervals. Each model gets its own yes/no cutoff, fit on half the data and scored on the other half, same seed as Arize’s earlier Jev-as-a-judge work. An earlier Arize Opus 5 judge scored 86.6% balanced accuracy at $14.30 per 1,000. Reversing the hallucination question drops Liquid d1 from 0.92 ROC AUC to 0.17; Jev, Clef 27B and OpenAI stay near 0.93, 0.92 and 0.90 on both wordings.
Diogo Almeida (@CompleteSkeptic)
Published:
Source checked:
[147] Almeida’s JevDay thread: danger-zone opt-in
Almeida opened a JevDay thread by separating early adopters from production infrastructure. He writes that the first Jev release favored stable infrastructure, and that this release is for tinkerers, with a gate so people can opt into the danger zone.
Opening post of the 14 October 2026 announcement thread. Product details sit in later posts in the same thread and in the TypeSafe blog.
Diogo Almeida (@CompleteSkeptic)
Published:
Source checked:
[148] Almeida: jev-1.13.0 was already an image model
Almeida writes that the shipping Jev model was multimodal the whole time, with images disabled because models tend to be less intelligent on other modalities. The opt-in header is TypeSafe-Danger-Zone-Images: true.
Second post in the JevDay thread. The Images docs add request format, size limits, and a production warning. This post does not publish accuracy numbers.
Diogo Almeida (@CompleteSkeptic)
Published:
Source checked:
[149] Almeida: Box is a single bounding box on jev-preview
Almeida announces a new question type, box, that returns a single bounding box. He places it in a gray area between decisions, and says it is only on jev-preview.
Third post in the JevDay thread. The Box docs require jev-1.14.0 or jev-preview, an image in the request, and the images header.
Diogo Almeida (@CompleteSkeptic)
Published:
Source checked:
[150] Almeida: context lengths should be up to 262k
Almeida writes that Jev already supported longer contexts than TypeSafe allowed in production. He says the earlier cap was a product choice, and that context lengths should be up to 262k. The opt-in header is TypeSafe-Danger-Zone-Long-Context: true.
Fourth post in the JevDay thread. The models page documents 262k as both the per-request and per-question limit when the header is set, on jev-1.13.0 and jev-1.14.0. Default limits stay 64k / 32k.
Diogo Almeida (@CompleteSkeptic)
Published:
Source checked:
[151] Almeida: an opt-in to help find jaggedness
Almeida says TypeSafe added an opt-in flag so users can help find model jaggedness. He writes that TypeSafe still will not train on that data.
Fifth post in the JevDay thread. The post does not name the header.
Diogo Almeida (@CompleteSkeptic)
Published:
Source checked:
[152] Almeida: jev-1.14.0 ships as jev-preview
Almeida lists jev-1.14.0 as jev-preview. He calls it better overall, mostly a drop-in replacement, and strongest on multilingual support. jev-1.13.0 stays available and is not deprecated.
Sixth post in the JevDay thread. The blog adds the one-week jev-latest cutover and the 60-day / 30-day deprecation warnings.
TypeSafe AI
Published:
Source checked:
[153] Jev: For the builders AND dreamers
The Jev Day blog ships jev-1.14.0 under jev-preview, with experimental images, a Box type, and longer context behind danger-zone headers. TypeSafe says jev-latest will move to jev-1.14.0 in a week, jev-1.13.0 will stay available, and it intends 60 days of warning before deprecating 1.13.0 and 30 days for 1.14.0. Images are composable through an assets map. TypeSafe says models, including Jev, perform worse on multimodality than on pure text.
Company blog, metadata timestamp 15 October 2026 00:22 UTC; the models page dates Jev 1.14 as available from 14 October 2026. The Measuring Intelligence section describes an internal ranking of open-weight models and asks readers not to treat it as settled. The text we opened does not print those scores, so this site does not repeat a table we could not read as numbers.
TypeSafe AI
Source checked:
[154] Long context: 262k behind TypeSafe-Danger-Zone-Long-Context
Long context is off by default and works on Jev 1.14 and Jev 1.13. The header TypeSafe-Danger-Zone-Long-Context: true raises both the per-request and per-question limits to 262k tokens. Requests without the header keep 64k / 32k. The same per-token price applies. The page says long context is slower and less accurate than the default limits.
Official models documentation as accessed 15 October 2026. There is no separate cap on the number of questions beyond those two context limits.
TypeSafe AI
Source checked:
[155] Images: assets map, size limits, and the images header
Images and Box questions are off until the request sends TypeSafe-Danger-Zone-Images: true. Each image is listed once in assets and referenced in state, instructions, or criteria. Jev 1.13 and Jev 1.14 both accept images. Formats are PNG or JPEG stills, 16 MiB per file, 256 to 1568 pixels on each side, with a 32 MiB request-body limit. The API strips EXIF and does not apply EXIF orientation. Image tokens are billed as input at the same per-token price as text.
Official Images page as accessed 15 October 2026. The page labels images and Box as dreamer-ready, below TypeSafe’s production bar, and says the format, answers, and limits can change between releases. A 256×256 image is below the minimum pixel area. The page does not publish a token-count formula for images.
TypeSafe AI
Source checked:
[156] Box: four edges as fractions of the image
A Box question returns top, bottom, left, and right as fractions from 0 to 1, measured from the top-left of the image, plus a confidence value. Box needs an image in the request, the images header, and jev-preview or jev-1.14.0. The type takes no criteria. A Box answer is always a box, so the page pairs it with a Noul that asks whether the target is present. On 36 test crops of paintings with a known cut, four edge questions caught 30 of the cuts, a single all-inside question caught 26, and an outline on the full image caught 20.
Official Box documentation as accessed 15 October 2026. The 36-crop check is TypeSafe’s. There is no recommended confidence threshold; the 0.6 figure in the padding example is labelled as an example to tune.
Erik Gafni (@EGafni)
Published:
Source checked:
[157] Gafni posts the Jev 1.14 blog with a Danger Zone line
TypeSafe CTO Erik Gafni linked the Jev 1.14 blog and named image inputs, a new output type, and longer context, with the line “Highway to... the Danger Zone.”
Posted two minutes after Almeida’s thread. The post is a pointer to the blog. It adds no extra measurements.
OpenRouter (@OpenRouter)
Published:
Source checked:
[158] OpenRouter says Jev 1.14 is on OpenRouter
OpenRouter posted that Jev 1.14 is on OpenRouter. The post lists small overall gains, multilingual as the largest upgrade, experimental image input, and an unchanged price of $0.042 per million input tokens with output free.
Posted 15 October 2026 at 00:16 UTC. The post does not print an OpenRouter model id. openrouter.ai/typesafe/jev-1.14 returned not found on 15 October 2026.
Peter Banda (@0xbnd)
Published:
Source checked:
[159] Peter Banda: a 128K-token danger-zone test
Banda posted “Happy JevDay! Tested the danger zone today - up to 128K tokens” with a screenshot.
A single field note on 14 October 2026. The post names no model id, no question count, no latency, and no accuracy. 128K is below the documented 262k header limit.
Meta
Source checked:
[160] Muse Wonder model page: variants, price, and Muse Code access
The page lists Muse Wonder (muse-wonder-1.0) on the Meta Model API and Muse Code at $4 input, $0.20 cached input, and $20 output per million tokens, with a 1M-token context window. Muse Wonder Full (muse-wonder-full-1.0) is listed for Muse Code Power Usage ($50/month) and Maximum Usage ($200/month), with no public per-token API price.
Official product page as accessed 15 October 2026. The benchmark table on the same page compares Muse Wonder and Muse Wonder Full with Claude Fable 5.1, GPT-6 Astra, and Claude Opus 5.5. The page does not mention Jev. Jev is listed here because the note is cited from the Jev and Muse Wonder report.
Meta Superintelligence Labs
Source checked:
[161] Muse Wonder research blog, 12 October 2026
The 12 October 2026 post introduces Muse Wonder and Muse Wonder Full for personal and professional agent work. Muse Wonder is available through the Meta Model API and Muse Code. Muse Wonder Full is available for Muse Code subscribers. The post says Muse Wonder will power experiences in the Muse assistant, first for subscribers.
Vendor research post. Active Memory and Active Focus are Meta’s names for context carry-over and sustained effort. The post does not mention Jev. Jev is listed here because the note is cited from the Jev and Muse Wonder report.
TypeSafe AI
Source checked:
[162] Jev is not a drop-in for Muse Spark or other coding-agent LLMs
TypeSafe says Jev is not a drop-in replacement for the LLM behind Claude Code, Cursor, opencode, Copilot, Muse Spark, Grok Bot, or similar tools. There is no model: "jev-latest" setting that turns a coding agent into a Jev-powered agent. The page tells readers to keep the coding-agent LLM and use Jev separately for typed decisions.
Official coding-agents page as accessed 15 October 2026. It names Muse Spark, not Muse Wonder. Muse Wonder is Meta’s later coding and agent model.
TypeSafe AI
Source checked:
[163] TypeSafe agent skill for coding agents
The official skill gives a coding agent the TypeSafe API types, patterns, and evaluation structure. Installers named on the page are the Claude Code plugin from typesafe-ai/skills and npx skills add typesafe-ai/skills --skill typesafe-ai.
Official skill page. It does not name Muse or Muse Code in the installers shown on 15 October 2026.
Artificial Analysis (@ArtificialAnlys)
Published:
Source checked:
[164] Artificial Analysis scores Muse Wonder 52 and Wonder Full 56
Artificial Analysis scores Muse Wonder (max) at 52 and Muse Wonder Full (max) at 56 on Intelligence Index v4.3, against 48 for Muse Spark 1.3 (max). Muse Wonder is on Meta’s first-party API. Muse Wonder Full is only for Muse Code users. Wonder token prices in the post are $4 input, $0.20 cached, and $20 output per million tokens. Full has no per-token price in the post.
Independent language-model index. These scores compare Meta’s models with other language models. They are a different test from Artificial Analysis’s Decision Index for Jev. Jev is listed here because the note is cited from the Jev and Muse Wonder report.
codila (@0xCodila)
Published:
Source checked:
[165] Jev + Muse playbook post with a seven-step setup
The post describes a prompt → Muse → Jev decision → Muse execution loop and links Bodila51/muse-jev-playbook. Shadow mode for 20–50 decisions, then active mode with act at 0.80 and above, advisory between 0.50 and 0.79, and human escalation below 0.50.
Personal setup post dated 22 September 2026, three weeks before Muse Wonder. The “100% of my life” and “200x” lines have no measurement next to them. The post names Muse, not muse-wonder-1.0.
codila
Source checked:
[166] muse-jev-playbook README at commit 70edf68
The README adapts grok-bot-jev for Muse/Hatch, custom harnesses, or scripts. Track A uses the TypeSafe SDK. Track B says a Muse/Hatch agent already has the credential and follows skill/jev-decision-layer.SKILL.md. Policy bands are act at 0.80 and above, surface between 0.50 and 0.79, and escalate below 0.50.
Single commit dated 22 September 2026. The file says the A/B numbers in examples/ are one local run, a method template rather than a benchmark.
codila
Source checked:
[167] Flight-date case: jev-1.13.0 on one local fare table
One local run on 22 September 2026. The agent collected Google Flights prices; jev-1.13.0 ranked months and days. On a 92-day one-way table, month confidence is 1.0 for October, day 0.67 for 5 October, and book-now 0.82. On the round-trip table, both trip lengths pick April 2027 at 1.0, and book-now is 0.79.
The file says Jev did not fetch the fares, the numbers are a snapshot, and the run is not a benchmark. Fares need re-checking before a purchase.
jkpelaez (@jkpelaez)
Published:
Source checked:
[168] Jev skill on Muse for CRM dedup and entity typing
The author says installing a Jev skill on Muse sent “same person?” and entity-type questions to Jev. He reports 1,300+ CRM records processed, three duplicates merged, and every record typed against his vocabulary. The agent routes on the typed result.
First-hand field note dated 30 September 2026. The post names Muse and a CLI call. It does not name a Muse model id, a Jev version, a timed comparison, or Muse Wonder.
aaddrick
Source checked:
[169] Unofficial Jev skill with Muse and Muse Code installers
The README at commit a46856c, dated 1 October 2026, says the unofficial skill installs in Claude Code, Codex, Antigravity CLI, Muse, and Muse Code. Muse copies the skill into ~/workspace/skills/. Muse Code uses muse skills install with --scope user. The skill is Markdown. A TypeSafe key is optional.
Community skill, not TypeSafe’s. The snapshot date in the file is 25 September 2026. The evals in the same README used claude-sonnet-5 writing Jev code, not a Muse Wonder runtime. The commit predates the 12 October Muse Wonder release.
theglitcharchitect
Source checked:
[170] muse-skills: Jev router plus 23 skills for Muse agents
Commit f0cc9cc installs a Jev decision layer and 23 skills into ~/workspace/skills/. The bundled demo runs against a mock gateway with no key. Live mode uses VERCEL_AI_GATEWAY_KEY. Shadow mode logs the verdict and still runs the action. Spending, sending, deleting, and publishing escalate to a person in both modes.
Commit dated 21 September 2026. The demo probabilities (including p = 0.03 on a mock rm -rf) are from that mock walkthrough. The README names Muse agents and does not name muse-wonder-1.0.
Phanindra Reddy (@phanindra_ai)
Published:
Source checked:
[171] Application for a Muse connector on jev.magicteams.ai
The author wrote that he had applied for a Muse connector for jev.magicteams.ai, so a Muse could build the Jev.
An application dated 19 September 2026. The post does not show an approved connector, a runtime log, or a Muse model id.
Diogo Almeida (@CompleteSkeptic)
Published:
Source checked:
[172] Almeida posts Jevtember quoting Vercel
Almeida posted the single word Jevtember, quoting Vercel’s 15 October thread that called Jev AI Gateway’s fastest-adopted model and September’s most-requested model. The attached image repeats the word.
Posted 18 October 2026. The post is a quote-tweet with no extra numbers. Replies include jokes and congratulations; they add no Gateway measurements.
Vercel (@vercel)
Published:
Source checked:
[173] Vercel names Jev fastest-adopted and most-requested
The second post in Vercel’s October 2026 AI Gateway Index thread said Jev became the gateway’s fastest-adopted model ever and September’s most-requested model.
Posted 15 October 2026 with a chart. The numeric tables live in the linked October Production Index, not in this post.
Aaron Batilo (@aaronbatilo)
Published:
Source checked:
[174] Wake me up when Jevtember ends
Aaron Batilo posted a one-line Green Day parody, “Wake me up, when Jevtember ends,” four days after TypeSafe launched Jev.
Dated 19 September 2026. The post has no adoption figures. It is an early public use of the nickname on X.
Rohan Taneja, Zachary Chen and Jerilyn Zheng (Vercel)
Published:
Source checked:
[175] Jev listed on Vercel AI Gateway
Vercel listed Jev on AI Gateway as typesafe-ai/jev, reached through AI SDK 7’s experimental evaluate API from version 7.0.105. The changelog describes Choice, Score, and Boolean questions against shared state.
Changelog dated 16 September 2026. Boolean here is Vercel’s name for TypeSafe’s Noul primitive. The October index later prints the gateway launch date as 17 September.
Amelia Charles, Harpreet Arora and Eric Dodds (Vercel)
Published:
Source checked:
[176] Jev is the fastest-adopted model in AI Gateway history
Vercel wrote that within 24 hours on AI Gateway, Jev reached more than twice as many paid teams as any previous model launch. By hour 24 nearly 13% of paid teams were using it, twice the GPT-5.6 family and more than six times Fable 5.1. A tenth of teams arrived within 18 hours. Every other recent launch stayed below 7% after a full day.
Blog dated 18 September 2026. The percentages are shares of paid AI Gateway teams. The post says the next test is whether first-day adoption lasts. It restates TypeSafe workflow evaluations as up to 194 times faster and 445 times cheaper; those figures originate with TypeSafe, not with a Vercel rerun.
Amelia Charles and Harpreet Arora (Vercel)
Published:
Source checked:
[177] AI Gateway Production Index, October 2026
The October 2026 index names Jev the fastest-adopted model in gateway history. In its first two weeks Jev reached 35.3% of paid teams, received the most requests of any September model, and saw 55.6% of week-one teams return in week two. The opening summary puts that two-week window at 0.27% of gateway spend. Later in the same article, Jev accounts for 1% of September tokens and 0.15% of September spend. The scorecard lists spend share 0.3% and a composite score of 65.9, sixth among September launches. The table’s gateway launch date is 17 September.
Published 15 October 2026. Launch metrics cover external paid teams in each model’s first 14 days. Reach uses paid teams active from the start of the launch month through the data cutoff. Spend share uses total paid gateway spending during the model’s window. Return rate is first-week teams that used the model again in week two. Spending is estimated at list prices. The traffic is Vercel AI Gateway only.
Russell Brandom
Published:
Source checked:
[178] Jev is Vercel’s fastest-adopted AI model ever
Brandom reports Vercel’s October index: within two weeks Jev was used by 35% of paid teams, nearly double Sonnet 5.5. He notes the gateway undercounts labs that sell their own APIs. He also writes that Vercel replaced GPT-6 Luna with Jev for classifying harmful commands.
TechCrunch, 16 October 2026. The 35% figure rounds Vercel’s 35.3% two-week reach. The Luna replacement is TechCrunch’s account of Vercel’s own use; this page does not have a separate Vercel engineering post for that swap.
Stephen Council
Published:
Source checked:
[179] A Jev hackathon at CodeRabbit in San Francisco
Council attended a Jev hackathon on a Saturday at CodeRabbit’s San Francisco office. Allie Laabs, TypeSafe’s head of developer relations, told the room Jev fits classification work such as sorting support queries. Council writes that TypeSafe’s Discord had more than 100,000 members and that the launch video had almost 40 million views on X. Named attendees describe faster tool selection and lower cost in their own products; the article prints no scored benchmark from the event.
Business Insider, 29 September 2026. The Saturday is not given a calendar date. Qualitative comments stay with their speakers. A valuation rumor attributed to The Information is outside this page.
Mikal Khoso
Published:
Source checked:
[180] Jev for SEO: what Tenpoint Labs measured on five jobs
Khoso ran jev-1.13.0 on five SEO jobs between 3 and 9 October 2026, across 3,566 AI answers and about 30,000 search queries and keywords. Brand mentions: 140 of 140 sampled named verdicts were correct, with about 6% of real mentions lost to a name-form question. Search intent on 14,902 queries across five unseen sites matched a reader on about 86%. A paid classifier scored about 48% on an earlier 1,326-keyword set. Jev agreed with that classifier on 54.9% of keywords; a blind reader sided with Jev on 86 of 120 sampled disagreements and with the classifier on 17. Fan-out readers disagreed on 24 of 30 queries. A branded split on 2,944 of Tenpoint’s own queries agreed with a regex on 99.4%. Keyword relevance on 2,804 keywords from one site was roughly level with a larger model.
Tenpoint Labs is a TypeSafe customer. tenpointlabs.com supplies 2,944 queries, Search Console web search from 9 June 2025 to 9 October 2026; anonymised queries are omitted and no GA4 data is used. Client accounts are described by industry and never named. The reader for most jobs is a larger model; Khoso labelled 120 rows himself. Production thresholds in the post are 0.8 as yes and 0.2 as no. A search query with one question used about 420 input tokens; median latency was near 400 ms. We have not rerun the jobs.
Nico Gorrono
Published:
Source checked:
[181] Jev for SEO: three tests on two live sites
Gorrono ran three jobs. On airankingskool.com, Jev judged 3,182 paragraphs for $0.12 after a local embedding shortlist of four pages. Rules required 70% confidence, at most three new links per page and eight per target. The first run proposed 147 links; he approved 109 and rejected 38. Two days of the same site matched on 96.7% of picks. On propiedadesaysen.com, 64 overlapping Search Console searches split into 26 wrong-page winners, 19 below 70%, 18 already right, and 1 splitting signals. The Jev bill for those 64 was $0.003. On 40 ChatGPT answers, the site was mentioned in 11 and recommended in 2; mentioned-or-not matched a text search on 40 of 40.
Article dated 24 September 2026. Internal-link rejections included hub-keyword clashes, quoted-prompt anchors, and related-but-misleading targets. Only 3 of 64 cannibalization fix labels reached 70% confidence. ChatGPT answers came from DataForSEO at $0.004 each, 20 Spanish from Chile and 20 English from the US. Gorrono founds AI Ranking. We have not rerun the jobs.
Nicolas Gorrono
Source checked:
[182] jev-internal-links skill README at commit 5f76ff2
The README at commit 5f76ff2 reports 3,182 Jev decisions on a 164-page site, 112 links proposed, and $0.75 total including Claude for anchors. It repeats the 96.7% same-pick figure, and 100% when confidence was 0.88 or more. Jev is about $0.04 per 1,000 paragraphs; Claude anchors about $0.60 per 150 pages.
Read at commit 5f76ff2, 23 September 2026. The $0.75 figure includes the writing model. Gorrono’s article prices the Jev layer alone at $0.12 for the same 3,182 decisions and reports 147 proposals before review. The skill does not edit the live site.
Agrici Daniel
Source checked:
[183] jev-seo README: crawl, 52 rules, and Jev page questions
jev-seo crawls from a homepage URL, checks 52 rules tied to Google Search Central, measures Core Web Vitals through PageSpeed Insights, and asks Jev 13 questions per page and 5 per site. Standard mode costs about a cent in Jev per site. Optional --full adds DataForSEO for about $0.30. Default page cap is 60. Scores rank work and do not predict rankings or traffic. Without a TypeSafe key, Jev sections are marked not assessed.
Read at commit 55a184a, 22 September 2026. MIT licensed. Jev cost is listed at $0.042 per million input tokens, about $0.00015 per page. DataForSEO volumes and PageSpeed lab scores are third-party estimates. Heuristic rules are labelled as heuristics.
Agrici Daniel
Source checked:
[184] jev-seo evaluation.md: repeatability and a second-model judge
Dated 2026-09-22. Fifty-nine pages judged twice: page type 58/59 identical, intent 59/59, suggested action 54/59. Score answers moved by 0.001 to 0.023 on average. When both runs were decisive on page type they agreed 44/44. Against a blind second model on 30 pages and 40 keywords, helpfulness matched on 29/30 at a 0.45 threshold and specificity on 28/30. After wording changes, decisive helpfulness and specificity were 28/29, keyword relevance 27/30, opens-with-the-point 23/23, and clear next step 16/16.
The second judge is another model, not a human. The file says there is no human-labelled answer key yet, and that thresholds are not tuned against human labels. Rule-layer claims on claude-seo.md were re-fetched from the live site; word counts were within 5%. Page type against the second judge was 20/30 before the wording change.
GitHubDaily (@GitHub_Daily)
Published:
Source checked:
[185] GitHubDaily posts jev-seo as a homepage-to-audit tool
GitHubDaily described jev-seo as an open-source audit that crawls from a homepage, checks 52 Google Search Central rules, and asks Jev what each page is for, how specific the content is, and whether pages compete for the same query. The post names PDF, Excel, and Markdown outputs, a standard-mode Jev cost of about one cent per site, and a two-run score gap of 0.03 on 59 pages.
X post, 30 September 2026. The 0.03 figure matches the README’s rounded repeatability line. evaluation.md at commit 55a184a prints score movement of 0.001 to 0.023. Use the evaluation file for the measured table. A reply noted that competitor data in --full mode depends on DataForSEO.
Sam Underwood
Published:
Source checked:
[186] Judging page quality with Jev in SERP Lens
Underwood put Jev in SERP Lens to score page quality for under a penny a page. A rubric drawn from Google’s helpful content guide and quality rater guidelines covers Depth, Production quality, Padding, Gets to the point, Written for people, Supported claims, and Trust signals. After one request per page, clearer levels, and stripping chrome from the capture, agreement with reviewers within one level moved from 64% to 82%. Reviewers were Underwood plus Astra, Fable, and Opus 5.5.
The page heading is Oct 2026. Underwood’s 20 October X post links this write-up. The first attempt used many calls per page, scored Wikipedia as low quality, and could differ across two runs. The held-out page count is not printed. Underwood builds SERP Lens with Brodie Clark.
Sam Underwood (@SamUnderwoodUK)
Published:
Source checked:
[187] SERP Lens uses Jev for page quality in the browser
Underwood posted that SERP Lens can evaluate page quality with TypeSafe’s Jev for under a penny a page. The same release checks whether a page answers Google Search Console queries, People Also Ask and AI Overview coverage, and labels SERP formats and site types. He pointed at the blog write-up for the method.
X post, 20 October 2026. Product announcement from a SERP Lens author. The 64% to 82% within-one-level figures live in the blog, not in this post.
Composio
Published:
Source checked:
[188] Introducing Composio Firewall
Composio’s 21 October 2026 blog introduces Firewall as a runtime boundary for agentic software. Every action through Composio is evaluated using Jev and custom small language models tuned on usage data, before execution, against user intent and developer and user limits. Tool results are checked before they reach the agent. Checking a result does not undo an action that already ran. The post says Firewall is live for all Composio developers and points at the pricing page.
Vendor launch post dated 21 October 2026, 12:00 UTC in the page’s BlogPosting markup. It prints no Jev version id, miss rate, false-alarm rate, or added latency. The “millions of users” line is Composio’s own.
Composio
Source checked:
[189] Composio Firewall docs: session hooks and condition checks
Firewall blocks tool calls or tool results that match a natural-language condition, and includes a prompt-injection check on results. Hooks attach to a session as before_execute or after_execute. after_execute cannot undo a tool that already ran. The page is labeled Experimental and says Firewall is available on paid plans with usage-based pricing. A classifier sees the tool, arguments, returned data on after_execute, and earlier session activity except on zero data retention projects. Hooks skip Composio meta tools.
Opened 24 October 2026. Example conditions include sharing internal pricing outside acme.com and emails that contain a password or API key. Blocked calls return result_type blocked and error Blocked by session policy. REST examples use POST https://backend.composio.dev/api/v3.1/tool_router/session.
Composio
Source checked:
[190] Composio Firewall product page
The page says Firewall reads every tool result before the agent does and blocks results that carry hidden instructions in emails, pages, and files. A specialised model screens results in full. Demos include an INV-0931 invoice email with 1px white text that tells the assistant to export HubSpot contacts, and GitHub pull request 482 whose README hides an instruction to approve and merge without waiting for CI. Four measurement headings (attacks that get through, false alarms, work that gets done, added time) have no printed numbers.
Opened 24 October 2026. The task-scope demo uses “Pay this week’s vendor invoices” with named vendors and account [email protected]. SoftwareApplication markup names Composio Firewall as a SecurityApplication. The page is a product demo, not a scored evaluation.
Composio
Source checked:
[191] Composio pricing: Firewall add-on rates
Firewall uses decision models to check tool requests and results against policies. The add-on table lists Hobby as No, Pro as $0.50 per 1M tokens evaluated, and Enterprise as included. A FAQ repeats the Pro rate and says Firewall is included on Enterprise. Sandbox LLM tokens are billed on a separate line.
Opened 24 October 2026 from composio.dev/pricing and the accompanying pricing.md. Hobby is the $0 plan with 100,000 tool calls per month. The Firewall docs still describe paid plans and usage-based pricing. Keep the Hobby “No” row next to the blog’s live-for-all-developers line.
Composio
Published:
Source checked:
[192] Session hooks in Python 0.30.0 and TypeScript 0.27.0
The 22 October 2026 changelog adds session hooks in Python composio 0.30.0 and TypeScript @composio/core 0.27.0, with @composio/slim also at 0.27.0. Hooks run before or after execution and can apply to selected toolkits. Updating hooks replaces the list. None or null removes them. Hooks cannot be combined with experimental.session_config_id in the same call. Blocked execution returns result_type blocked, empty data, and block_reason. A blocked proxy request has no HTTP response.
Same day’s changelog also ships Python 0.29.1 and 0.29.0 and TypeScript 0.26.0, which are separate from the hooks API. Provider integrations return a blocked execution as a failed tool call with the hook’s message.
Karan Vaidya (@KaranVaidya6)
Published:
Source checked:
[193] Karan Vaidya introduces Composio Firewall with Jev
Vaidya posted that Composio Firewall is the runtime boundary for agents, live for all Composio developers. Every action through Composio is evaluated using Jev and custom SLMs tuned on real-world usage data. The firewall checks actions and information coming back, including protection against prompt injection. The post includes a launch video.
X post, 22 October 2026, from the Composio founder. The “millions of users” line is his. The post prints no Jev version, price, or detection rate. TypeSafe quote-tweeted this post later the same day.
TypeSafe AI (@typesafeai)
Published:
Source checked:
[194] TypeSafe calls Composio Firewall a code firewall powered by Jev
TypeSafe quote-tweeted Karan Vaidya’s Firewall launch and described it as a code firewall powered by Jev. The quoted post is the 22 October Composio launch that names Jev and custom SLMs.
X post, 22 October 2026. Vendor acknowledgement of the Composio pairing. The post adds no version id, price, or scored evaluation.
Grantry
Published:
Source checked:
[195] Grantry notes on the Composio Firewall product page
Grantry retrieved the Firewall product page on 21 October 2026 and recorded the invoice and GitHub demos, a specialised-model screen on tool results, and four measurement headings without values. On that day the product page showed no pricing table and Grantry found no dedicated Firewall docs page.
Independent write-up dated 21 October 2026. Treat Grantry as a 21 October snapshot of the product page. Docs at docs.composio.dev/docs/firewall and the $0.50 Pro add-on were open on 24 October.
Composio
Source checked:
[196] Composio TypeSafe provider: Jev chooses a tool
The TypeSafe provider lets Jev choose a Composio tool from a request. Jev answers typed questions. The provider turns those answers into a tool call, a partial call that needs more arguments, or an abstain. decide does not execute a tool. Install pins typesafe-sdk to 0.6.0.
Opened 24 October 2026. This page documents the tool-selection provider. Firewall session hooks are a separate Composio product.
knut (@kmelve)
Published:
Source checked:
[197] Sanity’s internal harness routes agents with Jev
Knut Melvær writes that Sanity uses Jev as the router in an internal agent harness that lets anyone at the company spin up a bespoke agent. He names co-authors Rostislav Melkumyan (@rostimelk) and Even Westvang (@even) and links an engineering post.
X post, 22 October 2026. The post does not print a Jev version, a routing accuracy, or a latency. The linked post is the @s harness write-up.
TypeSafe AI (@typesafeai)
Published:
Source checked:
[198] TypeSafe on Sanity’s bespoke agents routed by Jev
TypeSafe quote-tweeted Melvær’s harness post and described bespoke agents with bespoke knowledge bases, instructions, tool-sets, and routing with Jev.
X post, 22 October 2026. Vendor acknowledgement of the Sanity pairing. The post adds no version id, score, or latency.
Even Westvang, Rostislav Melkumyan and Knut Melvær
Published:
Source checked:
[199] How Sanity built the @s company agent harness
Sanity’s @s harness treats each agent as a Studio document and asks Jev which published agent should handle a Slack message. At the time of writing, Jev chose among 66 agents plus one option for questions about @s itself. Confidence below 0.4 returns the three most likely agents for the user to pick. The state is the message plus up to the last 20 messages in the thread.
Engineering post last edited 22 October 2026. Follow-ups stay with the current agent unless the new question belongs elsewhere. Watched channels get a yes-or-no on whether the agent should reply. About 42 of 61 hackathon agents worked fully or partly; most failures were missing access, not routing. HAL, a meeting-hygiene agent, answered everyone when its description said “workstreams,” and the team tightened that description. The post names no Jev version and prints no router accuracy. Inference goes through OpenRouter with zero data retention. The post links an earlier 0.5-second docs-agent write-up.
John Siciliano and Knut Melvær
Published:
Source checked:
[200] Sanity’s 0.5-second docs agent with Jev, Haiku and gpt-oss-120b
The write-up times four decisions that used to sit on an LLM. Entry choice is 282 ms for Jev against 1,328 ms for Haiku. Model routing is about 300 ms against 818 ms. MCP choice is 170 ms against 829 ms. Speculation while the user types removes 1,828 ms at enter. The median full answer is 0.5 seconds. Three of four finish under 0.9 seconds.
Engineering post dated 24 September 2026. The answering model in the demo is openai/gpt-oss-120b on Cerebras. The authors score 20 of 22 answers as fully grounded in the docs, with the first token in about 350 ms. Routing made answers cheaper, about 4.5 times, and slower at the median, so they turned routing off for the 0.5-second demo. Combined entry, model, and MCP questions took about 340 ms. The post calls the numbers indicative: one week of testing on one corpus. Jev is called as jev-latest in the sample request, and the sample response logs jev-1.13.0.
Lightpanda (@lightpanda_io)
Published:
Source checked:
[201] Lightpanda puts Jev inside the browser as judge()
Lightpanda says it put Jev inside the browser binary and built judge() on top of it. The call asks predefined questions and reports whether the page has real content and what kind: article, listing, product, or docs. One call is about 0.2 seconds. A video of 30 sites runs in about 14 seconds total.
X post, 21 October 2026, with a follow-up that links the page-classify branch and a Python example. HTTP 200 pages can still be cookie walls, CAPTCHAs, logins, paywalls, error pages, empty searches, or pages still loading. The post prints no per-page accuracy and no Jev version.
TypeSafe AI (@typesafeai)
Published:
Source checked:
[202] TypeSafe on Lightpanda’s cookie-banner judge
TypeSafe quote-tweeted Lightpanda’s judge() post and asked what should get automated away next after cookie banners.
X post, 21 October 2026. Vendor acknowledgement. The post adds no version, score, or latency beyond the quoted video.
lightpanda-io
Source checked:
[203] Lightpanda Python example for page.judge()
The page-classify example calls page.judge() once per URL with a rate-limit Noul and a kind Choice among product, listing, article, docs, search, and other. Built-in wall checks cover consent, login, paywalls, and pages still loading. The script crawls 30 URLs with six sessions, backs off on rate limits, accepts a cookie wall and judges again, and needs TYPESAFE_API_KEY. A comment says judge() is not on PyPI yet.
File on the page-classify branch at commit 7b17601, 21 October 2026. The 30 URLs and the ~0.2 s / ~14 s figures in the X post are the launch demo, not a labelled accuracy table. The example records median and p90 Jev times at runtime and does not print those numbers in the file.
UiPath (@UiPath)
Published:
Source checked:
[204] UiPath announces a TypeSafe connector for Jev
UiPath posted the TypeSafe connector in Integration Service and linked a product blog. The post says the connector brings Jev models into workflows for fast, confident decisions at lower cost and latency.
X post, 21 October 2026. The post itself prints no version, score, or measured latency. The linked blog is the 20 October product note.
TypeSafe AI (@typesafeai)
Published:
Source checked:
[205] TypeSafe on Jev inside UiPath workflows
TypeSafe quote-tweeted UiPath’s connector post and wrote that Jev brings judgment to UiPath workflows about an order of magnitude cheaper and faster.
X post, 21 October 2026. The “order of magnitude” line is TypeSafe’s wording on that post. The UiPath blog’s internal figures are 7.7 times lower cost and 4 to 20 times faster than an equivalent LLM judge.
Sandeep Panda
Published:
Source checked:
[206] Introducing the TypeSafe connector in UiPath Integration Service
The TypeSafe connector is in preview in UiPath Integration Service. A workflow uses the customer’s own TypeSafe API key. Requests do not go through the UiPath AI Trust Layer. UiPath’s internal evaluation reports 94% agreement with an LLM-as-a-judge system, 7.7 times lower cost, and 4 to 20 times faster than that judge.
Product blog dated 20 October 2026. The 94%, 7.7x, and 4x-20x figures are labelled early internal experiments. The post names no Jev version, no public log, and no dataset size. Search terms in Maestro are Jev, TypeSafe, or System One. Choice, Score, and Noul are the three question types. Erik Gafni’s comment on the page is a vendor statement.
Takuma Yoshida (@TakumaYoshidaUJ)
Published:
Source checked:
[207] 100 made-up refund emails: Clairic, Jev, Haiku
Takuma Yoshida reports 100 made-up refund emails scored with Clairic, Jev, and Claude Haiku. Jev plus Python is 78% right at ¥1.5 per 100. Clairic plus Jev is 68 to 77% at ¥4.9 to 8.8, which he says could be his setup. Haiku is 91% at ¥8.1. Keeping only the cases Jev is sure of, then doing the rest himself, moves 78% to 83%.
X post, 26 October 2026, linking the English write-up. The 83% line is the experiment-4 split that leaves high-confidence Jev answers in place and sends the rest to the author. Thresholds on that page were drawn after seeing the 100 results.
Takuma Yoshida
Published:
Source checked:
[208] Clairic and Jev on 100 return emails: four experiments
Four layouts sort 100 fictional Japanese return emails for a kitchenware shop. Claude Haiku 5.5 scores 91 of 100 in one call, at 3.0 seconds and ¥8.1 per 100. The six-step Jev layout without Clairic, experiment 4, scores 78 at 0.3 seconds and ¥1.5, using jev-1.14.0. In that layout Jev’s final choice matched a Python encoding of the nine rules on all 100 cases, so all 22 misses came from reading the email.
English write-up dated 26 October 2026. The emails and order records are fictional, written by Claude after the correct answer was fixed. The author is the one person in the person row. Text sent to Jev and Clairic is Japanese. Clairic is Cloudflare’s preview extraction model @cf/cloudflare/clairic as of 24 October 2026. Confidence thresholds such as 0.8 were drawn after seeing the 100 results. Layouts were chosen after seeing other layouts on the same 100 cases. Haiku’s cost is a list-price estimate inside a subscription. Yen figures use 150 yen to the dollar. UnlockJapan states it has no partnership with TypeSafe or Cloudflare.
Zuplo
Published:
Source checked:
[209] Zuplo AI Gateway adds Jev as a provider
Zuplo’s 20 October 2026 changelog adds Jev as an AI Gateway provider. The catalog lists jev/jev-latest, jev/jev-1.13.0, and jev/jev-preview. Requests go to /v1/systemone under the app URL, priced at TypeSafe’s $0.042 per million input tokens, with output free. Data Loss Prevention, Akamai AI Firewall, and Prompt Injection Protection block System One requests by default as uninspectable.
Changelog dated 20 October 2026. Setting onUnknownShape to skip forwards those requests uninspected. Chat Completions, Responses, Messages, and embeddings refuse Jev models. The default /v1/models list leaves them out unless the request sends x-zuplo-models-endpoint: systemone. The page prints no independent latency or accuracy.
Wang Yang, Xinpeng Li, Yanyan Zhang, Chuang Ma, Shouren Wang, Jianzhe Wei, Liqin Ye, Xiaoran Cai, Mohsen Hariri, Vipin Chaudhary and Xiaotian Han
Published:
Source checked:
[210] What one-pass decision models cannot decide, arXiv:2610.24929
The paper evaluates jev-1.13.0 and 23 open Jev-style configurations. Retrieval and selection are perfect up to 20k tokens and 64 options. Final-state accuracy after sixteen updates of one user falls to 53 to 80%, because similar records of other users interfere. On 738 real items graded by how many thinking tokens Qwen3.8-27B needs, Jev answers 97.5% of the items that one forward pass of that model solves and 74% of the others.
arXiv:2610.24929v1, 18 October 2026. A stale one-line summary is followed even when the log contradicts it. Confidence flags Jev’s own errors (AUROC 0.83 on final-state items) and not errors inherited from a stale input (0.53). An offered hand-off option is taken on 1.6% of the real-task items and on 3 of Jev’s 40 errors. The 738-item set drops HelpSteer2 and 93 items whose thinking samples disagree with the gold label. Qwen3.8-27B is the reasoning reference, not a GPT API model. The paper does not score jev-1.14.0.
Allie (@allietheicon)
Published:
Source checked:
[211] Put Jev everywhere as a smart primitive
Allie writes that after building many dozens of Jev-powered projects, her top advice is to put Jev everywhere. On each new project she asks what Jev could do to make the system cooler, smarter, weirder, or sillier, then tries it. She frames Jev as smart primitives that can slot into code wherever a primitive can live, and as an enabler of a new kind of smart software.
X post, 30 October 2026, by TypeSafe DevRel. The post prints no accuracy table, latency figure, or project count beyond “many dozens.”
knut (@kmelve)
Published:
Source checked:
[212] jevrywhere
knut replies to Allie’s put-Jev-everywhere post with the portmanteau “jevrywhere.”
X reply on Allie’s 30 October 2026 thread. The reply is a nickname, not a measured workflow.
Allie (@allietheicon)
Published:
Source checked:
[213] Agent Tower: decorations, titles, and outfits via Jev
Allie says Agent Tower did not use Jev at first. She later used Jev to set room decorations from thread vibes, to shorten thread titles to the words that fit, and to choose subagent outfits by task type such as artists, coders, and reviewers. The post includes a screenshot of the tower UI.
X reply on the 30 October 2026 put-Jev-everywhere thread. This page has not opened a public Agent Tower repository.
Allie (@allietheicon)
Published:
Source checked:
[214] Jev picks the best substring for Agent Tower titles
Allie writes that Agent Tower thread titles were often too long and used to truncate, and that Jev now picks the best part of the string to keep the meaning. She calls intelligent substring choice one of her favorite small Jev uses.
X post, 18 October 2026, with a screenshot. The post prints no accuracy or latency figure for the title truncations.
Allie (@allietheicon)
Published:
Source checked:
[215] Substring selection as the biggest singular surprise
Allie says her biggest singular surprise was how well Jev works when asked to select the best substring from a larger string. She points at Justin Krup’s jeveryword repository as an illustration she did not write.
X reply on the 30 October 2026 put-Jev-everywhere thread.
Justin Krup
Published:
Source checked:
[216] jeveryword: exact substrings via numbered word choices
jeveryword numbers the words of a text, offers those ids as Jev Choice options, and maps the chosen ids back to exact substrings with character offsets. The README at commit 6fed151 reports ten fields from a 210-character message in one call at 9,234 input tokens and about $0.0004 at list price, and a 56-word PII label pass at about $0.0003. The author marks the project experimental and says the sample is too small for an accuracy claim.
GitHub repository pinned at commit 6fed151 (20 September 2026). Allie linked the project on 30 October 2026 when describing substring selection. List price on the README is $42 per billion input tokens.
Allie (@allietheicon)
Published:
Source checked:
[217] Line-by-line search with Noul or Score
Allie describes line-by-line search: put an entire document in state and ask a question for each line or paragraph. Use Noul when you expect one dominant answer. Use Score when you want to highlight lines by overall relevance.
X post, 27 October 2026, with screenshots. The post prints no accuracy table for the pattern.
Allie (@allietheicon)
Published:
Source checked:
[218] Line search keeps answers in the original text
Allie writes that line-by-line style uses replace lossy LLM summaries with answers that stay sourced from the original content, when the need is the exact spot to look rather than a summary.
X reply on the 27 October 2026 line-by-line thread.
Allie (@allietheicon)
Published:
Source checked:
[219] Noul versus Score inside a semantic-search plugin
Allie prefers Noul for a singular question such as whether a line addresses a refund request, and Score for relevance across a document. In her semantic-search browser plugin, Jev first decides which of those two query types the user typed, then runs that mode.
X reply on the 27 October 2026 line-by-line thread. The plugin is described in the post; this page does not open a repository for it.
Allie (@allietheicon)
Published:
Source checked:
[220] Intelliprompter revived with Jev
Allie says she shelved Intelliprompter because of LLM costs, then revived it the day she first learned about Jev. She points at a TypeSafe cookbook that shows how to build a version of the app.
X post, 27 October 2026. The linked cookbook is the TypeSafe Intelliprompter page.
TypeSafe AI
Source checked:
[221] Intelliprompter cookbook: Score over a live transcript
The cookbook scores a cumulative transcript with one Score question per unchecked talking point. A local threshold of 2.5 out of 5 marks a point covered and drops that question from later requests. The walkthrough hard-codes six transcript chunks. The live app integrates a microphone with ElevenLabs; the cookbook focuses on the TypeSafe calls.
Official cookbook opened 31 October 2026. The 2.5 gate is a local demo threshold, not a TypeSafe confidence field.
Allie (@allietheicon)
Published:
Source checked:
[222] NetworkChuck’s Intelliprompter build
Allie quote-tweets NetworkChuck’s decision-models video and writes that he built his own version of Intelliprompter.
X post, 27 October 2026. The YouTube link is in NetworkChuck’s reply on his own thread.
NetworkChuck (@NetworkChuck)
Published:
Source checked:
[223] Decision models video: Jev, Clef, and Bob
NetworkChuck posts a video introducing decision models and names Jev, Clef, and Bob. A reply on the same thread links the YouTube video at watch?v=Ck7wB7pdebQ.
X post, 27 October 2026. Allie later wrote that the video includes his own Intelliprompter build. This page does not score the video.
ClickHouse (@ClickHouseDB)
Published:
Source checked:
[224] ClickHouse thread on Jev table ranking
ClickHouse writes that it used TypeSafe’s Jev to label every failure on the Agentic Analytics Benchmark and to rank the tables each question needs, reporting 0.951 nDCG@10 in 0.4 seconds.
X post, 27 October 2026, linking the engineering blog. Detailed failure counts, the wide-catalog pilot, and the flat in-loop hint result sit in the article.
Eduardo Vellasques, Lionel Palacin and Al Brown
Published:
Source checked:
[225] Agentic Analytics Benchmark: ranking tables and labeling failures with Jev
On a 136-question development set over an 18-table warehouse, Jev ranks needed tables at mean nDCG@10 of 0.951, recall@10 of 0.991, and about 0.4 seconds per question, with every required table inside the top five for 84% of questions. A four-model development run of 544 attempts yields 277 labeled failures; wrong data selection is the largest bucket at 87 of 277. At a 0.65 threshold on “correct despite fail,” 18 answers look like grader errors. Feeding the ranking back as an in-loop hint leaves accuracy flat across 20 models: pooled net +12 on fired cells, 95% interval −18 to +42.
ClickHouse engineering blog dated 27 October 2026. A companion pilot on 250 public text-to-SQL cases with 75 to 710 tables reaches 87% recall@10 and finds every required table in 73% of cases, at a p90 of 3.4 seconds and about $0.50 for the run. The post does not name a Jev version. Code ships in ClickHouse/data-agent-mnist.
ClickHouse
Source checked:
[226] data-agent-mnist: optional Jev ranking and failure labeling
The harness README’s optional Jev section installs typesafe-sdk and points schema_retrieval.py, schema_retrieval_agent.py, and label_failure_modes.py at a TypeSafe key. Ranking scores each table with a yes/no judgment. The labeler reads a graded run and returns a failure sub-mode, a preventing rule, and a grader-error check. Apache 2.0.
Repository opened at commit 403f3e7 on 2 November 2026. The README says the tools are optional extras and name no fixed warehouse tables.
Levanto Labs (@levantolabs)
Published:
Source checked:
[227] Introducing Sage2: Extract and streaming Decide
Levanto Labs launches Sage2 with Extract (natural language to structured data, claimed 166x cheaper than Opus 5.5 and ahead of Cloudflare Clairic) and Decide (streaming decision model, claimed 3x faster on growing context). Sign-ups in the next 24 hours get 20M free tokens. Also listed on NEAR AI and via MPP with Stripe and Tempo.
X launch post, 2 November 2026, with a product video. Follow-up replies in the same thread point at the model page, playground, MPP guide, and Extract Bench scores. Cited on jevmodel.ai next to TypeSafe’s Jev as another decision-model product.
Levanto Labs (@levantolabs)
Published:
Source checked:
[228] Extract Bench: Sage2 first on accuracy and non-answers
Levanto says it could not find an extraction benchmark, so it built Extract Bench. On that board Sage2 ranked first in accuracy at 77.8 and in identifying non-answers at 72.2, ahead of Cloudflare Clairic at 75.3 and 59. The post invites open-source improvements.
Reply in the Sage2 launch thread, 2 November 2026. Detailed methodology and the full leaderboard sit in the extract-bench repository. Cited next to Jev on the decision-model category pages.
Levanto Labs
Source checked:
[229] Sage2 decision and extraction models
Product page for Sage2 Decide, streaming sessions, and Extract. Demo timings include a refund yes/no at 104.1 ms, three streamed escalation appends at 110.8 / 76.9 / 131.2 ms, and a $450/mo money extract at 182.1 ms. The page claims about 150 ms extraction on short documents, 3x faster decisions on progressive context, streaming as low as 50 ms, and a 16,384-token session cap.
Official model landing page opened 3 November 2026. It lists reasoning, web grounding, vision beta, thresholds, and batch. It does not print a full post-allowance per-token price table in the sections opened for this report. Cited next to Jev on jevmodel.ai.
Levanto Labs
Source checked:
[230] Extract Bench v0.1 leaderboard and methodology
Open eval for typed field extraction over 1,844 documents in 12 public datasets. Sage 2 (rc9 preview, fast) leads v0.1 at 77.8 field accuracy and 72.2 on “not in the document,” ahead of NuExtract3 (76.1 / 57.0) and Clairic (75.3 / 59.0). A footnote states Levanto maintains the benchmark and sells Sage 2. Abstention thresholds for Sage 2 were fit on the held-out half.
Repository opened at commit 5fb31b8 on 3 November 2026. Code is Apache-2.0. Predictions ship under results/v0.1 so extract-bench report can reproduce the board offline. Cited next to Jev; the board does not score Jev.
Akshay Pachaar (@akshay_pachaar)
Published:
Source checked:
[231] Sage2 as a streaming session instead of a full-context rebill
Akshay writes that a decision model which recomputes the full chat on every check would process early messages repeatedly in a long thread, and that Sage2 keeps a server-side session, appends new text, reuses cached prefix computation, and re-evaluates registered questions. He repeats Levanto’s 3x progressive-context claim, places responses in tens of ms, and notes Levanto launched its first decision model before Jev.
X post, 2 November 2026, with a car-dealership demo video. The author says he partnered with Levanto on the write-up. The post names Jev when comparing full-context rebilling to Sage2 sessions.
NORMIES (@normiesART)
Published:
Source checked:
[232] Normies Arena: 15M+ Sage decisions and a move to Sage2
Normies writes that its agents made over 15 million decisions on Sage in about a month, that Normies Arena is a fully autonomous agent game played 24/7, and that the team recently switched to Sage2. For a real-time game it was looking for 150–200 ms response time.
X quote-tweet of the Sage2 launch, 2 November 2026. The decision count and latency target are the author’s figures. Cited on jevmodel.ai next to Jev usage reports.
Perp Flow (@PerpFlow)
Published:
Source checked:
[233] Perp Flow: 500K+ Sage decisions in a month
Perp Flow writes that it has used Sage as an auto trader for weekly trade decisions and to organise a database and find patterns, reporting more than 500,000 decisions in the past month while reducing AI usage.
X quote-tweet of the Sage2 launch, 2 November 2026. Levanto replied that decision models were still not ideal for execution management and can unlock new speeds of quantitative analysis. Cited next to Jev on jevmodel.ai.
The Pond Street Ledger
Published:
Source checked:
[234] Sage2 launch with Normies, Perp Flow, and GrowBlocks
Launch write-up that paraphrases Levanto’s Extract and Decide claims and collects Normies’ 15M+ and Perp Flow’s 500K+ figures. For GrowBlocks City it records launch-day counters of 9,624 decisions that day and 24,478 in November so far, average 406 ms, and resident calls at 83–136 ms on the live feed.
Article dated 2 November 2026. Disclosure: the Ledger’s publisher founded GrowBlocks, and GrowBlocks had early access to Sage2. User volumes remain self-reported. Cited next to Jev on the decision-model category pages.