Moonshot AI · Community reports
Jev + Kimi: an email handoff, SemEval, and an unfinished 231-task run
A measured email handoff to Kimi K3, a SemEval chart against Kimi-K2 Thinking, one snapshot where a Kimi K3 alias returned 139 valid answers of 231, and a hook log that named kimi-k3 once.
Compiled by jevmodel.ai. We read the cited posts and documentation. The measurements stay with their authors. We have not rerun them.
Sending uncertain emails to Kimi K3
Hassan tests 100 emails, split evenly between legitimate and fraudulent examples. Jev performs the first classification pass in 1.42 seconds. The workflow sends 31 cases below a 95% confidence threshold to Kimi K3 for further analysis.
The reported full pipeline takes 16 seconds and gets 96 of 100 emails correct. The stated cost is about $0.07, with $0.003 attributed to Jev and $0.068 to Kimi. The 1.42-second number covers only the first stage. The combined workflow’s duration includes the Kimi work.
Sources:[19] Hassan
The 95% cutoff belongs to this demo
The post reports a live demonstration on one split of emails. Its 96 correct decisions still leave four errors. An even split of fraudulent and legitimate mail leaves open how the same cutoff would do in an inbox with a different mix.
TypeSafe recommends selecting confidence thresholds for the domain and the consequences of an action. The 95% boundary in Hassan’s demo describes his setup. Using it for a financial or account decision would go past what the post tested.
Sources:[19] Hassan[2] TypeSafe AI
Kimi can help build the application without running its classifier
yonsakhan describes pair-programming with Kimi K3 to build a userscript for X comments. The post says Jev handles semantic spam checks, including obfuscated text, while the script collapses matches, lets users recover mistakes and caches results locally.
This is a different meaning of using Jev with Kimi. In Hassan’s report, Kimi receives uncertain emails at runtime. In the comment-filter report, Kimi helped develop the software. The post attributes the classifications to Jev.
Sources:[20] yonsakhan[19] Hassan
Ten sentiment test sets against Kimi-K2 Thinking
Eastwood ran Jev 1.13.0 on all ten SemEval-2026 DimABSA subtask 1 test sets. The aspect is already given. The job is to score how positive or negative that aspect is, and how activated the emotion is. The chart uses the official RMSE_VA metric, where a lower number is a smaller error, and the caption says the figures skip normalization. Jev was left untuned. Its equal-weight mean is 2.4750. The official Kimi-K2 Thinking one-shot result is 1.8731. A QLoRA fine-tune of Qwen3-14B is 2.1889.
Against Kimi that is 0 wins and 10 losses. Against the Qwen fine-tune it is 3 wins and 7 losses. The mean error is 32.1% higher than Kimi’s. The caption says training and example conditions differ across methods. The per-dataset chart shows Jev’s error above Kimi’s on every set. Kimi-K2 Thinking is a different model from the Kimi K3 in Hassan’s email demo.
Sources:[45] Eastwood
Kimi K3 on an unfinished 231-task snapshot
On the Jev Cookbook snapshot of 231 planned public JevBench tasks, dated 26 September 2026, the Kimi K3 alias is the only one of the five with a recorded row for every planned task. It returns 139 valid answers and 139 correct out of 231, with 92 errors. Eighty-nine of those errors are empty probability parses. p50 and p95 are 9.63 and 15.60 seconds. Recorded spend is $0.011435. On the hard 111 it has 29 valid answers, against 1 for the Jev alias. Easy and original valid counts are 47 of 48 and 63 of 72.
The Jev alias in the same file has 120 valid answers and 116 correct, then leaves 110 tasks unattempted after the run exceeds its reservation. Conditional accuracy on those scored answers is 116 of 120. Its p50 is 0.77 seconds and its recorded spend is $0.001986. The file’s composite is higher for Jev than for Kimi, and the chapter says that score describes this incomplete run. The aliases do not carry a full version string. This Kimi K3 name matches Hassan’s email fallback and differs from the Kimi-K2 Thinking scores in the SemEval chart. The finished public-231 audit, Jev 1.13.0 at 200 of 231, is another run.
Sources:[93] wang yiming[94] Datawhale[92] Zefan Cai[19] Hassan[45] Eastwood
A hook log that named kimi-k3 once
Christoph Görn’s Claude Code hook called OpenRouter’s typesafe/jev-router and asked for a JSON size label. On the first 16 prompts the answered_by field names moonshotai/kimi-k3 once, openai/gpt-6-luna five times, and deepseek/deepseek-v4.1-flash three times. Seven calls returned no JSON. He writes that the alias did not answer as TypeSafe’s Jev. The latency and the 8-prompt scores are on the Claude report. This kimi-k3 string is the id in that log. It is separate from Hassan’s Kimi K3 email fallback and from the Kimi-K2 Thinking SemEval scores.
Questions about Jev + Kimi
Did the Jev + Kimi pipeline finish in 1.42 seconds?
No. Hassan reports 1.42 seconds for Jev’s first pass and 16 seconds for the complete pipeline including Kimi K3.
Sources:[19] Hassan
Does the Kimi-assisted comment filter call Kimi at runtime?
The cited post says Kimi K3 helped develop the userscript and Jev performs the semantic checks. A runtime Kimi call is outside what the post describes.
Sources:[20] yonsakhan
Is the SemEval comparison the same Kimi as the email demo?
Hassan names Kimi K3 as the runtime fallback. Eastwood compares untuned Jev 1.13.0 with published Kimi-K2 Thinking one-shot scores on SemEval-2026 DimABSA. The mean RMSE_VA values are 2.4750 and 1.8731, and lower is better.
Sources:[19] Hassan[45] Eastwood
Did Kimi K3 beat Jev on the 231-task snapshot?
The Kimi K3 alias returned 139 valid answers and 139 correct of 231. The Jev alias returned 120 valid and 116 correct, then left 110 tasks unattempted. Conditional accuracy on the scored Jev answers is 116 of 120. The aliases do not record a full model version, and this run is separate from Jev 1.13.0 scoring 200 of 231 on the finished public audit.
Sources:[94] Datawhale[92] Zefan Cai
Did a Claude Code hook call Kimi through a Jev alias?
Christoph Görn’s log names moonshotai/kimi-k3 once among the first 16 prompts sent to OpenRouter’s typesafe/jev-router. GPT-6 Luna is named five times and DeepSeek V4.1 Flash three times. Seven calls returned no JSON. He writes that the alias did not answer as TypeSafe’s Jev.
Sources:[99] Christoph Görn
Sources and reading notes
The summaries below are paraphrases. Each link opens the original source, including its surrounding context.
Hassan (@nutlope)
Published:
Source checked:
[19] Jev first-pass email screening with Kimi K3 fallback
Hassan tests 50 legitimate and 50 fraudulent emails. Jev classifies them in 1.42 seconds; 31 cases below a 95% confidence threshold go to Kimi K3. The full pipeline takes 16 seconds, gets 96 of 100 correct and costs about $0.07.
Author-reported demonstration, not independently rerun here. The stated cost split is $0.003 for Jev and $0.068 for Kimi. The 1.42-second figure excludes the Kimi stage.
TypeSafe AI
Source checked:
[2] Confidence, probabilities and fallback decisions
Choice and Score include confidence derived from their probability distributions. Noul has no separate confidence field. TypeSafe recommends choosing thresholds for the domain and consequences of an action.
Implementation reference. A high confidence value can still sit on a wrong decision.
yonsakhan (@yonsakhan)
Published:
Source checked:
[20] Using Kimi K3 to develop a Jev-powered comment filter
The author says they developed an X comment-filtering userscript with Kimi K3 and use Jev for semantic spam decisions, including obfuscated text. They describe reversible collapsing and a local cache.
A development report. Kimi helped write the software. Runtime classification is attributed to Jev.
Eastwood (@Tsj_estwld)
Published:
Source checked:
[45] SemEval-2026 DimABSA scores for Jev, Kimi-K2 Thinking and Qwen
Eastwood reports Jev 1.13.0 on all ten SemEval-2026 DimABSA subtask 1 test sets. Equal-weight mean RMSE_VA is 2.4750 for untuned Jev, 2.1889 for a Qwen3-14B QLoRA fine-tune, and 1.8731 for the official Kimi-K2 Thinking one-shot result. That is 0 wins and 10 losses against Kimi, and 3 wins and 7 losses against Qwen. Mean error is 32.1% higher than Kimi’s.
Lower RMSE_VA is better. The aspect is given, and the model scores polarity and activation. The chart caption says results use the official test metric without normalization, and that training and example conditions differ across methods. Kimi-K2 Thinking and the Kimi K3 in Hassan’s email demo are different models. The per-dataset chart shows Jev higher than Kimi on every set.
wang yiming (@wangyiming34110)
Published:
Source checked:
[93] A five-model snapshot of 231 planning tasks
The post points to chapter 6 of the Jev Cookbook and says that, on one snapshot of 231 planning tasks, Jev produced 120 valid answers at a p50 of 0.77 seconds, while Kimi produced 139 valid answers, the most of the five models.
The post gives valid-answer counts and one latency figure. Stop reasons, correct-answer counts and version strings are in the cookbook chapter at commit 546050a.
Datawhale
Published:
Source checked:
[94] Jev Cookbook chapter 6 snapshot for 26 September 2026
The chapter records one run of five aliases on 231 planned public tasks. Kimi K3 has 139 valid answers and 139 correct. Jev has 120 valid and 116 correct, then leaves 110 tasks unattempted after a reservation stop. DeepSeek Flash has 110 valid and 110 correct, then stops after consecutive errors. Conditional accuracy uses only scorable answers.
Read at commit 546050a. snapshot.json in the same commit matches the counts. Recorded spend is $0.001986 for Jev, $0.011435 for Kimi K3 and $0.020312 for DeepSeek Flash. p50 latency is 0.77, 9.63 and 1.40 seconds. The aliases do not include a full version string. Hard-tier valid answers are 1, 29 and 2 of 111. The stored composite is highest for Jev, and the chapter says that score describes this incomplete run. Raw prompts and provider responses are not in the repository. This run is separate from the completed 231-task audit in the Open-Jev note.
Zefan Cai
Published:
Source checked:
[92] Public JevBench note at Open-Jev commit 3308a15
The note scores six streams on all 231 available public tasks. Jev 1.13.0 is correct on 200. GPT-5.6 Luna, reasoning effort none, is correct on 206. GPT-6 Astra, reasoning effort low, is correct on 231. Easy tasks are 48 of 48 for each of those three. On the original 72 the counts are 71, 69 and 72. On the hard 111 they are 81, 89 and 111.
Read at commit 3308a15. The other 303 of 534 tasks were unavailable, and the note reports no 534-task score. Open-Jev and Jev return native probabilities. GPT returns verbalized probabilities in JSON. Candidate order differs on 119 of 139 Choice tasks. One Jev vector was renormalized. P50 full-response times, one observation per task, are 291.3 ms for Jev, 953.8 ms for Luna and 2,206.4 ms for Astra. The note says that sample is not a repeated latency test. Tariff estimates are $0.03842, $0.17914 and $8.81978 per 1,000 decisions, and the note says they are not billing receipts. Brier means are 0.1811, 0.2074 and 0.0085. Top-label ECE is 0.0318, 0.0932 and 0.0149. Open-Jev 27B v1.1 is 197 of 231, and its direct-execution timings are left out of the latency table. Upstream tasks are pinned at fstandhartinger/jevbench commit f8ce713.
Christoph Görn (@goern)
Published:
Source checked:
[98] A Claude Code router timed against local Kev
Christoph Görn says a local Kev-4B router on an RTX 4090 was 10 to 15 times faster than the cloud sizer he called Jev, 100 to 180 ms against 700 to 1400 ms, and that Kev’s accuracy was 14%. The post links his write-up.
The 14% figure is Kev’s score in the write-up, on 8 prompts. The cloud side of that table is OpenRouter’s typesafe/jev-router alias, and the write-up says the log named other models. The 16-prompt latency average is a different sample.
Christoph Görn
Published:
Source checked:
[99] What answered the Claude Code sizing hook
The hook calls OpenRouter’s typesafe/jev-router and asks for a JSON size label. On the first 16 prompts the log counts 3 tiny, 1 everyday, 1 large, 4 left on the main session, and 7 errors. The answered_by field names gpt-6-luna five times, deepseek-v4.1-flash three times, and kimi-k3 once.
Read on 28 September 2026. The 9 successful calls average 2.99 seconds, from 1.26 to 4.41, and sizing cost about $0.0032. A separate 8-prompt table scores Kev-4B at 1 of 7 and the alias at 1 of 5, with latencies of 100 to 180 ms and 700 to 1400 ms. The author says the router code is in a private infra repository and that the router log has no successful Kev decision yet.