Jev is cheap.Is it good enough?

Independent evidence for Jev.

Benchmarks · Cost · Accuracy · Fallback

Explore the evidence →Calculate fallback economics

Faster
Cheaper
Structured

But

Accurate
enough?

Where Jev wins

0 ms

Median latency

vs 687 ms · Claude Haiku 4.5

$0.000

Measured workload cost, per 1,000

vs $0.462

Where Jev loses

0.0%

Accuracy

vs 81.3% · Claude Haiku 4.5

Different tasks.
Different outcomes.

Across the runs recorded here Jev ranges from 62.6% to 95.4%. There is no universal Jev accuracy, and our schema has nowhere to store one.

PhishNChips v5.2 · Label an email as phishing or legitimate. · n=2,000 · evidence

Same model, same task, different prompt.

85.7%

Phishing recall, text-only prompt

98.4%

Same set, same model, enriched evidence

43.2%

A different set, text-only protocol

All three are Jev detecting phishing. The gap between the first two is nothing but the prompt, on the same 5,733-email set. A recall figure quoted without its protocol is not a fact about the model — which is the whole reason every number here travels with the run that produced it. See the run.

The author is explicit that the enriched criteria were informed by labelled errors, so part of that gain is supervision a cold-start deployment would not have.

Cost × accuracy

The trade-off, on one dataset where both were measured.

Scoped to a single run deliberately. A cost–accuracy plane implies the points are alternatives to one another, and that is only true when they were measured on the same data under the same protocol.

Hover or tap a point for its full record.

Recent benchmark runs

Verified runs. No marketing claims.

PhishNChips v5.2

Label an email as phishing or legitimate.

n = 2,000 · 2026-09-17

Jev jev-1.13.0	62.6%
Claude Haiku 4.5	81.3%

Result

Recorded — see limitations

Banking77 (test split)

Assign a customer banking utterance to one of 77 intent labels, zero-shot.

n = 208 · 2026-09-18

Jev	83.2%
GPT-5.4 nano gpt-5.4-nano-2026-03-17	79.3%
GPT-5.6 Terra	87.5%
BGE-small-en-v1.5 + logistic regression	93.3%

Result

AMBIGUOUS

CLINC150 (test + OOS test splits)

Assign an utterance to one of 150 intents or to an out-of-scope bucket, zero-shot.

n = 200 · 2026-09-18

Jev	87.0%
GPT-5.4 nano gpt-5.4-nano-2026-03-17	79.5%
GPT-5.6 Terra	91.5%

Result

AMBIGUOUS

Hand-labelled tool-call set (author's own)

Classify an agent tool call as readonly, destructive, privileged or exfiltration.

n = 60 · 2026-09-17

Jev jev-latest	91.7%
Jev jev-preview	91.7%

Result

Recorded — see limitations

Main paired set (author's own assembly)

Label an email as legitimate, spam or phishing.

n = 5,733 ·

Jev jev-1.13.0, text-only	93.6%
Jev jev-1.13.0, enriched evidence	98.0%
Jev jev-1.13.0, enriched + evidence wording	98.6%
TF-IDF + logistic regression enriched	98.9%
Jev + TF-IDF ensemble	99.3%

Result

Recorded — see limitations

Fresh set (author's own assembly)

Label an email as legitimate, spam or phishing, on a set held out from the main run.

n = 3,300 ·

Jev jev-1.13.0, enriched evidence	95.4%
TF-IDF + logistic regression enriched	96.3%

Result

Recorded — see limitations

Recent phishing set (author's own assembly)

Detect phishing in messages drawn from 2024–25, newer than the main training and evaluation material.

n = 853 ·

Result

Recorded — see limitations

Seven synthetic workloads (author's own)

Drive seven simulated application workloads — tickets, routing, driving, guardrails, approvals, scoring and home commands — validating every output before applying a simulated action.

n = 424 · 2026-09-17

Result

Recorded — see limitations

20 Newsgroups (three topic probes)

Rank rows by a Jev probability and test whether the resulting order is defensible, through calibration, inversion and invariant checks.

n = 360 · 2026-09-18

Result

PASS — all six gates

Amazon ESCI (hard probe)

Rank query–product pairs by a Jev probability against human relevance grades, using the same pre-registered gates as the easy corpus.

n = 306 · 2026-09-18

Result

FAIL — four of six gates

Banking77 (routing sample)

Answer with Jev when its confidence clears a threshold and escalate to a larger model otherwise, then measure the cascade end to end.

n = 500 ·

Jev → DeepSeek v4 Pro cascade

80.2%

Result

ROUTE — threshold 0.67

Web of Science (routing sample)

The same confidence-routing pipeline, applied to a second labelled dataset.

n = 500 ·

Result

DO NOT ROUTE

Evidence stream

09.19Jev pricing verified at OpenRouter
09.19Claude Haiku 4.5 pricing verified at Anthropic (first-party API)
09.19Source verified · OpenRouter
09.19Source verified · Anthropic
09.19Source verified · Latent Space (AINews)
09.19Source verified · anisselbd
09.19Source verified · ickma2311
09.19Source verified · themsquared

Fallback economics

How much quality can you recover before the saving is gone?

A Jev call on every request, escalating to Claude Haiku 4.5 on the ones it is not confident about. The break-even fallback rate is where that cascade stops being cheaper than running the baseline on everything.

Model your own workload →

0.0%

Break-even fallback rate

Jev can fall back on up to 99.8% of requests before this hybrid path costs more than baseline-only.

Jev only$12.60/mo

Hybrid at 20%$1,023/mo

Baseline only$5,050/mo

1M requests/month · Jev 300 input tokens · Claude Haiku 4.5 500 input + 910 output · verified list pricing both sides

Extraordinary claims need independent evidence.

Verified benchmarks, plus 2 routing evaluations — 4 of them pre-registered

Independent sources behind them

Independently reproduced — a public repository is not a reproduction, and we check

Read the methodology →

What this site will not do

No universal Jev score, no overall ranking, and no comparison page without a direct head-to-head. Where a number is missing it stays missing — 12 sources are recorded, and the ones we have not read yet sit in an intake queue rather than on a page.