AI model benchmarks

• Jev alternatives

  OpenAI Decisions API beats Jev on real-world checks

  1. OpenAI Decisions API75.2%±3.3
  2. Mercury Decide70.5%±3.6
  3. Kdeck-31B66.8%±3.3
  4. JKev-27B64.8%±3.8
  5. pplx-decider v1.1 27B64.7%±3.6

• LLM-as-a-Judge Pareto Frontier

  Frontier LLMs are up to 25 points more accurate than Jev, but cost over 100x more

  1. Claude Opus 5.589.2%±2.9
  2. GPT-6.1 Sol87.8%±2.8
  3. Claude Sonnet 5.584.6%±3.0
  4. Gemini 3.8 Flash83.1%±2.8
  5. GPT-6 Luna79.2%±3.2

• PII detection

  Gemma 4 31B leaves the least PII in on the LangWatch policy test

  1. Gemma 4 31B6.9%±4.3
  2. Ministral 3 14B10.2%±4.8
  3. Gemma 4 12B16.5%±6.5
  4. Qwen3.8 27B16.7%±7.6
  5. Gemma 4 26B A4B20.5%±7.7

• Jev vs frontier LLMs

  Frontier LLMs beat Jev on 9 of 11 tasks

  1. Claude Opus 5.584.5%±0.7
  2. Gemini 3.8 Flash84.0%±0.7
  3. GPT-6.1 Sol83.4%±0.7
  4. Claude Sonnet 5.582.8%±0.7
  5. Jev81.0%±0.8

• Jev vs open models up to 28B

  Open models caught up with Jev

  1. Jev81.9%±0.8
  2. CEikos-27B*81.6%±0.8
  3. CAutoJev-27B*81.2%±0.9
  4. JKev-9B*75.1%±3.1
  5. JKev-4B*69.4%±0.8

• Jev vs models that run on your laptop

  Jev beats every tiny open model

  1. Jev79.6%±0.7
  2. JKev-0.8B*55.9%±1.2
  3. JKev-0.6B*48.3%±1.1
  4. CLaya-typed*47.0%±1.1

Each bar is a score with its 95% interval. Orange: ahead alone. Green: tied for the lead. Gray: the rest. Hatched with *: trained on the test data, so not ranked. Average: the unweighted mean of a model's task scores; tasks use different metrics, so read it as a summary, not a ranking.