LangWatch
Sign Up

convaiinnovationsOpen weights model

Laya benchmarks

All models
  • General rank
    #22of 28Leads 0 of 26 (0%)non-coding tasks
  • Speed#2 / 8
    50 msMedian latency, p50Laptop GPU only
  • Cost
    Self-hostedRan on Laptop GPUNo hosted price
  • Search relevance#2 / 8 tied
    31.5%Best task · Accuracyvs small open models

Comparison summary

Strongest
General decisions
Leads 0 of 4 (0%)
Weakest
Agents
Leads 0 of 2 (0%)

A typed decision model: it answers a typed question, such as yes or no or one of a few options, with a probability for each answer and no text. Strongest at general decisions, weaker at agent actions.

Specifications

Lab
convaiinnovations
Weights
Open weights
Parameters
Not disclosed
Licence
Not recorded

Overall standing

Share of non-coding tasks led · #22 of 28

  1. Claude Opus 5.585%
  2. OpenAI Decisions API67%
  3. Jev61%
  4. Eikos-27B60%
  5. GPT-6.1 Sol54%
  6. AutoJev-27B53%
  7. Claude Sonnet 5.546%
  8. Laya
    0%

Safety and guardrails

Share of tasks led · #9 of 13

  1. Eikos-27BLeads 3 of 3
  2. AutoJev-27BLeads 2 of 2
  3. JevLeads 8 of 12
  4. Shisa DE-1Leads 1 of 4
  5. Kev-9BLeads 0 of 4
  6. Eikos-4BLeads 0 of 3
  7. SemIf (Qwen3.5-4B)Leads 0 of 4
  8. Laya
    Leads 0 of 6
See the 4 tasks

Prompt injection

Not ranked

Catch rate at 5% false alarms, % · Higher is better

  1. Jev94.6, interval 92.2 to 96.7, tied for the lead
  2. Eikos-27B94.2, interval 90.5 to 97.1, tied for the lead
  3. Shisa DE-194.0, interval 90.3 to 96.2, tied for the lead
  4. AutoJev-27B92.0, interval 86.6 to 95.0, tied for the lead
  5. Kev-4B91.6, interval 88.2 to 94.2, tier 2
  6. Eikos-4B90.8, interval 88.3 to 94.0, tier 2
  7. SemIf (Qwen3.5-4B)90.6, interval 87.8 to 93.5, tier 2
  8. Laya
    0.0, interval 0.0 to 0.0, not ranked
Tested against Jev and 14 open modelsAlso tested against small open models: not ranked

Moderation

Tier 5 of 6#9 / 12

AUROC, % · Higher is better

  1. AutoJev-27B*90.6*, interval 88.4 to 92.7, trained on this data, not ranked
  2. Eikos-27B90.3, interval 87.9 to 92.4, tied for the lead
  3. Jev90.3, interval 87.9 to 92.4, tied for the lead
  4. Eikos-4B88.8, interval 86.3 to 91.3, tier 2
  5. SemIf (Qwen3.5-4B)88.7, interval 86.2 to 91.2, tier 2
  6. Kev-9B88.5, interval 86.0 to 90.9, tier 2
  7. Shisa DE-188.3, interval 85.6 to 90.8, tier 2
  8. Laya
    69.8, interval 65.8 to 73.6, tier 5
Tested against Jev and 14 open modelsAlso tested against small open models: #3 of 5 tied

Off-topic

Tier 8 of 8#8 / 9

Balanced accuracy, % · Higher is better

  1. Jev93.4, interval 91.8 to 94.9, ahead alone
  2. AutoJev-27B*92.7*, interval 91.0 to 94.4, trained on this data, not ranked
  3. Eikos-27B*90.6*, interval 88.8 to 92.4, trained on this data, not ranked
  4. Kev-9B89.6, interval 87.7 to 91.5, tier 2
  5. Shisa DE-188.0, interval 86.1 to 89.9, tier 3
  6. Kev-4B84.4, interval 82.1 to 86.7, tier 4
  7. SemIf (Qwen3.5-4B)81.1, interval 79.0 to 83.4, tier 5
  8. Laya
    54.3, interval 51.4 to 57.1, tier 8
Tested against Jev and 14 open modelsAlso tested against small open models: #4 of 5 tied

PII

Tier 5 of 5#8 / 13

Catch rate at 5% false alarms, % · Higher is better

  1. Eikos-27B95.2, interval 92.8 to 97.8, tied for the lead
  2. AutoJev-27B92.8, interval 90.2 to 95.5, tied for the lead
  3. Jev90.8, interval 88.1 to 93.8, tier 2
  4. Shisa DE-184.2, interval 77.3 to 89.1, tier 3
  5. Eikos-4B78.6, interval 74.5 to 85.3, tier 3
  6. SemIf (Qwen3.5-4B)66.1, interval 60.9 to 74.6, tier 4
  7. Kev-9B62.5, interval 0.2 to 66.4, tier 4
  8. Laya
    15.8, interval 0.0 to 23.0, tier 5
Tested against Jev and 14 open modelsAlso tested against small open models: #2 of 5 tied

Routing and classification

Share of tasks led · #11 of 16

  1. AutoJev-27BLeads 4 of 5
  2. Shisa DE-1Leads 3 of 4
  3. JevLeads 9 of 14
  4. Eikos-27BLeads 1 of 3
  5. SemIf (Qwen3.5-4B)Leads 1 of 3
  6. Kev-4BLeads 0 of 3
  7. Kev-9BLeads 0 of 2
  8. Laya
    Leads 0 of 10
See the 5 tasks

Routing, 20 intents

Tier 4 of 5#5 / 7

Accuracy, % · Higher is better

  1. Kev-4B*93.0*, interval 91.4 to 94.5, trained on this data, not ranked
  2. Kev-9B*93.0*, interval 91.4 to 94.6, trained on this data, not ranked
  3. Kev-0.8B*91.3*, interval 89.5 to 93.0, trained on this data, not ranked
  4. AutoJev-27B89.7, interval 87.7 to 91.6, ahead alone
  5. Eikos-27B*89.6*, interval 87.7 to 91.4, trained on this data, not ranked
  6. Kev-0.6B*89.5*, interval 87.5 to 91.5, trained on this data, not ranked
  7. Jev89.1, interval 87.0 to 91.0, tier 2
  8. Laya
    76.0, interval 73.3 to 78.6, tier 4
Tested against Jev and 14 open modelsAlso tested against small open models: #2 of 4 tied

Routing, 77 intents

Tier 3 of 3#4 / 5

Accuracy, % · Higher is better

  1. Kev-4B*85.0*, interval 81.8 to 88.0, trained on this data, not ranked
  2. Kev-9B*85.0*, interval 81.8 to 88.2, trained on this data, not ranked
  3. Kev-0.8B*83.0*, interval 79.6 to 86.4, trained on this data, not ranked
  4. AutoJev-27B80.6, interval 77.0 to 84.0, tied for the lead
  5. Jev79.6, interval 76.0 to 83.0, tied for the lead
  6. Kev-0.6B*78.2*, interval 74.4 to 81.8, trained on this data, not ranked
  7. Eikos-27B*77.8*, interval 74.0 to 81.4, trained on this data, not ranked
  8. Laya
    42.4, interval 38.2 to 46.8, tier 3
Tested against Jev and 14 open modelsAlso tested against small open models: #2 of 3 tied

Tool routing

Tier 9 of 10#14 / 16

Accuracy, % · Higher is better

  1. Shisa DE-181.3, interval 78.9 to 83.5, tied for the lead
  2. AutoJev-27B80.6, interval 78.1 to 82.9, tied for the lead
  3. SemIf (Qwen3.5-4B)79.9, interval 77.4 to 82.2, tied for the lead
  4. Eikos-27B79.3, interval 76.8 to 81.8, tier 2
  5. Jev78.3, interval 75.7 to 80.7, tier 2
  6. Eikos-4B75.5, interval 72.8 to 78.0, tier 3
  7. Kev-9B72.4, interval 69.7 to 75.0, tier 4
  8. Laya
    22.9, interval 20.2 to 25.3, tier 9
Tested against Jev and 14 open modelsAlso tested against small open models: #6 of 8 tied

Complaint routing

Tier 7 of 8#13 / 15

Accuracy, % · Higher is better

  1. Jev78.7, interval 76.3 to 81.2, tied for the lead
  2. Shisa DE-178.0, interval 75.4 to 80.6, tied for the lead
  3. Kev-4B76.2, interval 73.6 to 79.0, tier 2
  4. AutoJev-27B75.9, interval 73.2 to 78.7, tier 2
  5. Eikos-27B75.6, interval 73.0 to 78.4, tier 2
  6. Kev-9B75.0, interval 72.4 to 77.7, tier 2
  7. SemIf (Qwen3.5-4B)72.9, interval 70.2 to 75.7, tier 3
  8. Laya
    44.2, interval 41.3 to 47.2, tier 7
Tested against Jev and 13 open modelsAlso tested against small open models: #6 of 8

Typed decisions

Tier 6 of 7#11 / 14

Accuracy, % · Higher is better

  1. Laya-typed*77.4*, interval 75.4 to 79.4, trained on this data, not ranked
  2. Shisa DE-175.1, interval 72.8 to 77.3, tied for the lead
  3. Jev73.9, interval 71.8 to 76.0, tied for the lead
  4. AutoJev-27B73.8, interval 71.7 to 75.9, tied for the lead
  5. Eikos-27B73.4, interval 71.2 to 75.6, tied for the lead
  6. Kev-4B65.8, interval 63.6 to 68.0, tier 2
  7. Eikos-4B65.3, interval 62.7 to 68.0, tier 2
  8. Laya
    36.1, interval 33.6 to 38.6, tier 6
Tested against Jev and 13 open modelsAlso tested against small open models: #4 of 7 tied

Grounding

Share of tasks led · #10 of 15

  1. Eikos-27BLeads 2 of 2
  2. JevLeads 4 of 6
  3. AutoJev-27BLeads 0 of 2
  4. Shisa DE-1Leads 0 of 2
  5. Eikos-4BLeads 0 of 2
  6. Kev-4BLeads 0 of 2
  7. Kev-9BLeads 0 of 2
  8. Laya
    Leads 0 of 4
See the 2 tasks

RAG faithfulness

Tier 5 of 5#11 / 13

Balanced accuracy, % · Higher is better

  1. Eikos-27B82.1, interval 79.2 to 84.9, tied for the lead
  2. Jev80.3, interval 77.5 to 83.1, tied for the lead
  3. Shisa DE-179.2, interval 76.2 to 82.2, tier 2
  4. AutoJev-27B79.1, interval 76.1 to 82.0, tier 2
  5. Kev-9B73.1, interval 70.0 to 76.0, tier 3
  6. Kev-4B72.5, interval 69.4 to 75.5, tier 3
  7. SemIf (Qwen3.5-4B)72.0, interval 68.9 to 75.2, tier 3
  8. Laya
    49.8, interval 48.6 to 51.0, tier 5
Tested against Jev and 14 open modelsAlso tested against small open models: #5 of 6 tied

Search relevance

Tier 5 of 7#9 / 16

Accuracy, % · Higher is better

  1. Jev57.7, interval 54.7 to 60.8, tied for the lead
  2. Eikos-27B56.6, interval 53.4 to 59.7, tied for the lead
  3. AutoJev-27B52.8, interval 49.6 to 55.9, tier 2
  4. Shisa DE-150.3, interval 47.1 to 53.6, tier 2
  5. Kev-4B44.8, interval 41.7 to 48.0, tier 3
  6. Kev-9B43.4, interval 40.2 to 46.5, tier 3
  7. Eikos-4B43.3, interval 40.1 to 46.3, tier 3
  8. Laya
    31.5, interval 28.7 to 34.3, tier 5
Tested against Jev and 14 open modelsAlso tested against small open models: #2 of 8 tied

Agents

Share of tasks led · #5 of 8

  1. JevLeads 2 of 2
  2. SimpleJev (Qwen3.5-0.8B)Leads 0 of 2
  3. Kev-0.8BLeads 0 of 2
  4. Kev-0.6BLeads 0 of 2
  5. Laya
    Leads 0 of 2
  6. Laya-typedLeads 0 of 2
  7. openJev Verdict 1.4Leads 0 of 2
  8. SemIf (Qwen3-0.6B)Leads 0 of 2
See the task

Web-agent actions

Tier 10 of 10#12 / 15

Accuracy, % · Higher is better

  1. Jev70.8, interval 68.9 to 72.6, ahead alone
  2. Eikos-27B68.4, interval 66.5 to 70.4, tier 2
  3. AutoJev-27B68.2, interval 66.2 to 70.2, tier 2
  4. Shisa DE-165.2, interval 63.3 to 67.2, tier 3
  5. Eikos-4B62.9, interval 60.9 to 64.8, tier 4
  6. SemIf (Qwen3.5-4B)59.5, interval 57.5 to 61.5, tier 5
  7. Kev-4B58.0, interval 55.9 to 60.2, tier 5
  8. Laya
    17.8, interval 16.3 to 19.4, tier 10
Tested against Jev and 13 open modelsAlso tested against small open models: #5 of 8 tied

General decisions

Share of tasks led · #5 of 10

  1. JevLeads 4 of 4
  2. AutoJev-27BLeads 2 of 2
  3. Kev-0.6BLeads 0 of 4
  4. Kev-0.8BLeads 0 of 4
  5. Laya
    Leads 0 of 4
  6. Laya-typedLeads 0 of 4
  7. SimpleJev (Qwen3.5-0.8B)Leads 0 of 2
  8. Kev-4BLeads 0 of 2
See the 2 tasks

Community sets

Tier 4 of 5#6 / 10

Accuracy, % · Higher is better

  1. AutoJev-27B63.0, interval 60.2 to 65.8, tied for the lead
  2. Jev62.3, interval 59.5 to 65.1, tied for the lead
  3. Eikos-27B59.8, interval 56.9 to 62.6, tier 2
  4. Eikos-4B55.8, interval 52.9 to 58.7, tier 3
  5. Kev-4B55.8, interval 53.0 to 58.7, tier 3
  6. Laya-typed47.8, interval 44.9 to 50.5, tier 4
  7. Kev-0.8B46.4, interval 43.5 to 49.3, tier 4
  8. Laya
    45.9, interval 43.1 to 48.8, tier 4
Tested against Jev and 13 open modelsAlso tested against small open models: #2 of 5 tied

JevBench public

Tier 4 of 5#6 / 12

Accuracy, % · Higher is better

  1. Eikos-27B*91.8*, interval 87.9 to 95.2, trained on this data, not ranked
  2. AutoJev-27B86.6, interval 81.9 to 90.9, tied for the lead
  3. Shisa DE-186.1, interval 81.2 to 90.6, tied for the lead
  4. Jev85.7, interval 81.0 to 90.1, tied for the lead
  5. Eikos-4B*85.3*, interval 80.4 to 89.8, trained on this data, not ranked
  6. SemIf (Qwen3.5-4B)80.1, interval 74.7 to 85.3, tier 2
  7. Kev-4B71.4, interval 65.2 to 77.4, tier 3
  8. Laya
    58.9, interval 52.4 to 65.2, tier 4
Tested against Jev and 13 open modelsAlso tested against small open models: #2 of 7 tied

Speed

Median latency, p50 · Laptop GPU only · lower is better

  1. openJev Verdict 1.445 ms
  2. Laya
    50 ms
  3. Laya-typed53 ms
  4. SemIf (Qwen3-0.6B)60 ms
  5. GLiNER2.5-Decide61 ms
  6. Kev-0.6B61 ms
  7. Kev-0.8B85 ms
  8. SimpleJev (Qwen3.5-0.8B)196 ms
How we count

A model leads a task when it is in the task's leading tie tier. Each task counts inside its own benchmark, against that benchmark's models; no score is averaged. An area chart shows the models ranked on at least 2 of the area's tasks in a benchmark this model is in too. The overall standing counts every non-coding tasks and needs 5 for a rank. Latency is compared only on one hardware tier.

LayaOther modelsTrained on the test data*Whisker: 95% interval