EryriLabs/Nev-2B-GGUF

Nev

Version 0.1.0, published 2026-09-30.

Nev answers typed questions about a piece of text with a probability for each option. It is Qwen/Qwen3.5-2B-Base, fine-tuned for one job: read a prompt that lists the options as letters, and put its next-token probability on the right letter. It does not write text. The GGUF files in this repo run in llama.cpp and Ollama. Each file carries its own fitted calibration, and a local server turns a running file into a Jev-style /v1/systemone endpoint.

What is new here

Each claim below has its receipt. The claims were checked on 2026-09-27.

• As of 2026-09-27, no other open decision model we could find was trained on the full option distributions of other open decision models. Nev learned from what Mapika/decider-4b v2 and crh225/Plumb-4B put on every option, not just their top answers.
  • Receipt: Section 3.1 of the survey shipped with this repo. The nearest things: JevK5 distils from general chat models, JEV-9B from the closed Jev, and the decider project keeps only teacher labels the teachers agree on.
• As of 2026-09-27, every open decision model we looked at treats the levels of a score question as unrelated classes. Nev trains them as ordered levels and answers with an expected value as well as the most likely level.
  • Receipt: Section 3.2 of the survey. Kev tried an ordinal loss and did not ship it.
• As of 2026-09-27, the surveyed models publish one set of temperatures per model. Each Nev file carries temperatures fitted on that file's own log-probabilities, read through llama-server, because a Q4 file and a Q8 file do not put out the same numbers.
  • Receipt: Section 3.3 of the survey, which lists the single temperature sets of decider-2b, decider-4b, JevK5 and Plumb-4B.
• As of 2026-09-27, no GGUF we found stores its calibration as data. Nev keeps it in the file's metadata under nev.calibration, and the endpoint reads it from there with --gguf.
  • Receipt: Section 3.4 of the survey. The one calibrated GGUF we found folds a temperature into the RMSNorm weights instead.
• As of 2026-09-27, no decision model we found corrects for the model's preference for certain answer letters. Nev is trained with a consistency term across two option orders, and the endpoint can average over several orders after dividing out a letter prior fitted on validation.
  • Receipt: Section 3.5 of the survey. AnyJev rotates answers with no prior; Kev shelved its permutation loss; the correction follows PriDe (arXiv 2309.03882). Training for order consistency on its own is not new.
• As of 2026-09-27, nobody had published a Jev-compatible /v1/systemone endpoint that runs on llama-server or Ollama. Nev ships one, with a band on every answer that says whether to accept it, look at it, or hand it to something bigger. Update, 2026-09-30: Ollama 0.35, released on 2026-09-28, now serves /v1/systemone itself for Qwen3.5-based GGUF files, and Nev runs there too (see Quick start). The shim is still the way to use Nev with llama-server or LM Studio, and the only one of the two that applies the calibration stored in the file.
  • Receipt: Section 3.6 of the survey. jevk5-serve serves that shape from PyTorch; the other GGUF releases ship a client or a proprietary scorer. Section 6 of the survey records the Ollama 0.35 release and how its endpoint works.

Quick start

These steps use the Q8_0 file, nev-Q8_0.gguf.

Ollama

Ollama 0.35.0 and later answer these requests themselves, at /v1/systemone, without the shim. Save these two lines as Modelfile:

FROM hf.co/EryriLabs/Nev-2B-GGUF:nev-Q8_0.gguf
PARAMETER num_ctx 8192

The tag names the file in full, because this repo holds two files of each quantisation (see llama.cpp below).

Create the model once:

ollama create nev -f Modelfile

Then send requests to http://localhost:11434/v1/systemone, with "model": "nev" added to the request shown under Request and response. Ollama's answers use its own field names: a yes/no answer gives noul, the probability of yes, and a score answer gives score, the expected level, with a legend. There is no band.

What we found when testing it with Ollama 0.35.0 on 2026-09-30:

• Nev needs no changes to run there. On the 231 JevBench public items it scored 0.797 through Ollama's endpoint, against 0.810 through the shim: 1.000 on the easy tier, 0.861 on the original tier and 0.667 on the hard tier.
• Ollama builds one prompt format for every decision model, so Nev reads a layout there that it was not trained on. Ollama applies no calibration of its own: the temperatures stored in the file and the letter-order correction are used only by the shim.
• Describe both sides of a yes/no question in criteria when you can. On the JevBench yes/no items Nev scored 0.838 with descriptions and 0.797 with Ollama's bare Yes and No.
• On a 32-thread desktop CPU with no GPU, Nev took about 3.5 seconds per decision through Ollama, against about 1.4 through the shim and llama-server.
• Ollama 0.35.0 accepts any GGUF of the Qwen3.5 family, so no extra line is needed. Ollama's development code after 0.35.0 asks decision models to declare themselves with a CAPABILITY decision line in the Modelfile, which 0.35.0 itself rejects. If a later version turns Nev away, add that line.

To run Nev on Ollama through the shim instead, which also applies the calibration stored in the file, use the shim below with --backend ollama:hf.co/EryriLabs/Nev-2B-GGUF:nev-Q8_0.gguf and pass --gguf with a downloaded copy of the same file.

llama.cpp

llama.cpp's server answers these requests itself from the builds of 3 October 2026 (b11365 and later), at /v1/systemone, without the shim. It needs a file that carries the model type, the prompt and the temperatures as metadata, so this repo has a second copy of each quantisation with those entries added: nev-systemone-Q4_K_M.gguf, nev-systemone-Q5_K_M.gguf, nev-systemone-Q6_K.gguf, nev-systemone-Q8_0.gguf. The weights are identical to the plain files.

llama-server -hf EryriLabs/Nev-2B-GGUF:nev-systemone-Q8_0.gguf -c 8192 --port 8080

Then send requests to http://localhost:8080/v1/systemone as shown under Request and response. The prompt llama.cpp builds from the metadata is Nev's own one-question prompt, so nothing is lost against the shim except the letter-order correction, which makes no measurable difference. What we found with build b11372 on 2026-10-03:

• Q8_0, on a 32-thread desktop CPU with no GPU: typed-decisions 0.571 against 0.570 through the shim, JevBench public 0.814 against 0.810.
• 99% of the answers on typed-decisions and 98% on JevBench were identical to the shim's; where they differed, the probabilities differed by 0.005 on average.
• About 0.8 seconds per typed-decisions decision, against 3.5 through Ollama's own route.

Use the plain files with Ollama and the nev-systemone- files with llama.cpp: Ollama 0.35.1 reads the same metadata entry and rejects llama.cpp's value, and llama.cpp treats a file without it as an ordinary language model.

llama-server and the shim

Download the file and start llama.cpp's server:

hf download EryriLabs/Nev-2B-GGUF nev-Q8_0.gguf --local-dir .
llama-server -m nev-Q8_0.gguf -c 8192 --port 8080

In a second terminal, start the shim. Its source ships in this repo, in the python/ folder: the nev package and the parts of hev it uses, plain Python files. The shim does not need torch or transformers; it needs Python 3.12 or later, fastapi, uvicorn, and gguf to read the file's calibration with --gguf.

hf download EryriLabs/Nev-2B-GGUF --include "python/*" --local-dir .
pip install fastapi uvicorn gguf
PYTHONPATH=python python -m nev.serve --backend llama-server:http://127.0.0.1:8080 --gguf nev-Q8_0.gguf

On Windows, set the path first: set PYTHONPATH=python, then run the last line without the prefix.

--gguf reads the calibration stored in the file. The shim listens on http://127.0.0.1:8090. It answers POST /v1/systemone (see Request and response) and GET /health.

Raw prompt

To call a runtime yourself, send the prompt as plain text with no chat template. This is the layout for one question, exactly as the shim builds it:

Context:
Help! My payouts have been failing for 3 days.

Question: Which team should handle this?
Options:
(A) billing: Payments, invoicing, refunds
(B) technical: Bugs, outages, integrations
(C) sales: Pricing, upgrades, new accounts
Answer: (

The prompt ends right after the (, with no space or newline. The model's next token is the answer letter. The shim turns the letters' log-probabilities into probabilities with nev.readout.logprobs_from_top and then calibrated_probs. To get the same numbers from a raw call:

1. Ask for one token and its top log-probabilities. With llama-server, POST the prompt to /completion with "n_predict": 1, "n_probs": 40, "temperature": 0 and "stream": false. With Ollama, POST to /api/generate with "model", "prompt", "raw": true, "stream": false, "logprobs": true, "top_logprobs": 20 and "options": {"num_predict": 1, "temperature": 0}. These are the requests the shim sends. When a question has more options than the list holds, some letters are always missing from it.
2. From the top list, keep the tokens for the letters in use: A, B and C here. Strip spaces from the token text, and if a letter appears more than once, keep its highest log-probability. A letter that is not in the list gets the lowest log-probability anywhere in the list minus 4.
3. Read the JSON stored in the GGUF metadata key nev.calibration. The temperature T for the question is bucket_temps[type][size] if that entry exists, where size is "2", "3-5", "6-10" or "11+" by the number of options. Otherwise T is temps[type], or 1 if that is missing too.
4. For a choice question only: if the calibration has a letter_prior entry for this number of options (the key is the count as text, such as "3"), subtract the log of the entry's j-th value from the log-probability of the letter at position j, counting A as position 0. Yes/no and score questions skip this step: the prior is fitted on, and removed from, choice questions only.
5. Divide each letter's log-probability by T, then take a softmax over the letters in use. This gives one probability per option, in the order listed.

Letter A is the first option as listed. Yes/no questions are listed as (A) no and (B) yes. Score questions list their levels in order, as 0: <description>, 1: <description> and so on. By default the shim sends one prompt like this per question.

Request and response

The shim takes the same request shape as Jev's /v1/systemone. Save this as request.json:

{
  "state": "Help! My payouts have been failing for 3 days.",
  "questions": {
    "department": {"type": "choice", "instructions": "Which team should handle this?",
                   "criteria": {"billing": "Payments, invoicing, refunds",
                                "technical": "Bugs, outages, integrations",
                                "sales": "Pricing, upgrades, new accounts"}},
    "urgent": {"type": "noul", "instructions": "This needs attention today."},
    "severity": {"type": "score", "instructions": "How bad is the customer impact?",
                 "criteria": ["none", "minor", "major", "critical"]}
  }
}

Then send it:

curl -s http://127.0.0.1:8090/v1/systemone -H "Content-Type: application/json" -d @request.json

The response has one answer per question. This example shows the shape only. Its numbers are made up.

{
  "model": "nev",
  "answers": {
    "department": {"type": "choice", "choice": "billing",
                   "probabilities": {"billing": 0.91, "technical": 0.07, "sales": 0.02},
                   "confidence": 0.91, "band": "accept"},
    "urgent": {"type": "noul", "answer": true, "probability": 0.78, "noul": 0.78,
               "probabilities": {"true": 0.78, "false": 0.22},
               "confidence": 0.78, "band": "review"},
    "severity": {"type": "score", "level": 2, "expected": 1.9, "score": 1.9,
                 "legend": ["none", "minor", "major", "critical"],
                 "probabilities": {"0": 0.02, "1": 0.2, "2": 0.64, "3": 0.14},
                 "confidence": 0.64, "band": "review"}
  },
  "usage": {"calls": 3, "permutations": 1, "backend": "llama-server"}
}

Every answer has probabilities (one per option), confidence (the largest probability) and band. The band is accept, review or escalate, set by comparing confidence with thresholds of 0.85 and 0.6 by default. A choice answer names the choice. A yes/no answer gives answer and probability, the probability of yes. A score answer gives the most likely level and the expected level. noul and score repeat probability and expected under the names Hev's answers use, and legend lists a score question's level descriptions. Add "permutations": 4 to a request to average over up to four orders (at most one per option) of each choice question's options. More permutations mean more runtime calls.

Results

Measured rows were run for this card with the files above. Reported rows are copied from the source named in the row and were not re-run here. acc is top-1 accuracy. brier is the multi-class Brier score, summed over options. ece is top-label expected calibration error with 15 equal-width bins. Lower is better for brier and ece. Reported rows use their source's own definitions, which can differ. n is the number of decisions.

typed-decisions test (LocalLLaMA/typed-decisions)

Held out from Nev's training: the Nev rows are zero-shot.

Nev saw no typed-decisions data in training, so its rows here are zero-shot. On the typed-decisions leaderboard, specialist rows are fitted on the train split of the same workflows, and generalist rows are scored zero-shot. The dataset card says the two kinds of number are not comparable, so this table is not a ranking across them.

Model	Kind	Source	acc	brier	ece	n
Nev Q8_0	measured	this release: nev-Q8_0.gguf through llama-server on CPU	0.570	0.191	0.079	2,000
Nev Q4_K_M	measured	this release: nev-Q4_K_M.gguf through llama-server on CPU	0.575	0.185	0.076	2,000
meraGPT Decider 1 (generalist, zero-shot)	reported	LocalLLaMA/typed-decisions dataset card, read 2026-09-27	0.768	0.052	-	-
TypeSafe Jev 1.13.0 (generalist)	reported	LocalLLaMA/typed-decisions dataset card, read 2026-09-27	0.727	0.148	-	-
Featherless Simple Jev	reported	LocalLLaMA/typed-decisions dataset card, read 2026-09-27	0.716	0.176	-	-
ModernBERT-base (specialist, fitted)	reported	LocalLLaMA/typed-decisions dataset card, read 2026-09-27	0.646	0.119	-	-
MiniLM-L6 (specialist, fitted)	reported	LocalLLaMA/typed-decisions dataset card, read 2026-09-27	0.587	0.143	-	-

JevBench public items

Held out from training. Accuracy and calibration on the public items only; the JevBench leaderboard composite (accuracy, calibration, speed and cost) is not computed here.

Model	Kind	Source	acc	brier	ece	n
Nev Q8_0	measured	this release: nev-Q8_0.gguf through llama-server on CPU	0.810	0.248	0.086	231
Nev Q4_K_M	measured	this release: nev-Q4_K_M.gguf through llama-server on CPU	0.810	0.238	0.079	231

fast-decisions public

Held out from training.

Model	Kind	Source	acc	brier	ece	n
Nev Q8_0	measured	this release: nev-Q8_0.gguf through llama-server on CPU	0.616	0.537	0.141	2,500
Nev Q4_K_M	measured	this release: nev-Q4_K_M.gguf through llama-server on CPU	0.611	0.542	0.145	2,500

validation_clean sample (600 examples)

The calibration was fitted on this sample, so these rows are in-sample; they show the effect of averaging over option orders.

Model	Kind	Source	acc	brier	ece	n
Nev Q8_0, one pass	measured	this release: nev-Q8_0.gguf through llama-server on CPU	0.788	0.299	0.046	600
Nev Q8_0, 4 option orders	measured	this release: nev-Q8_0.gguf through llama-server on CPU	0.778	0.301	0.045	600
Nev Q4_K_M, one pass	measured	this release: nev-Q4_K_M.gguf through llama-server on CPU	0.792	0.299	0.038	600
Nev Q4_K_M, 4 option orders	measured	this release: nev-Q4_K_M.gguf through llama-server on CPU	0.780	0.301	0.028	600

Calibration and quantisation

Each file has its own calibration, fitted on that file's own logits over the validation set. It is stored in the file's metadata under nev.calibration, and the shim reads it with --gguf. It always holds one temperature per question type (temps). It can also hold a temperature per question type and number of options (bucket_temps), used instead of the type's temperature where one was fitted, and a letter-position prior per number of options for choice questions (letter_prior). The raw prompt steps in the Quick start show how each one is applied. The T columns below are the per-type temperatures. A temperature above 1 softens the probabilities and one below 1 sharpens them. Top-1 agreement is the share of validation decisions on which the file picks the same option as the BF16 checkpoint.

File	Quant	Size (GB)	T choice (per type)	T noul (per type)	T score (per type)	Top-1 agreement with BF16
nev-Q4_K_M.gguf	Q4_K_M	1.27	0.911	0.884	0.825	0.967
nev-Q5_K_M.gguf	Q5_K_M	1.41	0.931	0.903	0.854	0.978
nev-Q6_K.gguf	Q6_K	1.56	0.924	0.871	0.847	0.973
nev-Q8_0.gguf (recommended)	Q8_0	2.01	0.922	0.917	0.846	0.992

Training

• Base model: Qwen/Qwen3.5-2B-Base
• Teachers: Mapika/decider-4b (revision v2, Apache-2.0): its option distributions on 42,752 training examples, read in its own prompt layout, crh225/Plumb-4B (Apache-2.0): its option distributions on 13,704 training examples, through the jevk5 package
• Distillation: Nev learned from the teachers' full probability distributions over the options, not only their top answers.
• Losses: log-loss on the gold answer, distillation to the teachers, a Brier term, an ordinal loss for score questions (a nearby level counts as partly right), and a consistency term between two orders of the same options.
• Training examples: 47,649
• GPU: one NVIDIA A100 80GB (RunPod, secure cloud)
• GPU hours: 2.77
• Cost: $4.37

Sources, with the number of training examples from each:

Source	Examples
converted/ag_news	270
converted/banking77	630
converted/clinc150	720
converted/dbpedia	360
converted/snips	270
jev-bench/arc_challenge	799
jev-bench/boolq	788
jev-bench/fever_evidence	800
jev-bench/helpsteer2_helpfulness	797
jev-bench/helpsteer2_verbosity	798
jev-bench/mmlu	285
jev-bench/mnli	800
jev-bench/paws	800
jev-bench/sms_spam	795
jev-bench/sst5	800
jev-bench/strategyqa_closed	800
jev-bench/strategyqa_grounded	800
jev-bench/stsb	800
jev-bench/yelp5	800
plumb-decisions	4,764
bev/browser_interaction	1,015
bev/civic_and_safety_operations	207
bev/contract_evidence	356
bev/engagement_and_ranking	1,006
bev/financial_reporting_and_banking	1,668
bev/game_state_decisions	362
bev/reading_comprehension	469
bev/response_preference_and_quality	1,701
bev/retail_product_and_shopping	3,791
bev/scientific_and_paper_understanding	2,022
bev/sensory_quality_rating	220
bev/sentiment_emotion_and_moderation	4,617
bev/software_engineering_and_code	1,405
bev/software_security	1,308
bev/spam_detection	265
bev/spatial_and_logical_reasoning	3,716
bev/support_and_intent_routing	2,340
bev/tool_and_workflow_decisions	1,905
jev-bench/civil_comments	800
jev-bench/measuring_hate_speech	800

Data licences: each source's licence is as declared on its Hub page.

Cloud-only data: Two sources were downloaded, converted and used on the training machine only, never on the author's own computers: avbiswas/bev-decision-150K (licence listed as unknown on its card; a 29,973-example sample of its train split, all 18 domains, and its test split held out) and the jev-bench configs civil_comments, measuring_hate_speech and go_emotions. Held out from all training: LocalLLaMA/typed-decisions (train and test), the JevBench public items, the jev-bench test split, fast-decisions and the bev test split; training rows whose state matched a held-out state were dropped (84 rows).

Limits

• Choice questions can have at most 26 options, one per letter from A to Z. The shim answers HTTP 422 above that.
• The state is cut to its first 20,000 characters (about 5,000 tokens) when read; training used a 6,000-character cut.
• At most 32 questions per request.
• Over llama-server or Ollama, the shim makes a separate runtime call for each question and each extra permutation.
• typed-decisions is the weak spot: 0.585 accuracy zero-shot (multi-question readout) against Jev's 0.727 and the teacher Decider-4b v2's own 0.679 on the same 2,000 decisions (measured here, its raw logits). The 2B student loses about ten points to its teacher on these four workflows, most on yes/no and urgency-style score questions; on the JevBench public items it matches the teacher. Treat Nev as a first release.
• A request with k questions costs k runtime calls that share a cached prefix, because llama-server returns probabilities only for the token it generates. The one-pass path needs a llama-cpp-python built against llama.cpp b11227 or newer; the pip wheel available on 2026-09-28 cannot load Qwen3.5 files.
• Qwen3.5's linear-attention layers had open llama.cpp issues on some AMD and Intel backends on 2026-09-27 (HIP #27556, SYCL #29183). CPU and CUDA are what was tested here.
• English only, text only.

Licence

Apache-2.0. The base model (Qwen/Qwen3.5-2B-Base), the teachers and the training sources keep their own licences.