NeuralAI-Nae1 / eval_fresh_step32000_q4_n512.json
Subject-Emu-5259's picture
step-32000 refresh: PPL 54.16 (corrected-label cycle), weights/eval/charts/badges + fresh-corpus cross-check
1b03f30 verified
Raw History Blame Contribute Delete
537 Bytes
{
"model": "artifacts/nae1_gguf/nae1-llama-Q4_K_M-step32000.gguf",
"context_size": 512,
"scoring": "teacher forcing: one llm.eval(chunk) + row j scores token j+1 (first token excluded)",
"probe": {
"text": "The quick brown fox",
"tokens": 10,
"total_nll_nats": 34.301195067379204,
"perplexity": 30.880332937026726
},
"heldout": {
"documents": 300,
"tokens": 350788,
"total_nll_nats": 1418830.3483974792,
"mean_nll_nats_per_token": 4.044694654313942,
"perplexity": 57.093750042001716
}
}