Byrne-15M Looped
#101
by Quazim0t0 - opened
https://huggingface.co/Quazim0t0/Byrne-15M-Looped
Evaluation
Full, uncapped. Same harness as Ultra-MC. Raw JSON:benchmarks/Byrne-15M-Looped.json
(base) andbenchmarks/Byrne-15M-DPOseg1.json.
Tables: BENCHMARKS.md.
| Metric | base | DPO seg1 | chance |
|---|---|---|---|
| WikiText-2 byte_ppl ↓ | 2.943 | 3.014 | — |
| BLiMP acc ↑ | 0.734 | 0.711 | 0.50 |
| arc_easy acc | 0.335 | 0.325 | 0.25 |
| arc_easy acc_norm | 0.328 | 0.326 | 0.25 |
| arc_challenge acc | 0.182 | 0.179 | 0.25 |
| arc_challenge acc_norm | 0.224 | 0.232 | 0.25 |
| hellaswag acc | 0.267 | 0.269 | 0.25 |
| hellaswag acc_norm | 0.272 | 0.271 | 0.25 |
| winogrande acc | 0.509 | 0.512 | 0.50 |
| piqa acc | 0.533 | 0.554 | 0.50 |
| piqa acc_norm | 0.521 | 0.539 | 0.50 |
| openbookqa acc | 0.126 | 0.138 | 0.25 |
| openbookqa acc_norm | 0.248 | 0.262 | 0.25 |
| boolq acc | 0.378 | 0.397 | 0.50 |
| ArithMark-3.0 acc | 0.345 | 0.322 | 0.25 |
| ArithMark-3.0 acc_norm | 0.347 | 0.327 | 0.25 |
Hi Dean, thanks for your submission, added!
Datdanboi25 changed discussion status to closed