Text Generation
Transformers
Safetensors
English
reasoning
reinforcement-learning
synthetic-data
extrapolation
Instructions to use ZachW/extrapolation_rl with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ZachW/extrapolation_rl with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="ZachW/extrapolation_rl")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("ZachW/extrapolation_rl", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use ZachW/extrapolation_rl with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ZachW/extrapolation_rl" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ZachW/extrapolation_rl", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/ZachW/extrapolation_rl
- SGLang
How to use ZachW/extrapolation_rl with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ZachW/extrapolation_rl" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ZachW/extrapolation_rl", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ZachW/extrapolation_rl" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ZachW/extrapolation_rl", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use ZachW/extrapolation_rl with Docker Model Runner:
docker model run hf.co/ZachW/extrapolation_rl
Update model card for intermediate checkpoints
Browse files- 020305/README.md +13 -8
- 050302/README.md +13 -8
- README.md +14 -9
020305/README.md
CHANGED
|
@@ -7,24 +7,28 @@ This release contains the 020305 and 050302 extrapolation checkpoint batches for
|
|
| 7 |
```text
|
| 8 |
020305/
|
| 9 |
README.md
|
| 10 |
-
pretrain/checkpoint-18809/
|
| 11 |
-
rl/
|
| 12 |
-
rl/
|
| 13 |
-
rl/
|
|
|
|
| 14 |
opd_refkl/op11_14_step200/
|
| 15 |
evals/
|
| 16 |
050302/
|
| 17 |
README.md
|
| 18 |
-
pretrain/checkpoint-18586/
|
| 19 |
-
rl/
|
| 20 |
-
rl/
|
| 21 |
-
rl/
|
|
|
|
| 22 |
opd_refkl/op13_20_step200/
|
| 23 |
evals/
|
| 24 |
```
|
| 25 |
|
| 26 |
The default upload includes compact evaluation files: `summary.csv`, `summary.json`, per-model `*_metrics.json`, graph `results.json`, and score reports. Raw generation JSONL files and per-sample graph-result JSONL files are large and are uploaded when the upload helper is run with `--include-generations` and `--include-per-sample-results`.
|
| 27 |
|
|
|
|
|
|
|
| 28 |
## Run Index
|
| 29 |
|
| 30 |
| Model path | Exp name | Script/config | Model name | Time |
|
|
@@ -59,6 +63,7 @@ All rows use 1,152,000 ID examples, 1,280,000 OOD examples, and 2,432,000 total
|
|
| 59 |
|
| 60 |
## Source Pointers
|
| 61 |
|
|
|
|
| 62 |
- Data: <https://huggingface.co/datasets/Interplay-LM-Reasoning/composition>
|
| 63 |
- Context data: <https://huggingface.co/datasets/Interplay-LM-Reasoning/context>
|
| 64 |
- Repository documentation: `docs/extrapolation_release_wrapup.md` in the GitHub repository.
|
|
|
|
| 7 |
```text
|
| 8 |
020305/
|
| 9 |
README.md
|
| 10 |
+
pretrain/checkpoint-{3762,7524,11286,15048,18809}/
|
| 11 |
+
rl/op11_14_step{20,40,60,80,100,120,140,160,180,200}/
|
| 12 |
+
rl/op13_20_step{20,40,60,80,100,120,140,160,180,200}/
|
| 13 |
+
rl/op15_20_step{20,40,60,80,100,120,140,160,180,200}/
|
| 14 |
+
opd_refkl/sweep/step{20,40,60,80,100,120,140,160,180}/
|
| 15 |
opd_refkl/op11_14_step200/
|
| 16 |
evals/
|
| 17 |
050302/
|
| 18 |
README.md
|
| 19 |
+
pretrain/checkpoint-{3718,7436,11154,14872,18586}/
|
| 20 |
+
rl/op11_14_step{20,40,60,80,100,120,140,160,180,200}/
|
| 21 |
+
rl/op13_20_step{20,40,60,80,100,120,140,160,180,200}/
|
| 22 |
+
rl/op15_20_step{20,40,60,80,100,120,140,160,180,200}/
|
| 23 |
+
opd_refkl/sweep/step{20,40,60,80,100,120,140,160,180}/
|
| 24 |
opd_refkl/op13_20_step200/
|
| 25 |
evals/
|
| 26 |
```
|
| 27 |
|
| 28 |
The default upload includes compact evaluation files: `summary.csv`, `summary.json`, per-model `*_metrics.json`, graph `results.json`, and score reports. Raw generation JSONL files and per-sample graph-result JSONL files are large and are uploaded when the upload helper is run with `--include-generations` and `--include-per-sample-results`.
|
| 29 |
|
| 30 |
+
The non-final pretrain/RL/OPD checkpoints are model-only unless an evaluation artifact path is separately listed below. The 050302 OPD sweep has intermediate evaluation artifacts under `050302/evals/opd_refkl_sweep/`.
|
| 31 |
+
|
| 32 |
## Run Index
|
| 33 |
|
| 34 |
| Model path | Exp name | Script/config | Model name | Time |
|
|
|
|
| 63 |
|
| 64 |
## Source Pointers
|
| 65 |
|
| 66 |
+
- Model release root: <https://huggingface.co/ZachW/extrapolation_rl>
|
| 67 |
- Data: <https://huggingface.co/datasets/Interplay-LM-Reasoning/composition>
|
| 68 |
- Context data: <https://huggingface.co/datasets/Interplay-LM-Reasoning/context>
|
| 69 |
- Repository documentation: `docs/extrapolation_release_wrapup.md` in the GitHub repository.
|
050302/README.md
CHANGED
|
@@ -7,24 +7,28 @@ This release contains the 020305 and 050302 extrapolation checkpoint batches for
|
|
| 7 |
```text
|
| 8 |
020305/
|
| 9 |
README.md
|
| 10 |
-
pretrain/checkpoint-18809/
|
| 11 |
-
rl/
|
| 12 |
-
rl/
|
| 13 |
-
rl/
|
|
|
|
| 14 |
opd_refkl/op11_14_step200/
|
| 15 |
evals/
|
| 16 |
050302/
|
| 17 |
README.md
|
| 18 |
-
pretrain/checkpoint-18586/
|
| 19 |
-
rl/
|
| 20 |
-
rl/
|
| 21 |
-
rl/
|
|
|
|
| 22 |
opd_refkl/op13_20_step200/
|
| 23 |
evals/
|
| 24 |
```
|
| 25 |
|
| 26 |
The default upload includes compact evaluation files: `summary.csv`, `summary.json`, per-model `*_metrics.json`, graph `results.json`, and score reports. Raw generation JSONL files and per-sample graph-result JSONL files are large and are uploaded when the upload helper is run with `--include-generations` and `--include-per-sample-results`.
|
| 27 |
|
|
|
|
|
|
|
| 28 |
## Run Index
|
| 29 |
|
| 30 |
| Model path | Exp name | Script/config | Model name | Time |
|
|
@@ -59,6 +63,7 @@ All rows use 1,152,000 ID examples, 1,280,000 OOD examples, and 2,432,000 total
|
|
| 59 |
|
| 60 |
## Source Pointers
|
| 61 |
|
|
|
|
| 62 |
- Data: <https://huggingface.co/datasets/Interplay-LM-Reasoning/composition>
|
| 63 |
- Context data: <https://huggingface.co/datasets/Interplay-LM-Reasoning/context>
|
| 64 |
- Repository documentation: `docs/extrapolation_release_wrapup.md` in the GitHub repository.
|
|
|
|
| 7 |
```text
|
| 8 |
020305/
|
| 9 |
README.md
|
| 10 |
+
pretrain/checkpoint-{3762,7524,11286,15048,18809}/
|
| 11 |
+
rl/op11_14_step{20,40,60,80,100,120,140,160,180,200}/
|
| 12 |
+
rl/op13_20_step{20,40,60,80,100,120,140,160,180,200}/
|
| 13 |
+
rl/op15_20_step{20,40,60,80,100,120,140,160,180,200}/
|
| 14 |
+
opd_refkl/sweep/step{20,40,60,80,100,120,140,160,180}/
|
| 15 |
opd_refkl/op11_14_step200/
|
| 16 |
evals/
|
| 17 |
050302/
|
| 18 |
README.md
|
| 19 |
+
pretrain/checkpoint-{3718,7436,11154,14872,18586}/
|
| 20 |
+
rl/op11_14_step{20,40,60,80,100,120,140,160,180,200}/
|
| 21 |
+
rl/op13_20_step{20,40,60,80,100,120,140,160,180,200}/
|
| 22 |
+
rl/op15_20_step{20,40,60,80,100,120,140,160,180,200}/
|
| 23 |
+
opd_refkl/sweep/step{20,40,60,80,100,120,140,160,180}/
|
| 24 |
opd_refkl/op13_20_step200/
|
| 25 |
evals/
|
| 26 |
```
|
| 27 |
|
| 28 |
The default upload includes compact evaluation files: `summary.csv`, `summary.json`, per-model `*_metrics.json`, graph `results.json`, and score reports. Raw generation JSONL files and per-sample graph-result JSONL files are large and are uploaded when the upload helper is run with `--include-generations` and `--include-per-sample-results`.
|
| 29 |
|
| 30 |
+
The non-final pretrain/RL/OPD checkpoints are model-only unless an evaluation artifact path is separately listed below. The 050302 OPD sweep has intermediate evaluation artifacts under `050302/evals/opd_refkl_sweep/`.
|
| 31 |
+
|
| 32 |
## Run Index
|
| 33 |
|
| 34 |
| Model path | Exp name | Script/config | Model name | Time |
|
|
|
|
| 63 |
|
| 64 |
## Source Pointers
|
| 65 |
|
| 66 |
+
- Model release root: <https://huggingface.co/ZachW/extrapolation_rl>
|
| 67 |
- Data: <https://huggingface.co/datasets/Interplay-LM-Reasoning/composition>
|
| 68 |
- Context data: <https://huggingface.co/datasets/Interplay-LM-Reasoning/context>
|
| 69 |
- Repository documentation: `docs/extrapolation_release_wrapup.md` in the GitHub repository.
|
README.md
CHANGED
|
@@ -22,20 +22,21 @@ Project code and documentation: <https://github.com/YichenZW/Interplay-LM-Reason
|
|
| 22 |
```text
|
| 23 |
020305/
|
| 24 |
README.md
|
| 25 |
-
pretrain/checkpoint-18809/
|
| 26 |
-
rl/
|
| 27 |
-
rl/
|
| 28 |
-
rl/
|
|
|
|
| 29 |
opd_refkl/op11_14_step200/
|
| 30 |
evals/
|
| 31 |
050302/
|
| 32 |
README.md
|
| 33 |
-
pretrain/checkpoint-18586/
|
| 34 |
-
rl/
|
| 35 |
-
rl/
|
| 36 |
-
rl/
|
|
|
|
| 37 |
opd_refkl/op13_20_step200/
|
| 38 |
-
opd_refkl/sweep/
|
| 39 |
evals/
|
| 40 |
```
|
| 41 |
|
|
@@ -82,6 +83,10 @@ Raw generations are uploaded next to the compact evaluation artifacts when avail
|
|
| 82 |
|
| 83 |
The 020305 far-OOD probes are under `020305/evals/far_ood/`. The 050302 OPD intermediate sweep evaluations are under `050302/evals/opd_refkl_sweep/`.
|
| 84 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 85 |
## Evaluation Summary
|
| 86 |
|
| 87 |
All rows use 1,152,000 ID examples, 1,280,000 OOD examples, and 2,432,000 total examples.
|
|
|
|
| 22 |
```text
|
| 23 |
020305/
|
| 24 |
README.md
|
| 25 |
+
pretrain/checkpoint-{3762,7524,11286,15048,18809}/
|
| 26 |
+
rl/op11_14_step{20,40,60,80,100,120,140,160,180,200}/
|
| 27 |
+
rl/op13_20_step{20,40,60,80,100,120,140,160,180,200}/
|
| 28 |
+
rl/op15_20_step{20,40,60,80,100,120,140,160,180,200}/
|
| 29 |
+
opd_refkl/sweep/step{20,40,60,80,100,120,140,160,180}/
|
| 30 |
opd_refkl/op11_14_step200/
|
| 31 |
evals/
|
| 32 |
050302/
|
| 33 |
README.md
|
| 34 |
+
pretrain/checkpoint-{3718,7436,11154,14872,18586}/
|
| 35 |
+
rl/op11_14_step{20,40,60,80,100,120,140,160,180,200}/
|
| 36 |
+
rl/op13_20_step{20,40,60,80,100,120,140,160,180,200}/
|
| 37 |
+
rl/op15_20_step{20,40,60,80,100,120,140,160,180,200}/
|
| 38 |
+
opd_refkl/sweep/step{20,40,60,80,100,120,140,160,180}/
|
| 39 |
opd_refkl/op13_20_step200/
|
|
|
|
| 40 |
evals/
|
| 41 |
```
|
| 42 |
|
|
|
|
| 83 |
|
| 84 |
The 020305 far-OOD probes are under `020305/evals/far_ood/`. The 050302 OPD intermediate sweep evaluations are under `050302/evals/opd_refkl_sweep/`.
|
| 85 |
|
| 86 |
+
## Non-final Checkpoints
|
| 87 |
+
|
| 88 |
+
The uploaded non-final pretrain/RL/OPD checkpoints are model-only unless an evaluation directory is listed above. The 050302 OPD sweep has intermediate evaluation artifacts under `050302/evals/opd_refkl_sweep/`; the newly added pretrain, RL, and 020305 OPD non-final checkpoints do not have separate evaluation artifacts in this release.
|
| 89 |
+
|
| 90 |
## Evaluation Summary
|
| 91 |
|
| 92 |
All rows use 1,152,000 ID examples, 1,280,000 OOD examples, and 2,432,000 total examples.
|