Hugging Face's logo

crossroderick
/
dalat5

Text Generation
Safetensors
Kazakh
t5
text2text-generation
transliteration
kazakh
low-resource
cultural-nlp
Eval Results
Model card Files
xet
Community
crossroderick commited on
Commit
d17d151
·
1 Parent(s): 5c07823

Model training update with 10 epochs

Browse files
Files changed (42) hide show
  1. checkpoints/{checkpoint-8500 → checkpoint-37000}/config.json +0 -0
  2. checkpoints/{checkpoint-8500 → checkpoint-37000}/generation_config.json +0 -0
  3. checkpoints/{checkpoint-8500 → checkpoint-37000}/model.safetensors +1 -1
  4. checkpoints/{checkpoint-9000 → checkpoint-37000}/optimizer.pt +1 -1
  5. checkpoints/{checkpoint-9000 → checkpoint-37000}/rng_state.pth +1 -1
  6. checkpoints/{checkpoint-9410 → checkpoint-37000}/scaler.pt +1 -1
  7. checkpoints/{checkpoint-8500 → checkpoint-37000}/scheduler.pt +1 -1
  8. checkpoints/{checkpoint-8500 → checkpoint-37000}/special_tokens_map.json +0 -0
  9. checkpoints/{checkpoint-8500 → checkpoint-37000}/tokenizer.json +0 -0
  10. checkpoints/{checkpoint-8500 → checkpoint-37000}/tokenizer_config.json +0 -0
  11. checkpoints/checkpoint-37000/trainer_state.json +552 -0
  12. checkpoints/{checkpoint-9000 → checkpoint-37000}/training_args.bin +1 -1
  13. checkpoints/{checkpoint-9000 → checkpoint-37500}/config.json +0 -0
  14. checkpoints/{checkpoint-9000 → checkpoint-37500}/generation_config.json +0 -0
  15. checkpoints/{checkpoint-9000 → checkpoint-37500}/model.safetensors +1 -1
  16. checkpoints/{checkpoint-8500 → checkpoint-37500}/optimizer.pt +1 -1
  17. checkpoints/{checkpoint-9410 → checkpoint-37500}/rng_state.pth +1 -1
  18. checkpoints/{checkpoint-9000 → checkpoint-37500}/scaler.pt +1 -1
  19. checkpoints/{checkpoint-9000 → checkpoint-37500}/scheduler.pt +1 -1
  20. checkpoints/{checkpoint-9000 → checkpoint-37500}/special_tokens_map.json +0 -0
  21. checkpoints/{checkpoint-9000 → checkpoint-37500}/tokenizer.json +0 -0
  22. checkpoints/{checkpoint-9000 → checkpoint-37500}/tokenizer_config.json +0 -0
  23. checkpoints/checkpoint-37500/trainer_state.json +559 -0
  24. checkpoints/{checkpoint-8500 → checkpoint-37500}/training_args.bin +1 -1
  25. checkpoints/{checkpoint-9410 → checkpoint-37660}/config.json +0 -0
  26. checkpoints/{checkpoint-9410 → checkpoint-37660}/generation_config.json +0 -0
  27. checkpoints/{checkpoint-9410 → checkpoint-37660}/model.safetensors +1 -1
  28. checkpoints/{checkpoint-9410 → checkpoint-37660}/optimizer.pt +1 -1
  29. checkpoints/{checkpoint-8500 → checkpoint-37660}/rng_state.pth +1 -1
  30. checkpoints/{checkpoint-8500 → checkpoint-37660}/scaler.pt +1 -1
  31. checkpoints/{checkpoint-9410 → checkpoint-37660}/scheduler.pt +1 -1
  32. checkpoints/{checkpoint-9410 → checkpoint-37660}/special_tokens_map.json +0 -0
  33. checkpoints/{checkpoint-9410 → checkpoint-37660}/tokenizer.json +0 -0
  34. checkpoints/{checkpoint-9410 → checkpoint-37660}/tokenizer_config.json +0 -0
  35. checkpoints/checkpoint-37660/trainer_state.json +559 -0
  36. checkpoints/{checkpoint-9410 → checkpoint-37660}/training_args.bin +1 -1
  37. checkpoints/checkpoint-8500/trainer_state.json +0 -153
  38. checkpoints/checkpoint-9000/trainer_state.json +0 -160
  39. checkpoints/checkpoint-9410/trainer_state.json +0 -160
  40. model.safetensors +1 -1
  41. src/evaluate_t5.py +1 -0
  42. src/train_t5.py +3 -3
checkpoints/{checkpoint-8500 → checkpoint-37000}/config.json RENAMED
File without changes
checkpoints/{checkpoint-8500 → checkpoint-37000}/generation_config.json RENAMED
File without changes
checkpoints/{checkpoint-8500 → checkpoint-37000}/model.safetensors RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:099bb1c8c42a74550254fb5fcb14f9da8d95dbe52a4f92a925e7ff4761442f0f
3
  size 242041896
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:29b1522edd113a0e6957aaaefb789a41231c072754a6740ee6dabc5183e2ab7c
3
  size 242041896
checkpoints/{checkpoint-9000 → checkpoint-37000}/optimizer.pt RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a98e0330442ac50b48e2cbd8da416200695a4ddd237c8f960fda0e58d677f421
3
  size 484163514
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3c9e020b49ebb44ca57e84d793a9643e505fb8255d4d357edc119fd54a72f91b
3
  size 484163514
checkpoints/{checkpoint-9000 → checkpoint-37000}/rng_state.pth RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e000f5cd5963eb68569b2bbfb3b5a89c397ee45872387e8a2d164a5b8897bd75
3
  size 14244
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2b6e4d98cfbb205caa76399a96ea3fc5c3960f39509b1e3fe6146e9237d2e38e
3
  size 14244
checkpoints/{checkpoint-9410 → checkpoint-37000}/scaler.pt RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9cfd9bc312260feb910a4759784b508d71564bb32d50da0e9bf3fbf3f20f8e9b
3
  size 988
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d64c876d78c9e6f6ab330d55f358e5c588f452bd34137dabaa6c33001baaf827
3
  size 988
checkpoints/{checkpoint-8500 → checkpoint-37000}/scheduler.pt RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:559c3b6e85f97a92dd50a2b91366d7f62d18f6b41bfa38aef84b3235b7b2db5e
3
  size 1064
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:692049a676411f6e31c0de3385e53b736cdc1c7f432b04e599e3367d6d86ea3b
3
  size 1064
checkpoints/{checkpoint-8500 → checkpoint-37000}/special_tokens_map.json RENAMED
File without changes
checkpoints/{checkpoint-8500 → checkpoint-37000}/tokenizer.json RENAMED
File without changes
checkpoints/{checkpoint-8500 → checkpoint-37000}/tokenizer_config.json RENAMED
File without changes
checkpoints/checkpoint-37000/trainer_state.json ADDED
@@ -0,0 +1,552 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": null,
3
+ "best_metric": null,
4
+ "best_model_checkpoint": null,
5
+ "epoch": 9.824747742963357,
6
+ "eval_steps": 500,
7
+ "global_step": 37000,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.1327668613913967,
14
+ "grad_norm": 0.7285390496253967,
15
+ "learning_rate": 4.934014869888476e-05,
16
+ "loss": 2.9202,
17
+ "step": 500
18
+ },
19
+ {
20
+ "epoch": 0.2655337227827934,
21
+ "grad_norm": 0.6645896434783936,
22
+ "learning_rate": 4.867631439192778e-05,
23
+ "loss": 1.7398,
24
+ "step": 1000
25
+ },
26
+ {
27
+ "epoch": 0.3983005841741901,
28
+ "grad_norm": 1.236431360244751,
29
+ "learning_rate": 4.8012480084970795e-05,
30
+ "loss": 1.4128,
31
+ "step": 1500
32
+ },
33
+ {
34
+ "epoch": 0.5310674455655868,
35
+ "grad_norm": 0.4946064054965973,
36
+ "learning_rate": 4.734864577801381e-05,
37
+ "loss": 1.2492,
38
+ "step": 2000
39
+ },
40
+ {
41
+ "epoch": 0.6638343069569835,
42
+ "grad_norm": 0.4114533066749573,
43
+ "learning_rate": 4.668481147105683e-05,
44
+ "loss": 1.1439,
45
+ "step": 2500
46
+ },
47
+ {
48
+ "epoch": 0.7966011683483802,
49
+ "grad_norm": 0.4554953873157501,
50
+ "learning_rate": 4.602097716409984e-05,
51
+ "loss": 1.0689,
52
+ "step": 3000
53
+ },
54
+ {
55
+ "epoch": 0.929368029739777,
56
+ "grad_norm": 0.41119593381881714,
57
+ "learning_rate": 4.5357142857142856e-05,
58
+ "loss": 1.0057,
59
+ "step": 3500
60
+ },
61
+ {
62
+ "epoch": 1.0621348911311737,
63
+ "grad_norm": 0.40032336115837097,
64
+ "learning_rate": 4.4693308550185877e-05,
65
+ "loss": 0.9672,
66
+ "step": 4000
67
+ },
68
+ {
69
+ "epoch": 1.1949017525225702,
70
+ "grad_norm": 0.3746163845062256,
71
+ "learning_rate": 4.402947424322889e-05,
72
+ "loss": 0.9234,
73
+ "step": 4500
74
+ },
75
+ {
76
+ "epoch": 1.327668613913967,
77
+ "grad_norm": 0.4201946556568146,
78
+ "learning_rate": 4.3365639936271904e-05,
79
+ "loss": 0.8978,
80
+ "step": 5000
81
+ },
82
+ {
83
+ "epoch": 1.4604354753053639,
84
+ "grad_norm": 0.48306697607040405,
85
+ "learning_rate": 4.2701805629314924e-05,
86
+ "loss": 0.8672,
87
+ "step": 5500
88
+ },
89
+ {
90
+ "epoch": 1.5932023366967605,
91
+ "grad_norm": 0.348651647567749,
92
+ "learning_rate": 4.2037971322357945e-05,
93
+ "loss": 0.8373,
94
+ "step": 6000
95
+ },
96
+ {
97
+ "epoch": 1.725969198088157,
98
+ "grad_norm": 0.3810581862926483,
99
+ "learning_rate": 4.137413701540096e-05,
100
+ "loss": 0.8161,
101
+ "step": 6500
102
+ },
103
+ {
104
+ "epoch": 1.858736059479554,
105
+ "grad_norm": 0.41756191849708557,
106
+ "learning_rate": 4.071030270844398e-05,
107
+ "loss": 0.8011,
108
+ "step": 7000
109
+ },
110
+ {
111
+ "epoch": 1.9915029208709507,
112
+ "grad_norm": 0.3746052384376526,
113
+ "learning_rate": 4.004646840148699e-05,
114
+ "loss": 0.7917,
115
+ "step": 7500
116
+ },
117
+ {
118
+ "epoch": 2.1242697822623473,
119
+ "grad_norm": 0.3314072787761688,
120
+ "learning_rate": 3.9382634094530006e-05,
121
+ "loss": 0.772,
122
+ "step": 8000
123
+ },
124
+ {
125
+ "epoch": 2.257036643653744,
126
+ "grad_norm": 0.36195287108421326,
127
+ "learning_rate": 3.871879978757303e-05,
128
+ "loss": 0.7519,
129
+ "step": 8500
130
+ },
131
+ {
132
+ "epoch": 2.3898035050451405,
133
+ "grad_norm": 0.36162152886390686,
134
+ "learning_rate": 3.805496548061604e-05,
135
+ "loss": 0.744,
136
+ "step": 9000
137
+ },
138
+ {
139
+ "epoch": 2.5225703664365375,
140
+ "grad_norm": 0.3393162786960602,
141
+ "learning_rate": 3.739245884227297e-05,
142
+ "loss": 0.7317,
143
+ "step": 9500
144
+ },
145
+ {
146
+ "epoch": 2.655337227827934,
147
+ "grad_norm": 0.42738890647888184,
148
+ "learning_rate": 3.67299522039299e-05,
149
+ "loss": 0.7227,
150
+ "step": 10000
151
+ },
152
+ {
153
+ "epoch": 2.7881040892193307,
154
+ "grad_norm": 0.34136396646499634,
155
+ "learning_rate": 3.6066117896972915e-05,
156
+ "loss": 0.7084,
157
+ "step": 10500
158
+ },
159
+ {
160
+ "epoch": 2.9208709506107278,
161
+ "grad_norm": 0.33245041966438293,
162
+ "learning_rate": 3.5402283590015936e-05,
163
+ "loss": 0.6993,
164
+ "step": 11000
165
+ },
166
+ {
167
+ "epoch": 3.0536378120021244,
168
+ "grad_norm": 0.46563392877578735,
169
+ "learning_rate": 3.473844928305895e-05,
170
+ "loss": 0.6865,
171
+ "step": 11500
172
+ },
173
+ {
174
+ "epoch": 3.186404673393521,
175
+ "grad_norm": 0.3868368864059448,
176
+ "learning_rate": 3.407461497610196e-05,
177
+ "loss": 0.6822,
178
+ "step": 12000
179
+ },
180
+ {
181
+ "epoch": 3.3191715347849176,
182
+ "grad_norm": 0.30418872833251953,
183
+ "learning_rate": 3.3410780669144984e-05,
184
+ "loss": 0.6775,
185
+ "step": 12500
186
+ },
187
+ {
188
+ "epoch": 3.451938396176314,
189
+ "grad_norm": 0.34485992789268494,
190
+ "learning_rate": 3.2746946362188e-05,
191
+ "loss": 0.6713,
192
+ "step": 13000
193
+ },
194
+ {
195
+ "epoch": 3.584705257567711,
196
+ "grad_norm": 0.33921709656715393,
197
+ "learning_rate": 3.208311205523102e-05,
198
+ "loss": 0.658,
199
+ "step": 13500
200
+ },
201
+ {
202
+ "epoch": 3.717472118959108,
203
+ "grad_norm": 0.36646100878715515,
204
+ "learning_rate": 3.141927774827403e-05,
205
+ "loss": 0.6516,
206
+ "step": 14000
207
+ },
208
+ {
209
+ "epoch": 3.8502389803505044,
210
+ "grad_norm": 0.32367828488349915,
211
+ "learning_rate": 3.075544344131705e-05,
212
+ "loss": 0.6479,
213
+ "step": 14500
214
+ },
215
+ {
216
+ "epoch": 3.9830058417419014,
217
+ "grad_norm": 0.32735565304756165,
218
+ "learning_rate": 3.0091609134360066e-05,
219
+ "loss": 0.6423,
220
+ "step": 15000
221
+ },
222
+ {
223
+ "epoch": 4.115772703133298,
224
+ "grad_norm": 0.43194663524627686,
225
+ "learning_rate": 2.9427774827403083e-05,
226
+ "loss": 0.6416,
227
+ "step": 15500
228
+ },
229
+ {
230
+ "epoch": 4.248539564524695,
231
+ "grad_norm": 0.29106882214546204,
232
+ "learning_rate": 2.8763940520446096e-05,
233
+ "loss": 0.6318,
234
+ "step": 16000
235
+ },
236
+ {
237
+ "epoch": 4.381306425916091,
238
+ "grad_norm": 0.2671768069267273,
239
+ "learning_rate": 2.810143388210303e-05,
240
+ "loss": 0.6311,
241
+ "step": 16500
242
+ },
243
+ {
244
+ "epoch": 4.514073287307488,
245
+ "grad_norm": 0.311146080493927,
246
+ "learning_rate": 2.7437599575146044e-05,
247
+ "loss": 0.6243,
248
+ "step": 17000
249
+ },
250
+ {
251
+ "epoch": 4.646840148698884,
252
+ "grad_norm": 0.3101503551006317,
253
+ "learning_rate": 2.677376526818906e-05,
254
+ "loss": 0.6157,
255
+ "step": 17500
256
+ },
257
+ {
258
+ "epoch": 4.779607010090281,
259
+ "grad_norm": 0.3017677366733551,
260
+ "learning_rate": 2.6109930961232075e-05,
261
+ "loss": 0.6086,
262
+ "step": 18000
263
+ },
264
+ {
265
+ "epoch": 4.9123738714816785,
266
+ "grad_norm": 0.31505176424980164,
267
+ "learning_rate": 2.5446096654275092e-05,
268
+ "loss": 0.6146,
269
+ "step": 18500
270
+ },
271
+ {
272
+ "epoch": 5.045140732873075,
273
+ "grad_norm": 0.35715609788894653,
274
+ "learning_rate": 2.4783590015932023e-05,
275
+ "loss": 0.602,
276
+ "step": 19000
277
+ },
278
+ {
279
+ "epoch": 5.177907594264472,
280
+ "grad_norm": 0.2930135726928711,
281
+ "learning_rate": 2.4119755708975043e-05,
282
+ "loss": 0.6024,
283
+ "step": 19500
284
+ },
285
+ {
286
+ "epoch": 5.310674455655868,
287
+ "grad_norm": 0.3474890887737274,
288
+ "learning_rate": 2.3455921402018057e-05,
289
+ "loss": 0.6002,
290
+ "step": 20000
291
+ },
292
+ {
293
+ "epoch": 5.443441317047265,
294
+ "grad_norm": 0.29057538509368896,
295
+ "learning_rate": 2.2792087095061074e-05,
296
+ "loss": 0.6006,
297
+ "step": 20500
298
+ },
299
+ {
300
+ "epoch": 5.5762081784386615,
301
+ "grad_norm": 0.3273596167564392,
302
+ "learning_rate": 2.212825278810409e-05,
303
+ "loss": 0.5912,
304
+ "step": 21000
305
+ },
306
+ {
307
+ "epoch": 5.708975039830058,
308
+ "grad_norm": 0.27121296525001526,
309
+ "learning_rate": 2.146574614976102e-05,
310
+ "loss": 0.593,
311
+ "step": 21500
312
+ },
313
+ {
314
+ "epoch": 5.8417419012214555,
315
+ "grad_norm": 0.29718518257141113,
316
+ "learning_rate": 2.0801911842804035e-05,
317
+ "loss": 0.591,
318
+ "step": 22000
319
+ },
320
+ {
321
+ "epoch": 5.974508762612852,
322
+ "grad_norm": 0.32410022616386414,
323
+ "learning_rate": 2.0138077535847052e-05,
324
+ "loss": 0.5857,
325
+ "step": 22500
326
+ },
327
+ {
328
+ "epoch": 6.107275624004249,
329
+ "grad_norm": 0.2846163213253021,
330
+ "learning_rate": 1.9475570897503983e-05,
331
+ "loss": 0.5802,
332
+ "step": 23000
333
+ },
334
+ {
335
+ "epoch": 6.240042485395645,
336
+ "grad_norm": 0.30459314584732056,
337
+ "learning_rate": 1.8811736590547e-05,
338
+ "loss": 0.5806,
339
+ "step": 23500
340
+ },
341
+ {
342
+ "epoch": 6.372809346787042,
343
+ "grad_norm": 0.5301333069801331,
344
+ "learning_rate": 1.8147902283590017e-05,
345
+ "loss": 0.5789,
346
+ "step": 24000
347
+ },
348
+ {
349
+ "epoch": 6.5055762081784385,
350
+ "grad_norm": 0.2727649509906769,
351
+ "learning_rate": 1.7484067976633034e-05,
352
+ "loss": 0.5783,
353
+ "step": 24500
354
+ },
355
+ {
356
+ "epoch": 6.638343069569835,
357
+ "grad_norm": 0.3003462255001068,
358
+ "learning_rate": 1.682023366967605e-05,
359
+ "loss": 0.5765,
360
+ "step": 25000
361
+ },
362
+ {
363
+ "epoch": 6.771109930961232,
364
+ "grad_norm": 0.35589084029197693,
365
+ "learning_rate": 1.6157727031332982e-05,
366
+ "loss": 0.5787,
367
+ "step": 25500
368
+ },
369
+ {
370
+ "epoch": 6.903876792352628,
371
+ "grad_norm": 0.2738860845565796,
372
+ "learning_rate": 1.5493892724375996e-05,
373
+ "loss": 0.5734,
374
+ "step": 26000
375
+ },
376
+ {
377
+ "epoch": 7.036643653744026,
378
+ "grad_norm": 0.42223164439201355,
379
+ "learning_rate": 1.4830058417419013e-05,
380
+ "loss": 0.5704,
381
+ "step": 26500
382
+ },
383
+ {
384
+ "epoch": 7.169410515135422,
385
+ "grad_norm": 0.2938649654388428,
386
+ "learning_rate": 1.4166224110462028e-05,
387
+ "loss": 0.5686,
388
+ "step": 27000
389
+ },
390
+ {
391
+ "epoch": 7.302177376526819,
392
+ "grad_norm": 0.275078147649765,
393
+ "learning_rate": 1.3503717472118959e-05,
394
+ "loss": 0.5666,
395
+ "step": 27500
396
+ },
397
+ {
398
+ "epoch": 7.434944237918216,
399
+ "grad_norm": 0.35505712032318115,
400
+ "learning_rate": 1.2839883165161976e-05,
401
+ "loss": 0.5631,
402
+ "step": 28000
403
+ },
404
+ {
405
+ "epoch": 7.567711099309612,
406
+ "grad_norm": 0.2507877051830292,
407
+ "learning_rate": 1.2176048858204993e-05,
408
+ "loss": 0.5688,
409
+ "step": 28500
410
+ },
411
+ {
412
+ "epoch": 7.700477960701009,
413
+ "grad_norm": 0.2846459746360779,
414
+ "learning_rate": 1.1512214551248008e-05,
415
+ "loss": 0.5594,
416
+ "step": 29000
417
+ },
418
+ {
419
+ "epoch": 7.833244822092405,
420
+ "grad_norm": 0.31158626079559326,
421
+ "learning_rate": 1.0848380244291025e-05,
422
+ "loss": 0.5653,
423
+ "step": 29500
424
+ },
425
+ {
426
+ "epoch": 7.966011683483803,
427
+ "grad_norm": 0.2899467647075653,
428
+ "learning_rate": 1.0184545937334042e-05,
429
+ "loss": 0.562,
430
+ "step": 30000
431
+ },
432
+ {
433
+ "epoch": 8.098778544875199,
434
+ "grad_norm": 0.27900761365890503,
435
+ "learning_rate": 9.520711630377058e-06,
436
+ "loss": 0.559,
437
+ "step": 30500
438
+ },
439
+ {
440
+ "epoch": 8.231545406266596,
441
+ "grad_norm": 0.29301363229751587,
442
+ "learning_rate": 8.856877323420075e-06,
443
+ "loss": 0.5604,
444
+ "step": 31000
445
+ },
446
+ {
447
+ "epoch": 8.364312267657992,
448
+ "grad_norm": 0.2812318801879883,
449
+ "learning_rate": 8.19304301646309e-06,
450
+ "loss": 0.559,
451
+ "step": 31500
452
+ },
453
+ {
454
+ "epoch": 8.49707912904939,
455
+ "grad_norm": 0.27755317091941833,
456
+ "learning_rate": 7.530536378120022e-06,
457
+ "loss": 0.5527,
458
+ "step": 32000
459
+ },
460
+ {
461
+ "epoch": 8.629845990440787,
462
+ "grad_norm": 0.3323802053928375,
463
+ "learning_rate": 6.866702071163038e-06,
464
+ "loss": 0.5576,
465
+ "step": 32500
466
+ },
467
+ {
468
+ "epoch": 8.762612851832182,
469
+ "grad_norm": 0.2453739196062088,
470
+ "learning_rate": 6.202867764206054e-06,
471
+ "loss": 0.557,
472
+ "step": 33000
473
+ },
474
+ {
475
+ "epoch": 8.89537971322358,
476
+ "grad_norm": 0.28488314151763916,
477
+ "learning_rate": 5.539033457249071e-06,
478
+ "loss": 0.5586,
479
+ "step": 33500
480
+ },
481
+ {
482
+ "epoch": 9.028146574614976,
483
+ "grad_norm": 0.2731677293777466,
484
+ "learning_rate": 4.876526818906001e-06,
485
+ "loss": 0.5529,
486
+ "step": 34000
487
+ },
488
+ {
489
+ "epoch": 9.160913436006373,
490
+ "grad_norm": 0.34274822473526,
491
+ "learning_rate": 4.214020180562932e-06,
492
+ "loss": 0.5562,
493
+ "step": 34500
494
+ },
495
+ {
496
+ "epoch": 9.293680297397769,
497
+ "grad_norm": 0.2875533103942871,
498
+ "learning_rate": 3.550185873605948e-06,
499
+ "loss": 0.5528,
500
+ "step": 35000
501
+ },
502
+ {
503
+ "epoch": 9.426447158789166,
504
+ "grad_norm": 0.2516155242919922,
505
+ "learning_rate": 2.8863515666489647e-06,
506
+ "loss": 0.5538,
507
+ "step": 35500
508
+ },
509
+ {
510
+ "epoch": 9.559214020180562,
511
+ "grad_norm": 0.2524682283401489,
512
+ "learning_rate": 2.222517259691981e-06,
513
+ "loss": 0.5546,
514
+ "step": 36000
515
+ },
516
+ {
517
+ "epoch": 9.69198088157196,
518
+ "grad_norm": 0.25429150462150574,
519
+ "learning_rate": 1.5586829527349974e-06,
520
+ "loss": 0.5537,
521
+ "step": 36500
522
+ },
523
+ {
524
+ "epoch": 9.824747742963357,
525
+ "grad_norm": 0.2699441611766815,
526
+ "learning_rate": 8.948486457780139e-07,
527
+ "loss": 0.5522,
528
+ "step": 37000
529
+ }
530
+ ],
531
+ "logging_steps": 500,
532
+ "max_steps": 37660,
533
+ "num_input_tokens_seen": 0,
534
+ "num_train_epochs": 10,
535
+ "save_steps": 500,
536
+ "stateful_callbacks": {
537
+ "TrainerControl": {
538
+ "args": {
539
+ "should_epoch_stop": false,
540
+ "should_evaluate": false,
541
+ "should_log": false,
542
+ "should_save": true,
543
+ "should_training_stop": false
544
+ },
545
+ "attributes": {}
546
+ }
547
+ },
548
+ "total_flos": 8.011321089982464e+16,
549
+ "train_batch_size": 32,
550
+ "trial_name": null,
551
+ "trial_params": null
552
+ }
checkpoints/{checkpoint-9000 → checkpoint-37000}/training_args.bin RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2067c784529f8fe2c2e4f09f57bcd2f668958cc835cfbd5a037d77580dd52cb6
3
  size 5240
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:344c4c96587b5961c7260f4c9743524e13a2580c248f0b960480131c9cd7dc77
3
  size 5240
checkpoints/{checkpoint-9000 → checkpoint-37500}/config.json RENAMED
File without changes
checkpoints/{checkpoint-9000 → checkpoint-37500}/generation_config.json RENAMED
File without changes
checkpoints/{checkpoint-9000 → checkpoint-37500}/model.safetensors RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6bc7a6d65c9b7421a91c72d1dec0d4f9b537f9b8ffde358969cd8173e4dbdc36
3
  size 242041896
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a338402e3bf19f5af3748021e8a94171ceca1c35df5582f413ff92c5d7228a92
3
  size 242041896
checkpoints/{checkpoint-8500 → checkpoint-37500}/optimizer.pt RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e1cbd3e585a640c2213514e2a2b6b133d128ce865508f3556b4336cfe5260f41
3
  size 484163514
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bb190324515d3eb9a2541f5086e592766f75af11d74431aad585b10d13f894d8
3
  size 484163514
checkpoints/{checkpoint-9410 → checkpoint-37500}/rng_state.pth RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7e543512c54779f4d268ad2028f33638bbd516a8989271d848b128b9f441a110
3
  size 14244
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36c312354a009a35011043e8b37fe022597fef3b47dab58890e9f101000c7480
3
  size 14244
checkpoints/{checkpoint-9000 → checkpoint-37500}/scaler.pt RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8d8ca94441765fa0c615f07e68b441bc9c63cda9f9e3ce738ffd6f3deb621cec
3
  size 988
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d417ea58bb57bcc1fcfdc67de914d6ad404bafbe873af35078ff3406669f3f98
3
  size 988
checkpoints/{checkpoint-9000 → checkpoint-37500}/scheduler.pt RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a4936b40fea92cafe7e9f57beefc1e6297923a92e80d4ea4d36ae312cc8d0567
3
  size 1064
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eb8fd32604fbb9aabbcbeef28b92eac4f06ac2ae6413a4773092416494ae47d7
3
  size 1064
checkpoints/{checkpoint-9000 → checkpoint-37500}/special_tokens_map.json RENAMED
File without changes
checkpoints/{checkpoint-9000 → checkpoint-37500}/tokenizer.json RENAMED
File without changes
checkpoints/{checkpoint-9000 → checkpoint-37500}/tokenizer_config.json RENAMED
File without changes
checkpoints/checkpoint-37500/trainer_state.json ADDED
@@ -0,0 +1,559 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": null,
3
+ "best_metric": null,
4
+ "best_model_checkpoint": null,
5
+ "epoch": 9.957514604354753,
6
+ "eval_steps": 500,
7
+ "global_step": 37500,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.1327668613913967,
14
+ "grad_norm": 0.7285390496253967,
15
+ "learning_rate": 4.934014869888476e-05,
16
+ "loss": 2.9202,
17
+ "step": 500
18
+ },
19
+ {
20
+ "epoch": 0.2655337227827934,
21
+ "grad_norm": 0.6645896434783936,
22
+ "learning_rate": 4.867631439192778e-05,
23
+ "loss": 1.7398,
24
+ "step": 1000
25
+ },
26
+ {
27
+ "epoch": 0.3983005841741901,
28
+ "grad_norm": 1.236431360244751,
29
+ "learning_rate": 4.8012480084970795e-05,
30
+ "loss": 1.4128,
31
+ "step": 1500
32
+ },
33
+ {
34
+ "epoch": 0.5310674455655868,
35
+ "grad_norm": 0.4946064054965973,
36
+ "learning_rate": 4.734864577801381e-05,
37
+ "loss": 1.2492,
38
+ "step": 2000
39
+ },
40
+ {
41
+ "epoch": 0.6638343069569835,
42
+ "grad_norm": 0.4114533066749573,
43
+ "learning_rate": 4.668481147105683e-05,
44
+ "loss": 1.1439,
45
+ "step": 2500
46
+ },
47
+ {
48
+ "epoch": 0.7966011683483802,
49
+ "grad_norm": 0.4554953873157501,
50
+ "learning_rate": 4.602097716409984e-05,
51
+ "loss": 1.0689,
52
+ "step": 3000
53
+ },
54
+ {
55
+ "epoch": 0.929368029739777,
56
+ "grad_norm": 0.41119593381881714,
57
+ "learning_rate": 4.5357142857142856e-05,
58
+ "loss": 1.0057,
59
+ "step": 3500
60
+ },
61
+ {
62
+ "epoch": 1.0621348911311737,
63
+ "grad_norm": 0.40032336115837097,
64
+ "learning_rate": 4.4693308550185877e-05,
65
+ "loss": 0.9672,
66
+ "step": 4000
67
+ },
68
+ {
69
+ "epoch": 1.1949017525225702,
70
+ "grad_norm": 0.3746163845062256,
71
+ "learning_rate": 4.402947424322889e-05,
72
+ "loss": 0.9234,
73
+ "step": 4500
74
+ },
75
+ {
76
+ "epoch": 1.327668613913967,
77
+ "grad_norm": 0.4201946556568146,
78
+ "learning_rate": 4.3365639936271904e-05,
79
+ "loss": 0.8978,
80
+ "step": 5000
81
+ },
82
+ {
83
+ "epoch": 1.4604354753053639,
84
+ "grad_norm": 0.48306697607040405,
85
+ "learning_rate": 4.2701805629314924e-05,
86
+ "loss": 0.8672,
87
+ "step": 5500
88
+ },
89
+ {
90
+ "epoch": 1.5932023366967605,
91
+ "grad_norm": 0.348651647567749,
92
+ "learning_rate": 4.2037971322357945e-05,
93
+ "loss": 0.8373,
94
+ "step": 6000
95
+ },
96
+ {
97
+ "epoch": 1.725969198088157,
98
+ "grad_norm": 0.3810581862926483,
99
+ "learning_rate": 4.137413701540096e-05,
100
+ "loss": 0.8161,
101
+ "step": 6500
102
+ },
103
+ {
104
+ "epoch": 1.858736059479554,
105
+ "grad_norm": 0.41756191849708557,
106
+ "learning_rate": 4.071030270844398e-05,
107
+ "loss": 0.8011,
108
+ "step": 7000
109
+ },
110
+ {
111
+ "epoch": 1.9915029208709507,
112
+ "grad_norm": 0.3746052384376526,
113
+ "learning_rate": 4.004646840148699e-05,
114
+ "loss": 0.7917,
115
+ "step": 7500
116
+ },
117
+ {
118
+ "epoch": 2.1242697822623473,
119
+ "grad_norm": 0.3314072787761688,
120
+ "learning_rate": 3.9382634094530006e-05,
121
+ "loss": 0.772,
122
+ "step": 8000
123
+ },
124
+ {
125
+ "epoch": 2.257036643653744,
126
+ "grad_norm": 0.36195287108421326,
127
+ "learning_rate": 3.871879978757303e-05,
128
+ "loss": 0.7519,
129
+ "step": 8500
130
+ },
131
+ {
132
+ "epoch": 2.3898035050451405,
133
+ "grad_norm": 0.36162152886390686,
134
+ "learning_rate": 3.805496548061604e-05,
135
+ "loss": 0.744,
136
+ "step": 9000
137
+ },
138
+ {
139
+ "epoch": 2.5225703664365375,
140
+ "grad_norm": 0.3393162786960602,
141
+ "learning_rate": 3.739245884227297e-05,
142
+ "loss": 0.7317,
143
+ "step": 9500
144
+ },
145
+ {
146
+ "epoch": 2.655337227827934,
147
+ "grad_norm": 0.42738890647888184,
148
+ "learning_rate": 3.67299522039299e-05,
149
+ "loss": 0.7227,
150
+ "step": 10000
151
+ },
152
+ {
153
+ "epoch": 2.7881040892193307,
154
+ "grad_norm": 0.34136396646499634,
155
+ "learning_rate": 3.6066117896972915e-05,
156
+ "loss": 0.7084,
157
+ "step": 10500
158
+ },
159
+ {
160
+ "epoch": 2.9208709506107278,
161
+ "grad_norm": 0.33245041966438293,
162
+ "learning_rate": 3.5402283590015936e-05,
163
+ "loss": 0.6993,
164
+ "step": 11000
165
+ },
166
+ {
167
+ "epoch": 3.0536378120021244,
168
+ "grad_norm": 0.46563392877578735,
169
+ "learning_rate": 3.473844928305895e-05,
170
+ "loss": 0.6865,
171
+ "step": 11500
172
+ },
173
+ {
174
+ "epoch": 3.186404673393521,
175
+ "grad_norm": 0.3868368864059448,
176
+ "learning_rate": 3.407461497610196e-05,
177
+ "loss": 0.6822,
178
+ "step": 12000
179
+ },
180
+ {
181
+ "epoch": 3.3191715347849176,
182
+ "grad_norm": 0.30418872833251953,
183
+ "learning_rate": 3.3410780669144984e-05,
184
+ "loss": 0.6775,
185
+ "step": 12500
186
+ },
187
+ {
188
+ "epoch": 3.451938396176314,
189
+ "grad_norm": 0.34485992789268494,
190
+ "learning_rate": 3.2746946362188e-05,
191
+ "loss": 0.6713,
192
+ "step": 13000
193
+ },
194
+ {
195
+ "epoch": 3.584705257567711,
196
+ "grad_norm": 0.33921709656715393,
197
+ "learning_rate": 3.208311205523102e-05,
198
+ "loss": 0.658,
199
+ "step": 13500
200
+ },
201
+ {
202
+ "epoch": 3.717472118959108,
203
+ "grad_norm": 0.36646100878715515,
204
+ "learning_rate": 3.141927774827403e-05,
205
+ "loss": 0.6516,
206
+ "step": 14000
207
+ },
208
+ {
209
+ "epoch": 3.8502389803505044,
210
+ "grad_norm": 0.32367828488349915,
211
+ "learning_rate": 3.075544344131705e-05,
212
+ "loss": 0.6479,
213
+ "step": 14500
214
+ },
215
+ {
216
+ "epoch": 3.9830058417419014,
217
+ "grad_norm": 0.32735565304756165,
218
+ "learning_rate": 3.0091609134360066e-05,
219
+ "loss": 0.6423,
220
+ "step": 15000
221
+ },
222
+ {
223
+ "epoch": 4.115772703133298,
224
+ "grad_norm": 0.43194663524627686,
225
+ "learning_rate": 2.9427774827403083e-05,
226
+ "loss": 0.6416,
227
+ "step": 15500
228
+ },
229
+ {
230
+ "epoch": 4.248539564524695,
231
+ "grad_norm": 0.29106882214546204,
232
+ "learning_rate": 2.8763940520446096e-05,
233
+ "loss": 0.6318,
234
+ "step": 16000
235
+ },
236
+ {
237
+ "epoch": 4.381306425916091,
238
+ "grad_norm": 0.2671768069267273,
239
+ "learning_rate": 2.810143388210303e-05,
240
+ "loss": 0.6311,
241
+ "step": 16500
242
+ },
243
+ {
244
+ "epoch": 4.514073287307488,
245
+ "grad_norm": 0.311146080493927,
246
+ "learning_rate": 2.7437599575146044e-05,
247
+ "loss": 0.6243,
248
+ "step": 17000
249
+ },
250
+ {
251
+ "epoch": 4.646840148698884,
252
+ "grad_norm": 0.3101503551006317,
253
+ "learning_rate": 2.677376526818906e-05,
254
+ "loss": 0.6157,
255
+ "step": 17500
256
+ },
257
+ {
258
+ "epoch": 4.779607010090281,
259
+ "grad_norm": 0.3017677366733551,
260
+ "learning_rate": 2.6109930961232075e-05,
261
+ "loss": 0.6086,
262
+ "step": 18000
263
+ },
264
+ {
265
+ "epoch": 4.9123738714816785,
266
+ "grad_norm": 0.31505176424980164,
267
+ "learning_rate": 2.5446096654275092e-05,
268
+ "loss": 0.6146,
269
+ "step": 18500
270
+ },
271
+ {
272
+ "epoch": 5.045140732873075,
273
+ "grad_norm": 0.35715609788894653,
274
+ "learning_rate": 2.4783590015932023e-05,
275
+ "loss": 0.602,
276
+ "step": 19000
277
+ },
278
+ {
279
+ "epoch": 5.177907594264472,
280
+ "grad_norm": 0.2930135726928711,
281
+ "learning_rate": 2.4119755708975043e-05,
282
+ "loss": 0.6024,
283
+ "step": 19500
284
+ },
285
+ {
286
+ "epoch": 5.310674455655868,
287
+ "grad_norm": 0.3474890887737274,
288
+ "learning_rate": 2.3455921402018057e-05,
289
+ "loss": 0.6002,
290
+ "step": 20000
291
+ },
292
+ {
293
+ "epoch": 5.443441317047265,
294
+ "grad_norm": 0.29057538509368896,
295
+ "learning_rate": 2.2792087095061074e-05,
296
+ "loss": 0.6006,
297
+ "step": 20500
298
+ },
299
+ {
300
+ "epoch": 5.5762081784386615,
301
+ "grad_norm": 0.3273596167564392,
302
+ "learning_rate": 2.212825278810409e-05,
303
+ "loss": 0.5912,
304
+ "step": 21000
305
+ },
306
+ {
307
+ "epoch": 5.708975039830058,
308
+ "grad_norm": 0.27121296525001526,
309
+ "learning_rate": 2.146574614976102e-05,
310
+ "loss": 0.593,
311
+ "step": 21500
312
+ },
313
+ {
314
+ "epoch": 5.8417419012214555,
315
+ "grad_norm": 0.29718518257141113,
316
+ "learning_rate": 2.0801911842804035e-05,
317
+ "loss": 0.591,
318
+ "step": 22000
319
+ },
320
+ {
321
+ "epoch": 5.974508762612852,
322
+ "grad_norm": 0.32410022616386414,
323
+ "learning_rate": 2.0138077535847052e-05,
324
+ "loss": 0.5857,
325
+ "step": 22500
326
+ },
327
+ {
328
+ "epoch": 6.107275624004249,
329
+ "grad_norm": 0.2846163213253021,
330
+ "learning_rate": 1.9475570897503983e-05,
331
+ "loss": 0.5802,
332
+ "step": 23000
333
+ },
334
+ {
335
+ "epoch": 6.240042485395645,
336
+ "grad_norm": 0.30459314584732056,
337
+ "learning_rate": 1.8811736590547e-05,
338
+ "loss": 0.5806,
339
+ "step": 23500
340
+ },
341
+ {
342
+ "epoch": 6.372809346787042,
343
+ "grad_norm": 0.5301333069801331,
344
+ "learning_rate": 1.8147902283590017e-05,
345
+ "loss": 0.5789,
346
+ "step": 24000
347
+ },
348
+ {
349
+ "epoch": 6.5055762081784385,
350
+ "grad_norm": 0.2727649509906769,
351
+ "learning_rate": 1.7484067976633034e-05,
352
+ "loss": 0.5783,
353
+ "step": 24500
354
+ },
355
+ {
356
+ "epoch": 6.638343069569835,
357
+ "grad_norm": 0.3003462255001068,
358
+ "learning_rate": 1.682023366967605e-05,
359
+ "loss": 0.5765,
360
+ "step": 25000
361
+ },
362
+ {
363
+ "epoch": 6.771109930961232,
364
+ "grad_norm": 0.35589084029197693,
365
+ "learning_rate": 1.6157727031332982e-05,
366
+ "loss": 0.5787,
367
+ "step": 25500
368
+ },
369
+ {
370
+ "epoch": 6.903876792352628,
371
+ "grad_norm": 0.2738860845565796,
372
+ "learning_rate": 1.5493892724375996e-05,
373
+ "loss": 0.5734,
374
+ "step": 26000
375
+ },
376
+ {
377
+ "epoch": 7.036643653744026,
378
+ "grad_norm": 0.42223164439201355,
379
+ "learning_rate": 1.4830058417419013e-05,
380
+ "loss": 0.5704,
381
+ "step": 26500
382
+ },
383
+ {
384
+ "epoch": 7.169410515135422,
385
+ "grad_norm": 0.2938649654388428,
386
+ "learning_rate": 1.4166224110462028e-05,
387
+ "loss": 0.5686,
388
+ "step": 27000
389
+ },
390
+ {
391
+ "epoch": 7.302177376526819,
392
+ "grad_norm": 0.275078147649765,
393
+ "learning_rate": 1.3503717472118959e-05,
394
+ "loss": 0.5666,
395
+ "step": 27500
396
+ },
397
+ {
398
+ "epoch": 7.434944237918216,
399
+ "grad_norm": 0.35505712032318115,
400
+ "learning_rate": 1.2839883165161976e-05,
401
+ "loss": 0.5631,
402
+ "step": 28000
403
+ },
404
+ {
405
+ "epoch": 7.567711099309612,
406
+ "grad_norm": 0.2507877051830292,
407
+ "learning_rate": 1.2176048858204993e-05,
408
+ "loss": 0.5688,
409
+ "step": 28500
410
+ },
411
+ {
412
+ "epoch": 7.700477960701009,
413
+ "grad_norm": 0.2846459746360779,
414
+ "learning_rate": 1.1512214551248008e-05,
415
+ "loss": 0.5594,
416
+ "step": 29000
417
+ },
418
+ {
419
+ "epoch": 7.833244822092405,
420
+ "grad_norm": 0.31158626079559326,
421
+ "learning_rate": 1.0848380244291025e-05,
422
+ "loss": 0.5653,
423
+ "step": 29500
424
+ },
425
+ {
426
+ "epoch": 7.966011683483803,
427
+ "grad_norm": 0.2899467647075653,
428
+ "learning_rate": 1.0184545937334042e-05,
429
+ "loss": 0.562,
430
+ "step": 30000
431
+ },
432
+ {
433
+ "epoch": 8.098778544875199,
434
+ "grad_norm": 0.27900761365890503,
435
+ "learning_rate": 9.520711630377058e-06,
436
+ "loss": 0.559,
437
+ "step": 30500
438
+ },
439
+ {
440
+ "epoch": 8.231545406266596,
441
+ "grad_norm": 0.29301363229751587,
442
+ "learning_rate": 8.856877323420075e-06,
443
+ "loss": 0.5604,
444
+ "step": 31000
445
+ },
446
+ {
447
+ "epoch": 8.364312267657992,
448
+ "grad_norm": 0.2812318801879883,
449
+ "learning_rate": 8.19304301646309e-06,
450
+ "loss": 0.559,
451
+ "step": 31500
452
+ },
453
+ {
454
+ "epoch": 8.49707912904939,
455
+ "grad_norm": 0.27755317091941833,
456
+ "learning_rate": 7.530536378120022e-06,
457
+ "loss": 0.5527,
458
+ "step": 32000
459
+ },
460
+ {
461
+ "epoch": 8.629845990440787,
462
+ "grad_norm": 0.3323802053928375,
463
+ "learning_rate": 6.866702071163038e-06,
464
+ "loss": 0.5576,
465
+ "step": 32500
466
+ },
467
+ {
468
+ "epoch": 8.762612851832182,
469
+ "grad_norm": 0.2453739196062088,
470
+ "learning_rate": 6.202867764206054e-06,
471
+ "loss": 0.557,
472
+ "step": 33000
473
+ },
474
+ {
475
+ "epoch": 8.89537971322358,
476
+ "grad_norm": 0.28488314151763916,
477
+ "learning_rate": 5.539033457249071e-06,
478
+ "loss": 0.5586,
479
+ "step": 33500
480
+ },
481
+ {
482
+ "epoch": 9.028146574614976,
483
+ "grad_norm": 0.2731677293777466,
484
+ "learning_rate": 4.876526818906001e-06,
485
+ "loss": 0.5529,
486
+ "step": 34000
487
+ },
488
+ {
489
+ "epoch": 9.160913436006373,
490
+ "grad_norm": 0.34274822473526,
491
+ "learning_rate": 4.214020180562932e-06,
492
+ "loss": 0.5562,
493
+ "step": 34500
494
+ },
495
+ {
496
+ "epoch": 9.293680297397769,
497
+ "grad_norm": 0.2875533103942871,
498
+ "learning_rate": 3.550185873605948e-06,
499
+ "loss": 0.5528,
500
+ "step": 35000
501
+ },
502
+ {
503
+ "epoch": 9.426447158789166,
504
+ "grad_norm": 0.2516155242919922,
505
+ "learning_rate": 2.8863515666489647e-06,
506
+ "loss": 0.5538,
507
+ "step": 35500
508
+ },
509
+ {
510
+ "epoch": 9.559214020180562,
511
+ "grad_norm": 0.2524682283401489,
512
+ "learning_rate": 2.222517259691981e-06,
513
+ "loss": 0.5546,
514
+ "step": 36000
515
+ },
516
+ {
517
+ "epoch": 9.69198088157196,
518
+ "grad_norm": 0.25429150462150574,
519
+ "learning_rate": 1.5586829527349974e-06,
520
+ "loss": 0.5537,
521
+ "step": 36500
522
+ },
523
+ {
524
+ "epoch": 9.824747742963357,
525
+ "grad_norm": 0.2699441611766815,
526
+ "learning_rate": 8.948486457780139e-07,
527
+ "loss": 0.5522,
528
+ "step": 37000
529
+ },
530
+ {
531
+ "epoch": 9.957514604354753,
532
+ "grad_norm": 0.35291793942451477,
533
+ "learning_rate": 2.3101433882103027e-07,
534
+ "loss": 0.5493,
535
+ "step": 37500
536
+ }
537
+ ],
538
+ "logging_steps": 500,
539
+ "max_steps": 37660,
540
+ "num_input_tokens_seen": 0,
541
+ "num_train_epochs": 10,
542
+ "save_steps": 500,
543
+ "stateful_callbacks": {
544
+ "TrainerControl": {
545
+ "args": {
546
+ "should_epoch_stop": false,
547
+ "should_evaluate": false,
548
+ "should_log": false,
549
+ "should_save": true,
550
+ "should_training_stop": false
551
+ },
552
+ "attributes": {}
553
+ }
554
+ },
555
+ "total_flos": 8.119594531160064e+16,
556
+ "train_batch_size": 32,
557
+ "trial_name": null,
558
+ "trial_params": null
559
+ }
checkpoints/{checkpoint-8500 → checkpoint-37500}/training_args.bin RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2067c784529f8fe2c2e4f09f57bcd2f668958cc835cfbd5a037d77580dd52cb6
3
  size 5240
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:344c4c96587b5961c7260f4c9743524e13a2580c248f0b960480131c9cd7dc77
3
  size 5240
checkpoints/{checkpoint-9410 → checkpoint-37660}/config.json RENAMED
File without changes
checkpoints/{checkpoint-9410 → checkpoint-37660}/generation_config.json RENAMED
File without changes
checkpoints/{checkpoint-9410 → checkpoint-37660}/model.safetensors RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0cee34848014a7023f7c44bcd836df67292a4f1655c491368c20d8d7294b9ebb
3
  size 242041896
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b78f8b4750e01e9e992daf6187c84c8ecfad35d6183f0336f1d4661a41534df9
3
  size 242041896
checkpoints/{checkpoint-9410 → checkpoint-37660}/optimizer.pt RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:672e63dc4c447d76eec534a78581bf8e4109db5311b1e83a2718dccd669c5aed
3
  size 484163514
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:866ce67c1ca64817c5bedb2cd226dd6b42eb9c6b9a197984e5c72c49f3fad0b4
3
  size 484163514
checkpoints/{checkpoint-8500 → checkpoint-37660}/rng_state.pth RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8cff220eace7c69859d8fbd62a4c67d275aeb247933de3ec0ca02703f87351d0
3
  size 14244
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c93c42240bdb62b933fb7098be37ca6ca9d1c6a51fc610080600494b7e572605
3
  size 14244
checkpoints/{checkpoint-8500 → checkpoint-37660}/scaler.pt RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7783f9544b279d0ae395116a4ef839913989d191585a47871c5d7d34778c019f
3
  size 988
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a1b02513987752f55c2a09cb46fb561fd6490aaf9c1a9fb121a08671c8653dcd
3
  size 988
checkpoints/{checkpoint-9410 → checkpoint-37660}/scheduler.pt RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ec0019f5b0ff4782538d43eaea19125b6d8430858f1e93f4e77e38ca6d897aa9
3
  size 1064
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:137105948ee6cac22e2b0464aacbd6b0a4d90a8f9ac6be52aa48e82e061b5d09
3
  size 1064
checkpoints/{checkpoint-9410 → checkpoint-37660}/special_tokens_map.json RENAMED
File without changes
checkpoints/{checkpoint-9410 → checkpoint-37660}/tokenizer.json RENAMED
File without changes
checkpoints/{checkpoint-9410 → checkpoint-37660}/tokenizer_config.json RENAMED
File without changes
checkpoints/checkpoint-37660/trainer_state.json ADDED
@@ -0,0 +1,559 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": null,
3
+ "best_metric": null,
4
+ "best_model_checkpoint": null,
5
+ "epoch": 10.0,
6
+ "eval_steps": 500,
7
+ "global_step": 37660,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.1327668613913967,
14
+ "grad_norm": 0.7285390496253967,
15
+ "learning_rate": 4.934014869888476e-05,
16
+ "loss": 2.9202,
17
+ "step": 500
18
+ },
19
+ {
20
+ "epoch": 0.2655337227827934,
21
+ "grad_norm": 0.6645896434783936,
22
+ "learning_rate": 4.867631439192778e-05,
23
+ "loss": 1.7398,
24
+ "step": 1000
25
+ },
26
+ {
27
+ "epoch": 0.3983005841741901,
28
+ "grad_norm": 1.236431360244751,
29
+ "learning_rate": 4.8012480084970795e-05,
30
+ "loss": 1.4128,
31
+ "step": 1500
32
+ },
33
+ {
34
+ "epoch": 0.5310674455655868,
35
+ "grad_norm": 0.4946064054965973,
36
+ "learning_rate": 4.734864577801381e-05,
37
+ "loss": 1.2492,
38
+ "step": 2000
39
+ },
40
+ {
41
+ "epoch": 0.6638343069569835,
42
+ "grad_norm": 0.4114533066749573,
43
+ "learning_rate": 4.668481147105683e-05,
44
+ "loss": 1.1439,
45
+ "step": 2500
46
+ },
47
+ {
48
+ "epoch": 0.7966011683483802,
49
+ "grad_norm": 0.4554953873157501,
50
+ "learning_rate": 4.602097716409984e-05,
51
+ "loss": 1.0689,
52
+ "step": 3000
53
+ },
54
+ {
55
+ "epoch": 0.929368029739777,
56
+ "grad_norm": 0.41119593381881714,
57
+ "learning_rate": 4.5357142857142856e-05,
58
+ "loss": 1.0057,
59
+ "step": 3500
60
+ },
61
+ {
62
+ "epoch": 1.0621348911311737,
63
+ "grad_norm": 0.40032336115837097,
64
+ "learning_rate": 4.4693308550185877e-05,
65
+ "loss": 0.9672,
66
+ "step": 4000
67
+ },
68
+ {
69
+ "epoch": 1.1949017525225702,
70
+ "grad_norm": 0.3746163845062256,
71
+ "learning_rate": 4.402947424322889e-05,
72
+ "loss": 0.9234,
73
+ "step": 4500
74
+ },
75
+ {
76
+ "epoch": 1.327668613913967,
77
+ "grad_norm": 0.4201946556568146,
78
+ "learning_rate": 4.3365639936271904e-05,
79
+ "loss": 0.8978,
80
+ "step": 5000
81
+ },
82
+ {
83
+ "epoch": 1.4604354753053639,
84
+ "grad_norm": 0.48306697607040405,
85
+ "learning_rate": 4.2701805629314924e-05,
86
+ "loss": 0.8672,
87
+ "step": 5500
88
+ },
89
+ {
90
+ "epoch": 1.5932023366967605,
91
+ "grad_norm": 0.348651647567749,
92
+ "learning_rate": 4.2037971322357945e-05,
93
+ "loss": 0.8373,
94
+ "step": 6000
95
+ },
96
+ {
97
+ "epoch": 1.725969198088157,
98
+ "grad_norm": 0.3810581862926483,
99
+ "learning_rate": 4.137413701540096e-05,
100
+ "loss": 0.8161,
101
+ "step": 6500
102
+ },
103
+ {
104
+ "epoch": 1.858736059479554,
105
+ "grad_norm": 0.41756191849708557,
106
+ "learning_rate": 4.071030270844398e-05,
107
+ "loss": 0.8011,
108
+ "step": 7000
109
+ },
110
+ {
111
+ "epoch": 1.9915029208709507,
112
+ "grad_norm": 0.3746052384376526,
113
+ "learning_rate": 4.004646840148699e-05,
114
+ "loss": 0.7917,
115
+ "step": 7500
116
+ },
117
+ {
118
+ "epoch": 2.1242697822623473,
119
+ "grad_norm": 0.3314072787761688,
120
+ "learning_rate": 3.9382634094530006e-05,
121
+ "loss": 0.772,
122
+ "step": 8000
123
+ },
124
+ {
125
+ "epoch": 2.257036643653744,
126
+ "grad_norm": 0.36195287108421326,
127
+ "learning_rate": 3.871879978757303e-05,
128
+ "loss": 0.7519,
129
+ "step": 8500
130
+ },
131
+ {
132
+ "epoch": 2.3898035050451405,
133
+ "grad_norm": 0.36162152886390686,
134
+ "learning_rate": 3.805496548061604e-05,
135
+ "loss": 0.744,
136
+ "step": 9000
137
+ },
138
+ {
139
+ "epoch": 2.5225703664365375,
140
+ "grad_norm": 0.3393162786960602,
141
+ "learning_rate": 3.739245884227297e-05,
142
+ "loss": 0.7317,
143
+ "step": 9500
144
+ },
145
+ {
146
+ "epoch": 2.655337227827934,
147
+ "grad_norm": 0.42738890647888184,
148
+ "learning_rate": 3.67299522039299e-05,
149
+ "loss": 0.7227,
150
+ "step": 10000
151
+ },
152
+ {
153
+ "epoch": 2.7881040892193307,
154
+ "grad_norm": 0.34136396646499634,
155
+ "learning_rate": 3.6066117896972915e-05,
156
+ "loss": 0.7084,
157
+ "step": 10500
158
+ },
159
+ {
160
+ "epoch": 2.9208709506107278,
161
+ "grad_norm": 0.33245041966438293,
162
+ "learning_rate": 3.5402283590015936e-05,
163
+ "loss": 0.6993,
164
+ "step": 11000
165
+ },
166
+ {
167
+ "epoch": 3.0536378120021244,
168
+ "grad_norm": 0.46563392877578735,
169
+ "learning_rate": 3.473844928305895e-05,
170
+ "loss": 0.6865,
171
+ "step": 11500
172
+ },
173
+ {
174
+ "epoch": 3.186404673393521,
175
+ "grad_norm": 0.3868368864059448,
176
+ "learning_rate": 3.407461497610196e-05,
177
+ "loss": 0.6822,
178
+ "step": 12000
179
+ },
180
+ {
181
+ "epoch": 3.3191715347849176,
182
+ "grad_norm": 0.30418872833251953,
183
+ "learning_rate": 3.3410780669144984e-05,
184
+ "loss": 0.6775,
185
+ "step": 12500
186
+ },
187
+ {
188
+ "epoch": 3.451938396176314,
189
+ "grad_norm": 0.34485992789268494,
190
+ "learning_rate": 3.2746946362188e-05,
191
+ "loss": 0.6713,
192
+ "step": 13000
193
+ },
194
+ {
195
+ "epoch": 3.584705257567711,
196
+ "grad_norm": 0.33921709656715393,
197
+ "learning_rate": 3.208311205523102e-05,
198
+ "loss": 0.658,
199
+ "step": 13500
200
+ },
201
+ {
202
+ "epoch": 3.717472118959108,
203
+ "grad_norm": 0.36646100878715515,
204
+ "learning_rate": 3.141927774827403e-05,
205
+ "loss": 0.6516,
206
+ "step": 14000
207
+ },
208
+ {
209
+ "epoch": 3.8502389803505044,
210
+ "grad_norm": 0.32367828488349915,
211
+ "learning_rate": 3.075544344131705e-05,
212
+ "loss": 0.6479,
213
+ "step": 14500
214
+ },
215
+ {
216
+ "epoch": 3.9830058417419014,
217
+ "grad_norm": 0.32735565304756165,
218
+ "learning_rate": 3.0091609134360066e-05,
219
+ "loss": 0.6423,
220
+ "step": 15000
221
+ },
222
+ {
223
+ "epoch": 4.115772703133298,
224
+ "grad_norm": 0.43194663524627686,
225
+ "learning_rate": 2.9427774827403083e-05,
226
+ "loss": 0.6416,
227
+ "step": 15500
228
+ },
229
+ {
230
+ "epoch": 4.248539564524695,
231
+ "grad_norm": 0.29106882214546204,
232
+ "learning_rate": 2.8763940520446096e-05,
233
+ "loss": 0.6318,
234
+ "step": 16000
235
+ },
236
+ {
237
+ "epoch": 4.381306425916091,
238
+ "grad_norm": 0.2671768069267273,
239
+ "learning_rate": 2.810143388210303e-05,
240
+ "loss": 0.6311,
241
+ "step": 16500
242
+ },
243
+ {
244
+ "epoch": 4.514073287307488,
245
+ "grad_norm": 0.311146080493927,
246
+ "learning_rate": 2.7437599575146044e-05,
247
+ "loss": 0.6243,
248
+ "step": 17000
249
+ },
250
+ {
251
+ "epoch": 4.646840148698884,
252
+ "grad_norm": 0.3101503551006317,
253
+ "learning_rate": 2.677376526818906e-05,
254
+ "loss": 0.6157,
255
+ "step": 17500
256
+ },
257
+ {
258
+ "epoch": 4.779607010090281,
259
+ "grad_norm": 0.3017677366733551,
260
+ "learning_rate": 2.6109930961232075e-05,
261
+ "loss": 0.6086,
262
+ "step": 18000
263
+ },
264
+ {
265
+ "epoch": 4.9123738714816785,
266
+ "grad_norm": 0.31505176424980164,
267
+ "learning_rate": 2.5446096654275092e-05,
268
+ "loss": 0.6146,
269
+ "step": 18500
270
+ },
271
+ {
272
+ "epoch": 5.045140732873075,
273
+ "grad_norm": 0.35715609788894653,
274
+ "learning_rate": 2.4783590015932023e-05,
275
+ "loss": 0.602,
276
+ "step": 19000
277
+ },
278
+ {
279
+ "epoch": 5.177907594264472,
280
+ "grad_norm": 0.2930135726928711,
281
+ "learning_rate": 2.4119755708975043e-05,
282
+ "loss": 0.6024,
283
+ "step": 19500
284
+ },
285
+ {
286
+ "epoch": 5.310674455655868,
287
+ "grad_norm": 0.3474890887737274,
288
+ "learning_rate": 2.3455921402018057e-05,
289
+ "loss": 0.6002,
290
+ "step": 20000
291
+ },
292
+ {
293
+ "epoch": 5.443441317047265,
294
+ "grad_norm": 0.29057538509368896,
295
+ "learning_rate": 2.2792087095061074e-05,
296
+ "loss": 0.6006,
297
+ "step": 20500
298
+ },
299
+ {
300
+ "epoch": 5.5762081784386615,
301
+ "grad_norm": 0.3273596167564392,
302
+ "learning_rate": 2.212825278810409e-05,
303
+ "loss": 0.5912,
304
+ "step": 21000
305
+ },
306
+ {
307
+ "epoch": 5.708975039830058,
308
+ "grad_norm": 0.27121296525001526,
309
+ "learning_rate": 2.146574614976102e-05,
310
+ "loss": 0.593,
311
+ "step": 21500
312
+ },
313
+ {
314
+ "epoch": 5.8417419012214555,
315
+ "grad_norm": 0.29718518257141113,
316
+ "learning_rate": 2.0801911842804035e-05,
317
+ "loss": 0.591,
318
+ "step": 22000
319
+ },
320
+ {
321
+ "epoch": 5.974508762612852,
322
+ "grad_norm": 0.32410022616386414,
323
+ "learning_rate": 2.0138077535847052e-05,
324
+ "loss": 0.5857,
325
+ "step": 22500
326
+ },
327
+ {
328
+ "epoch": 6.107275624004249,
329
+ "grad_norm": 0.2846163213253021,
330
+ "learning_rate": 1.9475570897503983e-05,
331
+ "loss": 0.5802,
332
+ "step": 23000
333
+ },
334
+ {
335
+ "epoch": 6.240042485395645,
336
+ "grad_norm": 0.30459314584732056,
337
+ "learning_rate": 1.8811736590547e-05,
338
+ "loss": 0.5806,
339
+ "step": 23500
340
+ },
341
+ {
342
+ "epoch": 6.372809346787042,
343
+ "grad_norm": 0.5301333069801331,
344
+ "learning_rate": 1.8147902283590017e-05,
345
+ "loss": 0.5789,
346
+ "step": 24000
347
+ },
348
+ {
349
+ "epoch": 6.5055762081784385,
350
+ "grad_norm": 0.2727649509906769,
351
+ "learning_rate": 1.7484067976633034e-05,
352
+ "loss": 0.5783,
353
+ "step": 24500
354
+ },
355
+ {
356
+ "epoch": 6.638343069569835,
357
+ "grad_norm": 0.3003462255001068,
358
+ "learning_rate": 1.682023366967605e-05,
359
+ "loss": 0.5765,
360
+ "step": 25000
361
+ },
362
+ {
363
+ "epoch": 6.771109930961232,
364
+ "grad_norm": 0.35589084029197693,
365
+ "learning_rate": 1.6157727031332982e-05,
366
+ "loss": 0.5787,
367
+ "step": 25500
368
+ },
369
+ {
370
+ "epoch": 6.903876792352628,
371
+ "grad_norm": 0.2738860845565796,
372
+ "learning_rate": 1.5493892724375996e-05,
373
+ "loss": 0.5734,
374
+ "step": 26000
375
+ },
376
+ {
377
+ "epoch": 7.036643653744026,
378
+ "grad_norm": 0.42223164439201355,
379
+ "learning_rate": 1.4830058417419013e-05,
380
+ "loss": 0.5704,
381
+ "step": 26500
382
+ },
383
+ {
384
+ "epoch": 7.169410515135422,
385
+ "grad_norm": 0.2938649654388428,
386
+ "learning_rate": 1.4166224110462028e-05,
387
+ "loss": 0.5686,
388
+ "step": 27000
389
+ },
390
+ {
391
+ "epoch": 7.302177376526819,
392
+ "grad_norm": 0.275078147649765,
393
+ "learning_rate": 1.3503717472118959e-05,
394
+ "loss": 0.5666,
395
+ "step": 27500
396
+ },
397
+ {
398
+ "epoch": 7.434944237918216,
399
+ "grad_norm": 0.35505712032318115,
400
+ "learning_rate": 1.2839883165161976e-05,
401
+ "loss": 0.5631,
402
+ "step": 28000
403
+ },
404
+ {
405
+ "epoch": 7.567711099309612,
406
+ "grad_norm": 0.2507877051830292,
407
+ "learning_rate": 1.2176048858204993e-05,
408
+ "loss": 0.5688,
409
+ "step": 28500
410
+ },
411
+ {
412
+ "epoch": 7.700477960701009,
413
+ "grad_norm": 0.2846459746360779,
414
+ "learning_rate": 1.1512214551248008e-05,
415
+ "loss": 0.5594,
416
+ "step": 29000
417
+ },
418
+ {
419
+ "epoch": 7.833244822092405,
420
+ "grad_norm": 0.31158626079559326,
421
+ "learning_rate": 1.0848380244291025e-05,
422
+ "loss": 0.5653,
423
+ "step": 29500
424
+ },
425
+ {
426
+ "epoch": 7.966011683483803,
427
+ "grad_norm": 0.2899467647075653,
428
+ "learning_rate": 1.0184545937334042e-05,
429
+ "loss": 0.562,
430
+ "step": 30000
431
+ },
432
+ {
433
+ "epoch": 8.098778544875199,
434
+ "grad_norm": 0.27900761365890503,
435
+ "learning_rate": 9.520711630377058e-06,
436
+ "loss": 0.559,
437
+ "step": 30500
438
+ },
439
+ {
440
+ "epoch": 8.231545406266596,
441
+ "grad_norm": 0.29301363229751587,
442
+ "learning_rate": 8.856877323420075e-06,
443
+ "loss": 0.5604,
444
+ "step": 31000
445
+ },
446
+ {
447
+ "epoch": 8.364312267657992,
448
+ "grad_norm": 0.2812318801879883,
449
+ "learning_rate": 8.19304301646309e-06,
450
+ "loss": 0.559,
451
+ "step": 31500
452
+ },
453
+ {
454
+ "epoch": 8.49707912904939,
455
+ "grad_norm": 0.27755317091941833,
456
+ "learning_rate": 7.530536378120022e-06,
457
+ "loss": 0.5527,
458
+ "step": 32000
459
+ },
460
+ {
461
+ "epoch": 8.629845990440787,
462
+ "grad_norm": 0.3323802053928375,
463
+ "learning_rate": 6.866702071163038e-06,
464
+ "loss": 0.5576,
465
+ "step": 32500
466
+ },
467
+ {
468
+ "epoch": 8.762612851832182,
469
+ "grad_norm": 0.2453739196062088,
470
+ "learning_rate": 6.202867764206054e-06,
471
+ "loss": 0.557,
472
+ "step": 33000
473
+ },
474
+ {
475
+ "epoch": 8.89537971322358,
476
+ "grad_norm": 0.28488314151763916,
477
+ "learning_rate": 5.539033457249071e-06,
478
+ "loss": 0.5586,
479
+ "step": 33500
480
+ },
481
+ {
482
+ "epoch": 9.028146574614976,
483
+ "grad_norm": 0.2731677293777466,
484
+ "learning_rate": 4.876526818906001e-06,
485
+ "loss": 0.5529,
486
+ "step": 34000
487
+ },
488
+ {
489
+ "epoch": 9.160913436006373,
490
+ "grad_norm": 0.34274822473526,
491
+ "learning_rate": 4.214020180562932e-06,
492
+ "loss": 0.5562,
493
+ "step": 34500
494
+ },
495
+ {
496
+ "epoch": 9.293680297397769,
497
+ "grad_norm": 0.2875533103942871,
498
+ "learning_rate": 3.550185873605948e-06,
499
+ "loss": 0.5528,
500
+ "step": 35000
501
+ },
502
+ {
503
+ "epoch": 9.426447158789166,
504
+ "grad_norm": 0.2516155242919922,
505
+ "learning_rate": 2.8863515666489647e-06,
506
+ "loss": 0.5538,
507
+ "step": 35500
508
+ },
509
+ {
510
+ "epoch": 9.559214020180562,
511
+ "grad_norm": 0.2524682283401489,
512
+ "learning_rate": 2.222517259691981e-06,
513
+ "loss": 0.5546,
514
+ "step": 36000
515
+ },
516
+ {
517
+ "epoch": 9.69198088157196,
518
+ "grad_norm": 0.25429150462150574,
519
+ "learning_rate": 1.5586829527349974e-06,
520
+ "loss": 0.5537,
521
+ "step": 36500
522
+ },
523
+ {
524
+ "epoch": 9.824747742963357,
525
+ "grad_norm": 0.2699441611766815,
526
+ "learning_rate": 8.948486457780139e-07,
527
+ "loss": 0.5522,
528
+ "step": 37000
529
+ },
530
+ {
531
+ "epoch": 9.957514604354753,
532
+ "grad_norm": 0.35291793942451477,
533
+ "learning_rate": 2.3101433882103027e-07,
534
+ "loss": 0.5493,
535
+ "step": 37500
536
+ }
537
+ ],
538
+ "logging_steps": 500,
539
+ "max_steps": 37660,
540
+ "num_input_tokens_seen": 0,
541
+ "num_train_epochs": 10,
542
+ "save_steps": 500,
543
+ "stateful_callbacks": {
544
+ "TrainerControl": {
545
+ "args": {
546
+ "should_epoch_stop": false,
547
+ "should_evaluate": false,
548
+ "should_log": false,
549
+ "should_save": true,
550
+ "should_training_stop": true
551
+ },
552
+ "attributes": {}
553
+ }
554
+ },
555
+ "total_flos": 8.154140525985792e+16,
556
+ "train_batch_size": 32,
557
+ "trial_name": null,
558
+ "trial_params": null
559
+ }
checkpoints/{checkpoint-9410 → checkpoint-37660}/training_args.bin RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2067c784529f8fe2c2e4f09f57bcd2f668958cc835cfbd5a037d77580dd52cb6
3
  size 5240
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:344c4c96587b5961c7260f4c9743524e13a2580c248f0b960480131c9cd7dc77
3
  size 5240
checkpoints/checkpoint-8500/trainer_state.json DELETED
@@ -1,153 +0,0 @@
1
- {
2
- "best_global_step": null,
3
- "best_metric": null,
4
- "best_model_checkpoint": null,
5
- "epoch": 4.516231826329417,
6
- "eval_steps": 500,
7
- "global_step": 8500,
8
- "is_hyper_param_search": false,
9
- "is_local_process_zero": true,
10
- "is_world_process_zero": true,
11
- "log_history": [
12
- {
13
- "epoch": 0.26555135099249816,
14
- "grad_norm": 0.47810718417167664,
15
- "learning_rate": 4.736450584484591e-05,
16
- "loss": 2.7717,
17
- "step": 500
18
- },
19
- {
20
- "epoch": 0.5311027019849963,
21
- "grad_norm": 0.413313627243042,
22
- "learning_rate": 4.470775770456961e-05,
23
- "loss": 1.591,
24
- "step": 1000
25
- },
26
- {
27
- "epoch": 0.7966540529774945,
28
- "grad_norm": 0.31432437896728516,
29
- "learning_rate": 4.205100956429331e-05,
30
- "loss": 1.3043,
31
- "step": 1500
32
- },
33
- {
34
- "epoch": 1.0626701188342296,
35
- "grad_norm": 0.36749106645584106,
36
- "learning_rate": 3.9394261424017006e-05,
37
- "loss": 1.1595,
38
- "step": 2000
39
- },
40
- {
41
- "epoch": 1.3282214698267278,
42
- "grad_norm": 0.29507818818092346,
43
- "learning_rate": 3.6737513283740703e-05,
44
- "loss": 1.0709,
45
- "step": 2500
46
- },
47
- {
48
- "epoch": 1.5937728208192259,
49
- "grad_norm": 0.3084051012992859,
50
- "learning_rate": 3.40807651434644e-05,
51
- "loss": 1.005,
52
- "step": 3000
53
- },
54
- {
55
- "epoch": 1.859324171811724,
56
- "grad_norm": 0.2812994718551636,
57
- "learning_rate": 3.14240170031881e-05,
58
- "loss": 0.9559,
59
- "step": 3500
60
- },
61
- {
62
- "epoch": 2.125340237668459,
63
- "grad_norm": 0.3291400074958801,
64
- "learning_rate": 2.8767268862911793e-05,
65
- "loss": 0.9273,
66
- "step": 4000
67
- },
68
- {
69
- "epoch": 2.390891588660957,
70
- "grad_norm": 0.28141409158706665,
71
- "learning_rate": 2.6110520722635495e-05,
72
- "loss": 0.8928,
73
- "step": 4500
74
- },
75
- {
76
- "epoch": 2.6564429396534557,
77
- "grad_norm": 0.2757486402988434,
78
- "learning_rate": 2.3453772582359192e-05,
79
- "loss": 0.8714,
80
- "step": 5000
81
- },
82
- {
83
- "epoch": 2.9219942906459537,
84
- "grad_norm": 0.2808317542076111,
85
- "learning_rate": 2.079702444208289e-05,
86
- "loss": 0.8498,
87
- "step": 5500
88
- },
89
- {
90
- "epoch": 3.188010356502689,
91
- "grad_norm": 0.25909456610679626,
92
- "learning_rate": 1.8140276301806588e-05,
93
- "loss": 0.8322,
94
- "step": 6000
95
- },
96
- {
97
- "epoch": 3.453561707495187,
98
- "grad_norm": 0.2701779901981354,
99
- "learning_rate": 1.5483528161530286e-05,
100
- "loss": 0.8248,
101
- "step": 6500
102
- },
103
- {
104
- "epoch": 3.719113058487685,
105
- "grad_norm": 0.2889029085636139,
106
- "learning_rate": 1.2826780021253987e-05,
107
- "loss": 0.8087,
108
- "step": 7000
109
- },
110
- {
111
- "epoch": 3.984664409480183,
112
- "grad_norm": 0.2585383653640747,
113
- "learning_rate": 1.0170031880977683e-05,
114
- "loss": 0.8027,
115
- "step": 7500
116
- },
117
- {
118
- "epoch": 4.250680475336918,
119
- "grad_norm": 0.2610950171947479,
120
- "learning_rate": 7.513283740701383e-06,
121
- "loss": 0.7994,
122
- "step": 8000
123
- },
124
- {
125
- "epoch": 4.516231826329417,
126
- "grad_norm": 0.27488183975219727,
127
- "learning_rate": 4.85653560042508e-06,
128
- "loss": 0.7947,
129
- "step": 8500
130
- }
131
- ],
132
- "logging_steps": 500,
133
- "max_steps": 9410,
134
- "num_input_tokens_seen": 0,
135
- "num_train_epochs": 5,
136
- "save_steps": 500,
137
- "stateful_callbacks": {
138
- "TrainerControl": {
139
- "args": {
140
- "should_epoch_stop": false,
141
- "should_evaluate": false,
142
- "should_log": false,
143
- "should_save": true,
144
- "should_training_stop": false
145
- },
146
- "attributes": {}
147
- }
148
- },
149
- "total_flos": 3.68173009380311e+16,
150
- "train_batch_size": 16,
151
- "trial_name": null,
152
- "trial_params": null
153
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoints/checkpoint-9000/trainer_state.json DELETED
@@ -1,160 +0,0 @@
1
- {
2
- "best_global_step": null,
3
- "best_metric": null,
4
- "best_model_checkpoint": null,
5
- "epoch": 4.781783177321914,
6
- "eval_steps": 500,
7
- "global_step": 9000,
8
- "is_hyper_param_search": false,
9
- "is_local_process_zero": true,
10
- "is_world_process_zero": true,
11
- "log_history": [
12
- {
13
- "epoch": 0.26555135099249816,
14
- "grad_norm": 0.47810718417167664,
15
- "learning_rate": 4.736450584484591e-05,
16
- "loss": 2.7717,
17
- "step": 500
18
- },
19
- {
20
- "epoch": 0.5311027019849963,
21
- "grad_norm": 0.413313627243042,
22
- "learning_rate": 4.470775770456961e-05,
23
- "loss": 1.591,
24
- "step": 1000
25
- },
26
- {
27
- "epoch": 0.7966540529774945,
28
- "grad_norm": 0.31432437896728516,
29
- "learning_rate": 4.205100956429331e-05,
30
- "loss": 1.3043,
31
- "step": 1500
32
- },
33
- {
34
- "epoch": 1.0626701188342296,
35
- "grad_norm": 0.36749106645584106,
36
- "learning_rate": 3.9394261424017006e-05,
37
- "loss": 1.1595,
38
- "step": 2000
39
- },
40
- {
41
- "epoch": 1.3282214698267278,
42
- "grad_norm": 0.29507818818092346,
43
- "learning_rate": 3.6737513283740703e-05,
44
- "loss": 1.0709,
45
- "step": 2500
46
- },
47
- {
48
- "epoch": 1.5937728208192259,
49
- "grad_norm": 0.3084051012992859,
50
- "learning_rate": 3.40807651434644e-05,
51
- "loss": 1.005,
52
- "step": 3000
53
- },
54
- {
55
- "epoch": 1.859324171811724,
56
- "grad_norm": 0.2812994718551636,
57
- "learning_rate": 3.14240170031881e-05,
58
- "loss": 0.9559,
59
- "step": 3500
60
- },
61
- {
62
- "epoch": 2.125340237668459,
63
- "grad_norm": 0.3291400074958801,
64
- "learning_rate": 2.8767268862911793e-05,
65
- "loss": 0.9273,
66
- "step": 4000
67
- },
68
- {
69
- "epoch": 2.390891588660957,
70
- "grad_norm": 0.28141409158706665,
71
- "learning_rate": 2.6110520722635495e-05,
72
- "loss": 0.8928,
73
- "step": 4500
74
- },
75
- {
76
- "epoch": 2.6564429396534557,
77
- "grad_norm": 0.2757486402988434,
78
- "learning_rate": 2.3453772582359192e-05,
79
- "loss": 0.8714,
80
- "step": 5000
81
- },
82
- {
83
- "epoch": 2.9219942906459537,
84
- "grad_norm": 0.2808317542076111,
85
- "learning_rate": 2.079702444208289e-05,
86
- "loss": 0.8498,
87
- "step": 5500
88
- },
89
- {
90
- "epoch": 3.188010356502689,
91
- "grad_norm": 0.25909456610679626,
92
- "learning_rate": 1.8140276301806588e-05,
93
- "loss": 0.8322,
94
- "step": 6000
95
- },
96
- {
97
- "epoch": 3.453561707495187,
98
- "grad_norm": 0.2701779901981354,
99
- "learning_rate": 1.5483528161530286e-05,
100
- "loss": 0.8248,
101
- "step": 6500
102
- },
103
- {
104
- "epoch": 3.719113058487685,
105
- "grad_norm": 0.2889029085636139,
106
- "learning_rate": 1.2826780021253987e-05,
107
- "loss": 0.8087,
108
- "step": 7000
109
- },
110
- {
111
- "epoch": 3.984664409480183,
112
- "grad_norm": 0.2585383653640747,
113
- "learning_rate": 1.0170031880977683e-05,
114
- "loss": 0.8027,
115
- "step": 7500
116
- },
117
- {
118
- "epoch": 4.250680475336918,
119
- "grad_norm": 0.2610950171947479,
120
- "learning_rate": 7.513283740701383e-06,
121
- "loss": 0.7994,
122
- "step": 8000
123
- },
124
- {
125
- "epoch": 4.516231826329417,
126
- "grad_norm": 0.27488183975219727,
127
- "learning_rate": 4.85653560042508e-06,
128
- "loss": 0.7947,
129
- "step": 8500
130
- },
131
- {
132
- "epoch": 4.781783177321914,
133
- "grad_norm": 0.22057265043258667,
134
- "learning_rate": 2.1997874601487783e-06,
135
- "loss": 0.7842,
136
- "step": 9000
137
- }
138
- ],
139
- "logging_steps": 500,
140
- "max_steps": 9410,
141
- "num_input_tokens_seen": 0,
142
- "num_train_epochs": 5,
143
- "save_steps": 500,
144
- "stateful_callbacks": {
145
- "TrainerControl": {
146
- "args": {
147
- "should_epoch_stop": false,
148
- "should_evaluate": false,
149
- "should_log": false,
150
- "should_save": true,
151
- "should_training_stop": false
152
- },
153
- "attributes": {}
154
- }
155
- },
156
- "total_flos": 3.89827697615831e+16,
157
- "train_batch_size": 16,
158
- "trial_name": null,
159
- "trial_params": null
160
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoints/checkpoint-9410/trainer_state.json DELETED
@@ -1,160 +0,0 @@
1
- {
2
- "best_global_step": null,
3
- "best_metric": null,
4
- "best_model_checkpoint": null,
5
- "epoch": 4.999535285135763,
6
- "eval_steps": 500,
7
- "global_step": 9410,
8
- "is_hyper_param_search": false,
9
- "is_local_process_zero": true,
10
- "is_world_process_zero": true,
11
- "log_history": [
12
- {
13
- "epoch": 0.26555135099249816,
14
- "grad_norm": 0.47810718417167664,
15
- "learning_rate": 4.736450584484591e-05,
16
- "loss": 2.7717,
17
- "step": 500
18
- },
19
- {
20
- "epoch": 0.5311027019849963,
21
- "grad_norm": 0.413313627243042,
22
- "learning_rate": 4.470775770456961e-05,
23
- "loss": 1.591,
24
- "step": 1000
25
- },
26
- {
27
- "epoch": 0.7966540529774945,
28
- "grad_norm": 0.31432437896728516,
29
- "learning_rate": 4.205100956429331e-05,
30
- "loss": 1.3043,
31
- "step": 1500
32
- },
33
- {
34
- "epoch": 1.0626701188342296,
35
- "grad_norm": 0.36749106645584106,
36
- "learning_rate": 3.9394261424017006e-05,
37
- "loss": 1.1595,
38
- "step": 2000
39
- },
40
- {
41
- "epoch": 1.3282214698267278,
42
- "grad_norm": 0.29507818818092346,
43
- "learning_rate": 3.6737513283740703e-05,
44
- "loss": 1.0709,
45
- "step": 2500
46
- },
47
- {
48
- "epoch": 1.5937728208192259,
49
- "grad_norm": 0.3084051012992859,
50
- "learning_rate": 3.40807651434644e-05,
51
- "loss": 1.005,
52
- "step": 3000
53
- },
54
- {
55
- "epoch": 1.859324171811724,
56
- "grad_norm": 0.2812994718551636,
57
- "learning_rate": 3.14240170031881e-05,
58
- "loss": 0.9559,
59
- "step": 3500
60
- },
61
- {
62
- "epoch": 2.125340237668459,
63
- "grad_norm": 0.3291400074958801,
64
- "learning_rate": 2.8767268862911793e-05,
65
- "loss": 0.9273,
66
- "step": 4000
67
- },
68
- {
69
- "epoch": 2.390891588660957,
70
- "grad_norm": 0.28141409158706665,
71
- "learning_rate": 2.6110520722635495e-05,
72
- "loss": 0.8928,
73
- "step": 4500
74
- },
75
- {
76
- "epoch": 2.6564429396534557,
77
- "grad_norm": 0.2757486402988434,
78
- "learning_rate": 2.3453772582359192e-05,
79
- "loss": 0.8714,
80
- "step": 5000
81
- },
82
- {
83
- "epoch": 2.9219942906459537,
84
- "grad_norm": 0.2808317542076111,
85
- "learning_rate": 2.079702444208289e-05,
86
- "loss": 0.8498,
87
- "step": 5500
88
- },
89
- {
90
- "epoch": 3.188010356502689,
91
- "grad_norm": 0.25909456610679626,
92
- "learning_rate": 1.8140276301806588e-05,
93
- "loss": 0.8322,
94
- "step": 6000
95
- },
96
- {
97
- "epoch": 3.453561707495187,
98
- "grad_norm": 0.2701779901981354,
99
- "learning_rate": 1.5483528161530286e-05,
100
- "loss": 0.8248,
101
- "step": 6500
102
- },
103
- {
104
- "epoch": 3.719113058487685,
105
- "grad_norm": 0.2889029085636139,
106
- "learning_rate": 1.2826780021253987e-05,
107
- "loss": 0.8087,
108
- "step": 7000
109
- },
110
- {
111
- "epoch": 3.984664409480183,
112
- "grad_norm": 0.2585383653640747,
113
- "learning_rate": 1.0170031880977683e-05,
114
- "loss": 0.8027,
115
- "step": 7500
116
- },
117
- {
118
- "epoch": 4.250680475336918,
119
- "grad_norm": 0.2610950171947479,
120
- "learning_rate": 7.513283740701383e-06,
121
- "loss": 0.7994,
122
- "step": 8000
123
- },
124
- {
125
- "epoch": 4.516231826329417,
126
- "grad_norm": 0.27488183975219727,
127
- "learning_rate": 4.85653560042508e-06,
128
- "loss": 0.7947,
129
- "step": 8500
130
- },
131
- {
132
- "epoch": 4.781783177321914,
133
- "grad_norm": 0.22057265043258667,
134
- "learning_rate": 2.1997874601487783e-06,
135
- "loss": 0.7842,
136
- "step": 9000
137
- }
138
- ],
139
- "logging_steps": 500,
140
- "max_steps": 9410,
141
- "num_input_tokens_seen": 0,
142
- "num_train_epochs": 5,
143
- "save_steps": 500,
144
- "stateful_callbacks": {
145
- "TrainerControl": {
146
- "args": {
147
- "should_epoch_stop": false,
148
- "should_evaluate": false,
149
- "should_log": false,
150
- "should_save": true,
151
- "should_training_stop": true
152
- },
153
- "attributes": {}
154
- }
155
- },
156
- "total_flos": 4.075845419689574e+16,
157
- "train_batch_size": 16,
158
- "trial_name": null,
159
- "trial_params": null
160
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0cee34848014a7023f7c44bcd836df67292a4f1655c491368c20d8d7294b9ebb
3
  size 242041896
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b78f8b4750e01e9e992daf6187c84c8ecfad35d6183f0336f1d4661a41534df9
3
  size 242041896
src/evaluate_t5.py CHANGED
@@ -31,6 +31,7 @@ print(f"Evaluating on {len(examples)} examples...\n")
31
  for src, tgt in tqdm(examples):
32
  input_prompt = f"Cyrillic2Latin: {src}"
33
  output = pipe(input_prompt, max_length = 128, do_sample = False)[0]["generated_text"]
 
34
  predictions.append(output.strip())
35
  references.append([tgt.strip()]) # wrap in list for BLEU
36
 
 
31
  for src, tgt in tqdm(examples):
32
  input_prompt = f"Cyrillic2Latin: {src}"
33
  output = pipe(input_prompt, max_length = 128, do_sample = False)[0]["generated_text"]
34
+
35
  predictions.append(output.strip())
36
  references.append([tgt.strip()]) # wrap in list for BLEU
37
 
src/train_t5.py CHANGED
@@ -49,9 +49,9 @@ data_collator = DataCollatorForSeq2Seq(tokenizer = tokeniser, model = model)
49
  training_args = TrainingArguments(
50
  output_dir = output_dir,
51
  overwrite_output_dir = True,
52
- num_train_epochs = 5,
53
- per_device_train_batch_size = 16,
54
- gradient_accumulation_steps = 8,
55
  save_strategy = "steps",
56
  save_steps = 500,
57
  save_total_limit = 3,
 
49
  training_args = TrainingArguments(
50
  output_dir = output_dir,
51
  overwrite_output_dir = True,
52
+ num_train_epochs = 10,
53
+ per_device_train_batch_size = 32,
54
+ gradient_accumulation_steps = 2,
55
  save_strategy = "steps",
56
  save_steps = 500,
57
  save_total_limit = 3,