train_sst2_123_1768397591

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the sst2 dataset. It achieves the following results on the evaluation set:

• Loss: 0.0863
• Num Input Tokens Seen: 30587136

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0005	0.5000	15154	0.1189	1531648
0.2507	1.0000	30308	0.0995	3059936
0.5481	1.5000	45462	0.0976	4592832
0.2943	2.0001	60616	0.0863	6119584
0.0011	2.5001	75770	0.0948	7649168
0.0042	3.0001	90924	0.0939	9178224
0.0027	3.5001	106078	0.1009	10707152
0.0003	4.0001	121232	0.0964	12237344
0.0006	4.5001	136386	0.0918	13766448
0.0029	5.0002	151540	0.0938	15296208
0.32	5.5002	166694	0.0933	16825280
0.0016	6.0002	181848	0.1049	18354816
0.0001	6.5002	197002	0.0951	19883504
0.0007	7.0002	212156	0.0979	21413952
0.0002	7.5002	227310	0.1151	22941760
0.232	8.0003	242464	0.1049	24472480
0.1768	8.5003	257618	0.1080	26001792
0.0029	9.0003	272772	0.1126	27530688
0.0002	9.5003	287926	0.1135	29056064

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.1+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4