Model save

README.md

@@ -18,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [google/codegemma-7b](https://huggingface.co/google/codegemma-7b) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.0947
+- Loss: 0.0653
 
 ## Model description
 
@@ -37,7 +37,7 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 5e-05
+- learning_rate: 2e-08
 - train_batch_size: 1
 - eval_batch_size: 3
 - seed: 42
@@ -45,9 +45,9 @@ The following hyperparameters were used during training:
 - gradient_accumulation_steps: 8
 - total_train_batch_size: 8
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
-- lr_scheduler_type: cosine
+- lr_scheduler_type: linear
 - lr_scheduler_warmup_ratio: 0.03
-- num_epochs: 2
+- num_epochs: 3
 - mixed_precision_training: Native AMP
 
 ### Training results
@@ -90,7 +90,80 @@ The following hyperparameters were used during training:
 | 0.0987        | 1.8032 | 1700 | 0.0968          |
 | 0.1037        | 1.8563 | 1750 | 0.0960          |
 | 0.1047        | 1.9093 | 1800 | 0.0957          |
-| 0.1045        | 1.9623 | 1850 | 0.0947          |
+| 0.109         | 1.9631 | 1850 | 0.0953          |
+| 0.1099        | 2.0162 | 1900 | 0.0938          |
+| 0.102         | 2.0692 | 1950 | 0.0927          |
+| 0.1063        | 2.1222 | 2000 | 0.0929          |
+| 0.0985        | 2.1753 | 2050 | 0.0910          |
+| 0.0936        | 2.2283 | 2100 | 0.0908          |
+| 0.0998        | 2.2814 | 2150 | 0.0908          |
+| 0.0935        | 2.3344 | 2200 | 0.0905          |
+| 0.1028        | 2.3874 | 2250 | 0.0904          |
+| 0.1036        | 2.4405 | 2300 | 0.0899          |
+| 0.0998        | 2.4943 | 2350 | 0.0888          |
+| 0.0923        | 2.5473 | 2400 | 0.0890          |
+| 0.0979        | 2.6004 | 2450 | 0.0887          |
+| 0.1012        | 2.6534 | 2500 | 0.0879          |
+| 0.0936        | 2.7064 | 2550 | 0.0882          |
+| 0.0948        | 2.7595 | 2600 | 0.0876          |
+| 0.0862        | 2.8125 | 2650 | 0.0858          |
+| 0.0979        | 2.8656 | 2700 | 0.0853          |
+| 0.0873        | 2.9186 | 2750 | 0.0858          |
+| 0.0901        | 2.9716 | 2800 | 0.0856          |
+| 0.0862        | 3.0247 | 2850 | 0.0838          |
+| 0.0901        | 3.0777 | 2900 | 0.0825          |
+| 0.0838        | 3.1307 | 2950 | 0.0829          |
+| 0.0873        | 3.1838 | 3000 | 0.0830          |
+| 0.0798        | 3.2368 | 3050 | 0.0816          |
+| 0.0845        | 3.2898 | 3100 | 0.0804          |
+| 0.0831        | 3.3429 | 3150 | 0.0804          |
+| 0.081         | 3.3959 | 3200 | 0.0792          |
+| 0.0842        | 3.4490 | 3250 | 0.0790          |
+| 0.0749        | 3.5020 | 3300 | 0.0792          |
+| 0.0873        | 3.5550 | 3350 | 0.0795          |
+| 0.0754        | 3.6081 | 3400 | 0.0794          |
+| 0.0779        | 3.6611 | 3450 | 0.0793          |
+| 0.0809        | 3.7141 | 3500 | 0.0774          |
+| 0.0807        | 3.7672 | 3550 | 0.0773          |
+| 0.0761        | 3.8202 | 3600 | 0.0775          |
+| 0.0736        | 3.8732 | 3650 | 0.0757          |
+| 0.0746        | 3.9263 | 3700 | 0.0759          |
+| 0.0844        | 3.9793 | 3750 | 0.0760          |
+| 0.0764        | 4.0324 | 3800 | 0.0758          |
+| 0.0722        | 4.0854 | 3850 | 0.0754          |
+| 0.068         | 4.1384 | 3900 | 0.0752          |
+| 0.0641        | 4.1915 | 3950 | 0.0736          |
+| 0.0665        | 4.2445 | 4000 | 0.0733          |
+| 0.0674        | 4.2975 | 4050 | 0.0736          |
+| 0.0693        | 4.3506 | 4100 | 0.0724          |
+| 0.072         | 4.4036 | 4150 | 0.0714          |
+| 0.0683        | 4.4566 | 4200 | 0.0716          |
+| 0.061         | 4.5097 | 4250 | 0.0718          |
+| 0.0653        | 4.5627 | 4300 | 0.0706          |
+| 0.0707        | 4.6158 | 4350 | 0.0702          |
+| 0.0719        | 4.6688 | 4400 | 0.0714          |
+| 0.0669        | 4.7218 | 4450 | 0.0706          |
+| 0.0673        | 4.7749 | 4500 | 0.0710          |
+| 0.0677        | 4.8279 | 4550 | 0.0714          |
+| 0.0795        | 4.8809 | 4600 | 0.0700          |
+| 0.0724        | 4.9340 | 4650 | 0.0699          |
+| 0.0648        | 4.9870 | 4700 | 0.0707          |
+| 0.0614        | 5.0400 | 4750 | 0.0696          |
+| 0.0606        | 5.0931 | 4800 | 0.0691          |
+| 0.0579        | 5.1461 | 4850 | 0.0691          |
+| 0.0645        | 5.1992 | 4900 | 0.0680          |
+| 0.0648        | 5.2522 | 4950 | 0.0673          |
+| 0.0624        | 5.3052 | 5000 | 0.0672          |
+| 0.0604        | 5.3583 | 5050 | 0.0676          |
+| 0.0582        | 5.4113 | 5100 | 0.0672          |
+| 0.0622        | 5.4643 | 5150 | 0.0664          |
+| 0.0589        | 5.5174 | 5200 | 0.0665          |
+| 0.0586        | 5.5704 | 5250 | 0.0664          |
+| 0.0577        | 5.6234 | 5300 | 0.0666          |
+| 0.0538        | 5.6765 | 5350 | 0.0660          |
+| 0.0616        | 5.7295 | 5400 | 0.0657          |
+| 0.0582        | 5.7826 | 5450 | 0.0661          |
+| 0.0599        | 5.8356 | 5500 | 0.0653          |
 
 
 ### Framework versions

adapter_config.json

@@ -20,13 +20,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "gate_proj",
-    "o_proj",
+    "q_proj",
+    "v_proj",
     "up_proj",
-    "k_proj",
     "down_proj",
-    "q_proj",
-    "v_proj"
+    "o_proj",
+    "k_proj",
+    "gate_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

runs/Sep11_00-04-42_m3h110/events.out.tfevents.1757513677.m3h110.2487544.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:34c4a7ef12d2ae465950764decbeb8c10ed6b66e885a7546a83ece8e3a72a098
-size 90262
+oid sha256:167a3010acf32d2e7203bae87bc4c6f6c4cfa5aa071385a9d093bfdf93f543fd
+size 92221

runs/Sep11_08-55-19_m3h110/events.out.tfevents.1757545405.m3h110.2688810.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5449209741021928bf5eb96bb66d4cc94e9584cb8bac065b259ea5e8b668686b
+size 5752

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e068e4a9936fdc74d21572c03b2077a60a41df5f9d0e5e93787674d03d000212
+oid sha256:48715564d8294ad94f4c1d121a1672238f39ea1a2f24db9adb0ef5f6c68fbd7d
 size 6520