Training in progress, epoch 1

README.md

@@ -31,6 +31,18 @@ More information needed
 ## Training procedure
 
 
+The following `bitsandbytes` quantization config was used during training:
+- quant_method: bitsandbytes
+- load_in_8bit: True
+- load_in_4bit: False
+- llm_int8_threshold: 6.0
+- llm_int8_skip_modules: None
+- llm_int8_enable_fp32_cpu_offload: False
+- llm_int8_has_fp16_weight: False
+- bnb_4bit_quant_type: fp4
+- bnb_4bit_use_double_quant: False
+- bnb_4bit_compute_dtype: float32
+
 The following `bitsandbytes` quantization config was used during training:
 - quant_method: bitsandbytes
 - load_in_8bit: True
@@ -65,6 +77,7 @@ The following hyperparameters were used during training:
 
 ### Framework versions
 
+- PEFT 0.6.0.dev0
 - PEFT 0.6.0.dev0
 - Transformers 4.34.0.dev0
 - Pytorch 2.0.1+cu117

adapter_config.json

@@ -17,11 +17,11 @@
   "revision": null,
   "target_modules": [
     "gate_proj",
+    "down_proj",
     "v_proj",
+    "k_proj",
     "up_proj",
-    "down_proj",
     "q_proj",
-    "k_proj",
     "o_proj"
   ],
   "task_type": "CAUSAL_LM"

adapter_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:629b8b83267e82fc04c0f4dbae0aedb7ddd0d6b922cb5c2fcc758ac96964e943
+oid sha256:ad4ea2397138ed2d852004167835dbb64b00ba4100354b13d396654bc6e1ce5e
 size 319977229

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3fbb5692d84707ba0da73df5c8db6584484c9710f3dd89c322d18f3312498cb9
+oid sha256:17b8a61bed456f0e64f13280369796447b5a0eb70ef6f1bbaa8ff6c0edeefe45
 size 4475