Upload folder using huggingface_hub

README.md

@@ -0,0 +1,59 @@
+---
+license: other
+library_name: peft
+tags:
+- llama-factory
+- lora
+- generated_from_trainer
+base_model: Qwen/Qwen1.5-0.5B-Chat
+model-index:
+- name: train_2024-02-22-19
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# train_2024-02-22-19
+
+This model is a fine-tuned version of [Qwen/Qwen1.5-0.5B-Chat](https://huggingface.co/Qwen/Qwen1.5-0.5B-Chat) on the TCM and the oaast_sft_zh datasets.
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 4
+- eval_batch_size: 8
+- seed: 42
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 16
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- num_epochs: 3.0
+- mixed_precision_training: Native AMP
+
+### Training results
+
+
+
+### Framework versions
+
+- PEFT 0.8.2
+- Transformers 4.37.2
+- Pytorch 2.1.0+cu121
+- Datasets 2.17.1
+- Tokenizers 0.15.2

adapter_config.json

@@ -0,0 +1,27 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen1.5-0.5B-Chat",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7cba12641e146e6d11b7fc8fa1ecffa29cc7baeef55c21fd6b3226e4f38bff71
+size 3158328

added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

all_results.json

@@ -0,0 +1,7 @@
+{
+    "epoch": 3.0,
+    "train_loss": 2.399462188845504,
+    "train_runtime": 8474.8141,
+    "train_samples_per_second": 14.368,
+    "train_steps_per_second": 0.898
+}

checkpoint-100/README.md

@@ -0,0 +1,204 @@
+---
+library_name: peft
+base_model: Qwen/Qwen1.5-0.5B-Chat
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+
+### Framework versions
+
+- PEFT 0.8.2

checkpoint-100/adapter_config.json

@@ -0,0 +1,27 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen1.5-0.5B-Chat",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_rslora": false
+}

checkpoint-100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8cd2092e26417801679225b9a398170b6bb2c82b42f953f8593aca417335b46c
+size 3158328

checkpoint-100/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

checkpoint-100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4f3828e57cbb2ba5d6a23c600fcf7f27b0396271d195f963a29e873d2e80e26c
+size 6372346

checkpoint-100/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8e8ebe0c5b5b3f651ffe699f7081217241d448d7bdc246a087e9bffcb4184e48
+size 14244

checkpoint-100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:825a1afeec51887fdc143b454e27fe7538d8978963d3e380e26de659d47cb424
+size 1064

checkpoint-100/special_tokens_map.json

@@ -0,0 +1,20 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-100/tokenizer_config.json

@@ -0,0 +1,44 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content']}}{% if (loop.last and add_generation_prompt) or not loop.last %}{{ '<|im_end|>' + '\n'}}{% endif %}{% endfor %}{% if add_generation_prompt and messages[-1]['role'] != 'assistant' %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-100/trainer_state.json

@@ -0,0 +1,141 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.039416633819471816,
+  "eval_steps": 500,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0,
+      "learning_rate": 1.999997870262096e-05,
+      "loss": 2.714,
+      "step": 5
+    },
+    {
+      "epoch": 0.0,
+      "learning_rate": 1.999991481057455e-05,
+      "loss": 2.7679,
+      "step": 10
+    },
+    {
+      "epoch": 0.01,
+      "learning_rate": 1.9999808324132915e-05,
+      "loss": 2.6887,
+      "step": 15
+    },
+    {
+      "epoch": 0.01,
+      "learning_rate": 1.999965924374964e-05,
+      "loss": 2.6986,
+      "step": 20
+    },
+    {
+      "epoch": 0.01,
+      "learning_rate": 1.999946757005972e-05,
+      "loss": 2.7536,
+      "step": 25
+    },
+    {
+      "epoch": 0.01,
+      "learning_rate": 1.9999233303879592e-05,
+      "loss": 2.7516,
+      "step": 30
+    },
+    {
+      "epoch": 0.01,
+      "learning_rate": 1.99989564462071e-05,
+      "loss": 2.6283,
+      "step": 35
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 1.999863699822152e-05,
+      "loss": 2.708,
+      "step": 40
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 1.9998274961283523e-05,
+      "loss": 2.6932,
+      "step": 45
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 1.9997870336935207e-05,
+      "loss": 2.6321,
+      "step": 50
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 1.9997423126900056e-05,
+      "loss": 2.7129,
+      "step": 55
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 1.9996933333082945e-05,
+      "loss": 2.6179,
+      "step": 60
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 1.9996400957570148e-05,
+      "loss": 2.6052,
+      "step": 65
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 1.99958260026293e-05,
+      "loss": 2.7054,
+      "step": 70
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 1.9995208470709405e-05,
+      "loss": 2.6568,
+      "step": 75
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 1.9994548364440836e-05,
+      "loss": 2.6436,
+      "step": 80
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 1.999384568663529e-05,
+      "loss": 2.7091,
+      "step": 85
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 1.9993100440285805e-05,
+      "loss": 2.5557,
+      "step": 90
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 1.999231262856675e-05,
+      "loss": 2.5945,
+      "step": 95
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 1.999148225483378e-05,
+      "loss": 2.4729,
+      "step": 100
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 7611,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "total_flos": 1370020913479680.0,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e79a241514d107c9eee3780ef385cef96ff7663902d9e94f80f6c1be656e9697
+size 4920

checkpoint-1000/README.md

@@ -0,0 +1,204 @@
+---
+library_name: peft
+base_model: Qwen/Qwen1.5-0.5B-Chat
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+
+### Framework versions
+
+- PEFT 0.8.2

checkpoint-1000/adapter_config.json

@@ -0,0 +1,27 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen1.5-0.5B-Chat",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_rslora": false
+}

checkpoint-1000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e14d2b38eae34dc841cc10fd0812843c8509dfb2b9beb9a1026b8b293a7059e5
+size 3158328

checkpoint-1000/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

checkpoint-1000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:527de9d9156faabfac42d56c5ce99d877ae3ce631a4bef78ec0d641658f94f09
+size 6372346

checkpoint-1000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:56aa5cc349e8ae08626e350a812a07346280d96d2fc08a28422c002f4773b58c
+size 14244

checkpoint-1000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cec8d348a5445d851ce6873becbb6550d5dea646bf05d301f93674de0796d139
+size 1064

checkpoint-1000/special_tokens_map.json

@@ -0,0 +1,20 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-1000/tokenizer_config.json

@@ -0,0 +1,44 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content']}}{% if (loop.last and add_generation_prompt) or not loop.last %}{{ '<|im_end|>' + '\n'}}{% endif %}{% endfor %}{% if add_generation_prompt and messages[-1]['role'] != 'assistant' %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-1000/trainer_state.json

@@ -0,0 +1,1221 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.3941663381947182,
+  "eval_steps": 500,
+  "global_step": 1000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0,
+      "learning_rate": 1.999997870262096e-05,
+      "loss": 2.714,
+      "step": 5
+    },
+    {
+      "epoch": 0.0,
+      "learning_rate": 1.999991481057455e-05,
+      "loss": 2.7679,
+      "step": 10
+    },
+    {
+      "epoch": 0.01,
+      "learning_rate": 1.9999808324132915e-05,
+      "loss": 2.6887,
+      "step": 15
+    },
+    {
+      "epoch": 0.01,
+      "learning_rate": 1.999965924374964e-05,
+      "loss": 2.6986,
+      "step": 20
+    },
+    {
+      "epoch": 0.01,
+      "learning_rate": 1.999946757005972e-05,
+      "loss": 2.7536,
+      "step": 25
+    },
+    {
+      "epoch": 0.01,
+      "learning_rate": 1.9999233303879592e-05,
+      "loss": 2.7516,
+      "step": 30
+    },
+    {
+      "epoch": 0.01,
+      "learning_rate": 1.99989564462071e-05,
+      "loss": 2.6283,
+      "step": 35
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 1.999863699822152e-05,
+      "loss": 2.708,
+      "step": 40
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 1.9998274961283523e-05,
+      "loss": 2.6932,
+      "step": 45
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 1.9997870336935207e-05,
+      "loss": 2.6321,
+      "step": 50
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 1.9997423126900056e-05,
+      "loss": 2.7129,
+      "step": 55
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 1.9996933333082945e-05,
+      "loss": 2.6179,
+      "step": 60
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 1.9996400957570148e-05,
+      "loss": 2.6052,
+      "step": 65
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 1.99958260026293e-05,
+      "loss": 2.7054,
+      "step": 70
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 1.9995208470709405e-05,
+      "loss": 2.6568,
+      "step": 75
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 1.9994548364440836e-05,
+      "loss": 2.6436,
+      "step": 80
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 1.999384568663529e-05,
+      "loss": 2.7091,
+      "step": 85
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 1.9993100440285805e-05,
+      "loss": 2.5557,
+      "step": 90
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 1.999231262856675e-05,
+      "loss": 2.5945,
+      "step": 95
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 1.999148225483378e-05,
+      "loss": 2.4729,
+      "step": 100
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 1.9990609322623854e-05,
+      "loss": 2.6724,
+      "step": 105
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 1.9989693835655205e-05,
+      "loss": 2.5272,
+      "step": 110
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 1.9988735797827336e-05,
+      "loss": 2.6145,
+      "step": 115
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 1.9987735213220975e-05,
+      "loss": 2.6772,
+      "step": 120
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 1.9986692086098095e-05,
+      "loss": 2.6388,
+      "step": 125
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 1.998560642090187e-05,
+      "loss": 2.4973,
+      "step": 130
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 1.998447822225666e-05,
+      "loss": 2.5338,
+      "step": 135
+    },
+    {
+      "epoch": 0.06,
+      "learning_rate": 1.9983307494968e-05,
+      "loss": 2.6469,
+      "step": 140
+    },
+    {
+      "epoch": 0.06,
+      "learning_rate": 1.9982094244022582e-05,
+      "loss": 2.6008,
+      "step": 145
+    },
+    {
+      "epoch": 0.06,
+      "learning_rate": 1.9980838474588214e-05,
+      "loss": 2.5936,
+      "step": 150
+    },
+    {
+      "epoch": 0.06,
+      "learning_rate": 1.9979540192013814e-05,
+      "loss": 2.5862,
+      "step": 155
+    },
+    {
+      "epoch": 0.06,
+      "learning_rate": 1.997819940182939e-05,
+      "loss": 2.568,
+      "step": 160
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 1.9976816109746e-05,
+      "loss": 2.5343,
+      "step": 165
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 1.9975390321655745e-05,
+      "loss": 2.5575,
+      "step": 170
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 1.9973922043631737e-05,
+      "loss": 2.5399,
+      "step": 175
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 1.9972411281928068e-05,
+      "loss": 2.6394,
+      "step": 180
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 1.9970858042979794e-05,
+      "loss": 2.5378,
+      "step": 185
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 1.9969262333402893e-05,
+      "loss": 2.5373,
+      "step": 190
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 1.9967624159994262e-05,
+      "loss": 2.5414,
+      "step": 195
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 1.9965943529731646e-05,
+      "loss": 2.5606,
+      "step": 200
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 1.9964220449773664e-05,
+      "loss": 2.5154,
+      "step": 205
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 1.9962454927459723e-05,
+      "loss": 2.5468,
+      "step": 210
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 1.9960646970310027e-05,
+      "loss": 2.5137,
+      "step": 215
+    },
+    {
+      "epoch": 0.09,
+      "learning_rate": 1.9958796586025527e-05,
+      "loss": 2.5453,
+      "step": 220
+    },
+    {
+      "epoch": 0.09,
+      "learning_rate": 1.9956903782487885e-05,
+      "loss": 2.5325,
+      "step": 225
+    },
+    {
+      "epoch": 0.09,
+      "learning_rate": 1.9954968567759456e-05,
+      "loss": 2.5054,
+      "step": 230
+    },
+    {
+      "epoch": 0.09,
+      "learning_rate": 1.9952990950083236e-05,
+      "loss": 2.5139,
+      "step": 235
+    },
+    {
+      "epoch": 0.09,
+      "learning_rate": 1.995097093788285e-05,
+      "loss": 2.5097,
+      "step": 240
+    },
+    {
+      "epoch": 0.1,
+      "learning_rate": 1.994890853976248e-05,
+      "loss": 2.5546,
+      "step": 245
+    },
+    {
+      "epoch": 0.1,
+      "learning_rate": 1.994680376450686e-05,
+      "loss": 2.4609,
+      "step": 250
+    },
+    {
+      "epoch": 0.1,
+      "learning_rate": 1.994465662108124e-05,
+      "loss": 2.5063,
+      "step": 255
+    },
+    {
+      "epoch": 0.1,
+      "learning_rate": 1.9942467118631322e-05,
+      "loss": 2.5162,
+      "step": 260
+    },
+    {
+      "epoch": 0.1,
+      "learning_rate": 1.994023526648323e-05,
+      "loss": 2.5317,
+      "step": 265
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 1.9937961074143492e-05,
+      "loss": 2.4621,
+      "step": 270
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 1.9935644551298976e-05,
+      "loss": 2.5879,
+      "step": 275
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 1.993328570781685e-05,
+      "loss": 2.4742,
+      "step": 280
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 1.993088455374456e-05,
+      "loss": 2.4661,
+      "step": 285
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 1.992844109930975e-05,
+      "loss": 2.4943,
+      "step": 290
+    },
+    {
+      "epoch": 0.12,
+      "learning_rate": 1.9925955354920265e-05,
+      "loss": 2.4322,
+      "step": 295
+    },
+    {
+      "epoch": 0.12,
+      "learning_rate": 1.9923427331164072e-05,
+      "loss": 2.484,
+      "step": 300
+    },
+    {
+      "epoch": 0.12,
+      "learning_rate": 1.9920857038809223e-05,
+      "loss": 2.5334,
+      "step": 305
+    },
+    {
+      "epoch": 0.12,
+      "learning_rate": 1.991824448880382e-05,
+      "loss": 2.6554,
+      "step": 310
+    },
+    {
+      "epoch": 0.12,
+      "learning_rate": 1.9915589692275955e-05,
+      "loss": 2.4749,
+      "step": 315
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 1.991289266053367e-05,
+      "loss": 2.5076,
+      "step": 320
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 1.9910153405064904e-05,
+      "loss": 2.4635,
+      "step": 325
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 1.990737193753745e-05,
+      "loss": 2.5443,
+      "step": 330
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 1.9904548269798906e-05,
+      "loss": 2.6012,
+      "step": 335
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 1.990168241387662e-05,
+      "loss": 2.4818,
+      "step": 340
+    },
+    {
+      "epoch": 0.14,
+      "learning_rate": 1.9898774381977618e-05,
+      "loss": 2.4717,
+      "step": 345
+    },
+    {
+      "epoch": 0.14,
+      "learning_rate": 1.989582418648861e-05,
+      "loss": 2.4756,
+      "step": 350
+    },
+    {
+      "epoch": 0.14,
+      "learning_rate": 1.9892831839975874e-05,
+      "loss": 2.4868,
+      "step": 355
+    },
+    {
+      "epoch": 0.14,
+      "learning_rate": 1.9889797355185237e-05,
+      "loss": 2.5652,
+      "step": 360
+    },
+    {
+      "epoch": 0.14,
+      "learning_rate": 1.9886720745042017e-05,
+      "loss": 2.3907,
+      "step": 365
+    },
+    {
+      "epoch": 0.15,
+      "learning_rate": 1.988360202265096e-05,
+      "loss": 2.4984,
+      "step": 370
+    },
+    {
+      "epoch": 0.15,
+      "learning_rate": 1.9880441201296186e-05,
+      "loss": 2.5654,
+      "step": 375
+    },
+    {
+      "epoch": 0.15,
+      "learning_rate": 1.987723829444114e-05,
+      "loss": 2.581,
+      "step": 380
+    },
+    {
+      "epoch": 0.15,
+      "learning_rate": 1.9873993315728523e-05,
+      "loss": 2.5189,
+      "step": 385
+    },
+    {
+      "epoch": 0.15,
+      "learning_rate": 1.987070627898025e-05,
+      "loss": 2.491,
+      "step": 390
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 1.9867377198197367e-05,
+      "loss": 2.4875,
+      "step": 395
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 1.9864006087560016e-05,
+      "loss": 2.4828,
+      "step": 400
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 1.9860592961427358e-05,
+      "loss": 2.5298,
+      "step": 405
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 1.9857137834337527e-05,
+      "loss": 2.4873,
+      "step": 410
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 1.985364072100755e-05,
+      "loss": 2.5371,
+      "step": 415
+    },
+    {
+      "epoch": 0.17,
+      "learning_rate": 1.98501016363333e-05,
+      "loss": 2.4167,
+      "step": 420
+    },
+    {
+      "epoch": 0.17,
+      "learning_rate": 1.9846520595389415e-05,
+      "loss": 2.4312,
+      "step": 425
+    },
+    {
+      "epoch": 0.17,
+      "learning_rate": 1.984289761342926e-05,
+      "loss": 2.4471,
+      "step": 430
+    },
+    {
+      "epoch": 0.17,
+      "learning_rate": 1.9839232705884836e-05,
+      "loss": 2.4971,
+      "step": 435
+    },
+    {
+      "epoch": 0.17,
+      "learning_rate": 1.9835525888366727e-05,
+      "loss": 2.5206,
+      "step": 440
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 1.9831777176664035e-05,
+      "loss": 2.5067,
+      "step": 445
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 1.9827986586744302e-05,
+      "loss": 2.4903,
+      "step": 450
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 1.982415413475346e-05,
+      "loss": 2.4391,
+      "step": 455
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 1.9820279837015742e-05,
+      "loss": 2.4358,
+      "step": 460
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 1.981636371003363e-05,
+      "loss": 2.4809,
+      "step": 465
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 1.9812405770487763e-05,
+      "loss": 2.516,
+      "step": 470
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 1.9808406035236897e-05,
+      "loss": 2.4091,
+      "step": 475
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 1.9804364521317806e-05,
+      "loss": 2.5363,
+      "step": 480
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 1.9800281245945217e-05,
+      "loss": 2.51,
+      "step": 485
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 1.9796156226511747e-05,
+      "loss": 2.5049,
+      "step": 490
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 1.9791989480587815e-05,
+      "loss": 2.4722,
+      "step": 495
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 1.978778102592157e-05,
+      "loss": 2.4111,
+      "step": 500
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 1.9783530880438832e-05,
+      "loss": 2.4078,
+      "step": 505
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 1.9779239062242988e-05,
+      "loss": 2.3981,
+      "step": 510
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 1.9774905589614935e-05,
+      "loss": 2.5459,
+      "step": 515
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 1.977053048101299e-05,
+      "loss": 2.4432,
+      "step": 520
+    },
+    {
+      "epoch": 0.21,
+      "learning_rate": 1.976611375507283e-05,
+      "loss": 2.5329,
+      "step": 525
+    },
+    {
+      "epoch": 0.21,
+      "learning_rate": 1.9761655430607384e-05,
+      "loss": 2.5107,
+      "step": 530
+    },
+    {
+      "epoch": 0.21,
+      "learning_rate": 1.975715552660678e-05,
+      "loss": 2.4389,
+      "step": 535
+    },
+    {
+      "epoch": 0.21,
+      "learning_rate": 1.9752614062238256e-05,
+      "loss": 2.4939,
+      "step": 540
+    },
+    {
+      "epoch": 0.21,
+      "learning_rate": 1.974803105684606e-05,
+      "loss": 2.5349,
+      "step": 545
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 1.9743406529951403e-05,
+      "loss": 2.5268,
+      "step": 550
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 1.9738740501252337e-05,
+      "loss": 2.4477,
+      "step": 555
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 1.9734032990623702e-05,
+      "loss": 2.4597,
+      "step": 560
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 1.972928401811703e-05,
+      "loss": 2.4938,
+      "step": 565
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 1.9724493603960443e-05,
+      "loss": 2.4756,
+      "step": 570
+    },
+    {
+      "epoch": 0.23,
+      "learning_rate": 1.9719661768558604e-05,
+      "loss": 2.3664,
+      "step": 575
+    },
+    {
+      "epoch": 0.23,
+      "learning_rate": 1.9714788532492595e-05,
+      "loss": 2.5166,
+      "step": 580
+    },
+    {
+      "epoch": 0.23,
+      "learning_rate": 1.9709873916519853e-05,
+      "loss": 2.4143,
+      "step": 585
+    },
+    {
+      "epoch": 0.23,
+      "learning_rate": 1.9704917941574053e-05,
+      "loss": 2.4666,
+      "step": 590
+    },
+    {
+      "epoch": 0.23,
+      "learning_rate": 1.9699920628765065e-05,
+      "loss": 2.4617,
+      "step": 595
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 1.969488199937881e-05,
+      "loss": 2.4912,
+      "step": 600
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 1.9689802074877216e-05,
+      "loss": 2.4386,
+      "step": 605
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 1.9684680876898096e-05,
+      "loss": 2.5002,
+      "step": 610
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 1.967951842725507e-05,
+      "loss": 2.5668,
+      "step": 615
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 1.9674314747937462e-05,
+      "loss": 2.4955,
+      "step": 620
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 1.9669069861110225e-05,
+      "loss": 2.5317,
+      "step": 625
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 1.9663783789113827e-05,
+      "loss": 2.4709,
+      "step": 630
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 1.9658456554464157e-05,
+      "loss": 2.4066,
+      "step": 635
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 1.9653088179852448e-05,
+      "loss": 2.4462,
+      "step": 640
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 1.9647678688145163e-05,
+      "loss": 2.5093,
+      "step": 645
+    },
+    {
+      "epoch": 0.26,
+      "learning_rate": 1.9642228102383894e-05,
+      "loss": 2.4634,
+      "step": 650
+    },
+    {
+      "epoch": 0.26,
+      "learning_rate": 1.9636736445785288e-05,
+      "loss": 2.424,
+      "step": 655
+    },
+    {
+      "epoch": 0.26,
+      "learning_rate": 1.963120374174092e-05,
+      "loss": 2.4968,
+      "step": 660
+    },
+    {
+      "epoch": 0.26,
+      "learning_rate": 1.9625630013817204e-05,
+      "loss": 2.3947,
+      "step": 665
+    },
+    {
+      "epoch": 0.26,
+      "learning_rate": 1.9620015285755306e-05,
+      "loss": 2.3954,
+      "step": 670
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 1.961435958147102e-05,
+      "loss": 2.3764,
+      "step": 675
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 1.9608662925054684e-05,
+      "loss": 2.386,
+      "step": 680
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 1.960292534077107e-05,
+      "loss": 2.4218,
+      "step": 685
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 1.9597146853059273e-05,
+      "loss": 2.4479,
+      "step": 690
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 1.959132748653263e-05,
+      "loss": 2.4301,
+      "step": 695
+    },
+    {
+      "epoch": 0.28,
+      "learning_rate": 1.9585467265978585e-05,
+      "loss": 2.5069,
+      "step": 700
+    },
+    {
+      "epoch": 0.28,
+      "learning_rate": 1.957956621635861e-05,
+      "loss": 2.4144,
+      "step": 705
+    },
+    {
+      "epoch": 0.28,
+      "learning_rate": 1.9573624362808078e-05,
+      "loss": 2.4491,
+      "step": 710
+    },
+    {
+      "epoch": 0.28,
+      "learning_rate": 1.9567641730636174e-05,
+      "loss": 2.4723,
+      "step": 715
+    },
+    {
+      "epoch": 0.28,
+      "learning_rate": 1.956161834532578e-05,
+      "loss": 2.4839,
+      "step": 720
+    },
+    {
+      "epoch": 0.29,
+      "learning_rate": 1.955555423253335e-05,
+      "loss": 2.4881,
+      "step": 725
+    },
+    {
+      "epoch": 0.29,
+      "learning_rate": 1.9549449418088832e-05,
+      "loss": 2.5076,
+      "step": 730
+    },
+    {
+      "epoch": 0.29,
+      "learning_rate": 1.9543303927995536e-05,
+      "loss": 2.4704,
+      "step": 735
+    },
+    {
+      "epoch": 0.29,
+      "learning_rate": 1.9537117788430024e-05,
+      "loss": 2.3958,
+      "step": 740
+    },
+    {
+      "epoch": 0.29,
+      "learning_rate": 1.953089102574201e-05,
+      "loss": 2.5256,
+      "step": 745
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 1.9524623666454243e-05,
+      "loss": 2.4658,
+      "step": 750
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 1.951831573726238e-05,
+      "loss": 2.4636,
+      "step": 755
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 1.9511967265034904e-05,
+      "loss": 2.4249,
+      "step": 760
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 1.9505578276812964e-05,
+      "loss": 2.5412,
+      "step": 765
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 1.9499148799810314e-05,
+      "loss": 2.4355,
+      "step": 770
+    },
+    {
+      "epoch": 0.31,
+      "learning_rate": 1.949267886141315e-05,
+      "loss": 2.5814,
+      "step": 775
+    },
+    {
+      "epoch": 0.31,
+      "learning_rate": 1.948616848918002e-05,
+      "loss": 2.5195,
+      "step": 780
+    },
+    {
+      "epoch": 0.31,
+      "learning_rate": 1.9479617710841693e-05,
+      "loss": 2.5177,
+      "step": 785
+    },
+    {
+      "epoch": 0.31,
+      "learning_rate": 1.9473026554301057e-05,
+      "loss": 2.4418,
+      "step": 790
+    },
+    {
+      "epoch": 0.31,
+      "learning_rate": 1.946639504763298e-05,
+      "loss": 2.5187,
+      "step": 795
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 1.94597232190842e-05,
+      "loss": 2.479,
+      "step": 800
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 1.9453011097073217e-05,
+      "loss": 2.4451,
+      "step": 805
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 1.9446258710190152e-05,
+      "loss": 2.5426,
+      "step": 810
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 1.9439466087196627e-05,
+      "loss": 2.4299,
+      "step": 815
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 1.943263325702566e-05,
+      "loss": 2.5317,
+      "step": 820
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 1.9425760248781525e-05,
+      "loss": 2.4384,
+      "step": 825
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 1.9418847091739638e-05,
+      "loss": 2.4445,
+      "step": 830
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 1.9411893815346418e-05,
+      "loss": 2.4872,
+      "step": 835
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 1.9404900449219178e-05,
+      "loss": 2.5123,
+      "step": 840
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 1.9397867023146e-05,
+      "loss": 2.3569,
+      "step": 845
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 1.9390793567085585e-05,
+      "loss": 2.467,
+      "step": 850
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 1.9383680111167146e-05,
+      "loss": 2.4097,
+      "step": 855
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 1.937652668569028e-05,
+      "loss": 2.4653,
+      "step": 860
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 1.9369333321124832e-05,
+      "loss": 2.4164,
+      "step": 865
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 1.936210004811076e-05,
+      "loss": 2.3906,
+      "step": 870
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 1.9354826897458016e-05,
+      "loss": 2.4433,
+      "step": 875
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 1.9347513900146412e-05,
+      "loss": 2.42,
+      "step": 880
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 1.9340161087325483e-05,
+      "loss": 2.4606,
+      "step": 885
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 1.9332768490314354e-05,
+      "loss": 2.4242,
+      "step": 890
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 1.9325336140601612e-05,
+      "loss": 2.461,
+      "step": 895
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 1.931786406984518e-05,
+      "loss": 2.3949,
+      "step": 900
+    },
+    {
+      "epoch": 0.36,
+      "learning_rate": 1.9310352309872153e-05,
+      "loss": 2.4449,
+      "step": 905
+    },
+    {
+      "epoch": 0.36,
+      "learning_rate": 1.9302800892678693e-05,
+      "loss": 2.4864,
+      "step": 910
+    },
+    {
+      "epoch": 0.36,
+      "learning_rate": 1.9295209850429884e-05,
+      "loss": 2.5432,
+      "step": 915
+    },
+    {
+      "epoch": 0.36,
+      "learning_rate": 1.9287579215459585e-05,
+      "loss": 2.4357,
+      "step": 920
+    },
+    {
+      "epoch": 0.36,
+      "learning_rate": 1.9279909020270294e-05,
+      "loss": 2.4625,
+      "step": 925
+    },
+    {
+      "epoch": 0.37,
+      "learning_rate": 1.9272199297533027e-05,
+      "loss": 2.3937,
+      "step": 930
+    },
+    {
+      "epoch": 0.37,
+      "learning_rate": 1.9264450080087163e-05,
+      "loss": 2.5051,
+      "step": 935
+    },
+    {
+      "epoch": 0.37,
+      "learning_rate": 1.9256661400940303e-05,
+      "loss": 2.4894,
+      "step": 940
+    },
+    {
+      "epoch": 0.37,
+      "learning_rate": 1.9248833293268144e-05,
+      "loss": 2.4117,
+      "step": 945
+    },
+    {
+      "epoch": 0.37,
+      "learning_rate": 1.9240965790414312e-05,
+      "loss": 2.4892,
+      "step": 950
+    },
+    {
+      "epoch": 0.38,
+      "learning_rate": 1.923305892589025e-05,
+      "loss": 2.4059,
+      "step": 955
+    },
+    {
+      "epoch": 0.38,
+      "learning_rate": 1.9225112733375057e-05,
+      "loss": 2.3975,
+      "step": 960
+    },
+    {
+      "epoch": 0.38,
+      "learning_rate": 1.9217127246715344e-05,
+      "loss": 2.4867,
+      "step": 965
+    },
+    {
+      "epoch": 0.38,
+      "learning_rate": 1.92091024999251e-05,
+      "loss": 2.4757,
+      "step": 970
+    },
+    {
+      "epoch": 0.38,
+      "learning_rate": 1.9201038527185546e-05,
+      "loss": 2.4164,
+      "step": 975
+    },
+    {
+      "epoch": 0.39,
+      "learning_rate": 1.919293536284497e-05,
+      "loss": 2.4024,
+      "step": 980
+    },
+    {
+      "epoch": 0.39,
+      "learning_rate": 1.9184793041418607e-05,
+      "loss": 2.4706,
+      "step": 985
+    },
+    {
+      "epoch": 0.39,
+      "learning_rate": 1.917661159758848e-05,
+      "loss": 2.465,
+      "step": 990
+    },
+    {
+      "epoch": 0.39,
+      "learning_rate": 1.9168391066203248e-05,
+      "loss": 2.555,
+      "step": 995
+    },
+    {
+      "epoch": 0.39,
+      "learning_rate": 1.9160131482278068e-05,
+      "loss": 2.4465,
+      "step": 1000
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 7611,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "total_flos": 1.343199895683072e+16,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e79a241514d107c9eee3780ef385cef96ff7663902d9e94f80f6c1be656e9697
+size 4920

checkpoint-1100/README.md

@@ -0,0 +1,204 @@
+---
+library_name: peft
+base_model: Qwen/Qwen1.5-0.5B-Chat
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+
+### Framework versions
+
+- PEFT 0.8.2

checkpoint-1100/adapter_config.json

@@ -0,0 +1,27 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen1.5-0.5B-Chat",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_rslora": false
+}

checkpoint-1100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:003d6eff75d5a884430e864481a14e49bdff71d867c5c812fcb459ba0f0bd58c
+size 3158328

checkpoint-1100/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

checkpoint-1100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:24338cc9c46eca94138f5bedcc8f6265318b6f11a967a6e99ff471b454784d7f
+size 6372346

checkpoint-1100/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d4008fe917b3fbfa4c268bf4981887a5eab68261905ca5a71913bfa192d198b
+size 14244

checkpoint-1100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:551022aae2533e7211c8c5ba6c2d8ab576bfc90187789b68a39d22bf1194c76d
+size 1064

checkpoint-1100/special_tokens_map.json

@@ -0,0 +1,20 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-1100/tokenizer_config.json

@@ -0,0 +1,44 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content']}}{% if (loop.last and add_generation_prompt) or not loop.last %}{{ '<|im_end|>' + '\n'}}{% endif %}{% endfor %}{% if add_generation_prompt and messages[-1]['role'] != 'assistant' %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-1100/trainer_state.json

@@ -0,0 +1,1341 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.43358297201419,
+  "eval_steps": 500,
+  "global_step": 1100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0,
+      "learning_rate": 1.999997870262096e-05,
+      "loss": 2.714,
+      "step": 5
+    },
+    {
+      "epoch": 0.0,
+      "learning_rate": 1.999991481057455e-05,
+      "loss": 2.7679,
+      "step": 10
+    },
+    {
+      "epoch": 0.01,
+      "learning_rate": 1.9999808324132915e-05,
+      "loss": 2.6887,
+      "step": 15
+    },
+    {
+      "epoch": 0.01,
+      "learning_rate": 1.999965924374964e-05,
+      "loss": 2.6986,
+      "step": 20
+    },
+    {
+      "epoch": 0.01,
+      "learning_rate": 1.999946757005972e-05,
+      "loss": 2.7536,
+      "step": 25
+    },
+    {
+      "epoch": 0.01,
+      "learning_rate": 1.9999233303879592e-05,
+      "loss": 2.7516,
+      "step": 30
+    },
+    {
+      "epoch": 0.01,
+      "learning_rate": 1.99989564462071e-05,
+      "loss": 2.6283,
+      "step": 35
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 1.999863699822152e-05,
+      "loss": 2.708,
+      "step": 40
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 1.9998274961283523e-05,
+      "loss": 2.6932,
+      "step": 45
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 1.9997870336935207e-05,
+      "loss": 2.6321,
+      "step": 50
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 1.9997423126900056e-05,
+      "loss": 2.7129,
+      "step": 55
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 1.9996933333082945e-05,
+      "loss": 2.6179,
+      "step": 60
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 1.9996400957570148e-05,
+      "loss": 2.6052,
+      "step": 65
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 1.99958260026293e-05,
+      "loss": 2.7054,
+      "step": 70
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 1.9995208470709405e-05,
+      "loss": 2.6568,
+      "step": 75
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 1.9994548364440836e-05,
+      "loss": 2.6436,
+      "step": 80
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 1.999384568663529e-05,
+      "loss": 2.7091,
+      "step": 85
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 1.9993100440285805e-05,
+      "loss": 2.5557,
+      "step": 90
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 1.999231262856675e-05,
+      "loss": 2.5945,
+      "step": 95
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 1.999148225483378e-05,
+      "loss": 2.4729,
+      "step": 100
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 1.9990609322623854e-05,
+      "loss": 2.6724,
+      "step": 105
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 1.9989693835655205e-05,
+      "loss": 2.5272,
+      "step": 110
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 1.9988735797827336e-05,
+      "loss": 2.6145,
+      "step": 115
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 1.9987735213220975e-05,
+      "loss": 2.6772,
+      "step": 120
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 1.9986692086098095e-05,
+      "loss": 2.6388,
+      "step": 125
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 1.998560642090187e-05,
+      "loss": 2.4973,
+      "step": 130
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 1.998447822225666e-05,
+      "loss": 2.5338,
+      "step": 135
+    },
+    {
+      "epoch": 0.06,
+      "learning_rate": 1.9983307494968e-05,
+      "loss": 2.6469,
+      "step": 140
+    },
+    {
+      "epoch": 0.06,
+      "learning_rate": 1.9982094244022582e-05,
+      "loss": 2.6008,
+      "step": 145
+    },
+    {
+      "epoch": 0.06,
+      "learning_rate": 1.9980838474588214e-05,
+      "loss": 2.5936,
+      "step": 150
+    },
+    {
+      "epoch": 0.06,
+      "learning_rate": 1.9979540192013814e-05,
+      "loss": 2.5862,
+      "step": 155
+    },
+    {
+      "epoch": 0.06,
+      "learning_rate": 1.997819940182939e-05,
+      "loss": 2.568,
+      "step": 160
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 1.9976816109746e-05,
+      "loss": 2.5343,
+      "step": 165
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 1.9975390321655745e-05,
+      "loss": 2.5575,
+      "step": 170
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 1.9973922043631737e-05,
+      "loss": 2.5399,
+      "step": 175
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 1.9972411281928068e-05,
+      "loss": 2.6394,
+      "step": 180
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 1.9970858042979794e-05,
+      "loss": 2.5378,
+      "step": 185
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 1.9969262333402893e-05,
+      "loss": 2.5373,
+      "step": 190
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 1.9967624159994262e-05,
+      "loss": 2.5414,
+      "step": 195
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 1.9965943529731646e-05,
+      "loss": 2.5606,
+      "step": 200
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 1.9964220449773664e-05,
+      "loss": 2.5154,
+      "step": 205
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 1.9962454927459723e-05,
+      "loss": 2.5468,
+      "step": 210
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 1.9960646970310027e-05,
+      "loss": 2.5137,
+      "step": 215
+    },
+    {
+      "epoch": 0.09,
+      "learning_rate": 1.9958796586025527e-05,
+      "loss": 2.5453,
+      "step": 220
+    },
+    {
+      "epoch": 0.09,
+      "learning_rate": 1.9956903782487885e-05,
+      "loss": 2.5325,
+      "step": 225
+    },
+    {
+      "epoch": 0.09,
+      "learning_rate": 1.9954968567759456e-05,
+      "loss": 2.5054,
+      "step": 230
+    },
+    {
+      "epoch": 0.09,
+      "learning_rate": 1.9952990950083236e-05,
+      "loss": 2.5139,
+      "step": 235
+    },
+    {
+      "epoch": 0.09,
+      "learning_rate": 1.995097093788285e-05,
+      "loss": 2.5097,
+      "step": 240
+    },
+    {
+      "epoch": 0.1,
+      "learning_rate": 1.994890853976248e-05,
+      "loss": 2.5546,
+      "step": 245
+    },
+    {
+      "epoch": 0.1,
+      "learning_rate": 1.994680376450686e-05,
+      "loss": 2.4609,
+      "step": 250
+    },
+    {
+      "epoch": 0.1,
+      "learning_rate": 1.994465662108124e-05,
+      "loss": 2.5063,
+      "step": 255
+    },
+    {
+      "epoch": 0.1,
+      "learning_rate": 1.9942467118631322e-05,
+      "loss": 2.5162,
+      "step": 260
+    },
+    {
+      "epoch": 0.1,
+      "learning_rate": 1.994023526648323e-05,
+      "loss": 2.5317,
+      "step": 265
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 1.9937961074143492e-05,
+      "loss": 2.4621,
+      "step": 270
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 1.9935644551298976e-05,
+      "loss": 2.5879,
+      "step": 275
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 1.993328570781685e-05,
+      "loss": 2.4742,
+      "step": 280
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 1.993088455374456e-05,
+      "loss": 2.4661,
+      "step": 285
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 1.992844109930975e-05,
+      "loss": 2.4943,
+      "step": 290
+    },
+    {
+      "epoch": 0.12,
+      "learning_rate": 1.9925955354920265e-05,
+      "loss": 2.4322,
+      "step": 295
+    },
+    {
+      "epoch": 0.12,
+      "learning_rate": 1.9923427331164072e-05,
+      "loss": 2.484,
+      "step": 300
+    },
+    {
+      "epoch": 0.12,
+      "learning_rate": 1.9920857038809223e-05,
+      "loss": 2.5334,
+      "step": 305
+    },
+    {
+      "epoch": 0.12,
+      "learning_rate": 1.991824448880382e-05,
+      "loss": 2.6554,
+      "step": 310
+    },
+    {
+      "epoch": 0.12,
+      "learning_rate": 1.9915589692275955e-05,
+      "loss": 2.4749,
+      "step": 315
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 1.991289266053367e-05,
+      "loss": 2.5076,
+      "step": 320
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 1.9910153405064904e-05,
+      "loss": 2.4635,
+      "step": 325
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 1.990737193753745e-05,
+      "loss": 2.5443,
+      "step": 330
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 1.9904548269798906e-05,
+      "loss": 2.6012,
+      "step": 335
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 1.990168241387662e-05,
+      "loss": 2.4818,
+      "step": 340
+    },
+    {
+      "epoch": 0.14,
+      "learning_rate": 1.9898774381977618e-05,
+      "loss": 2.4717,
+      "step": 345
+    },
+    {
+      "epoch": 0.14,
+      "learning_rate": 1.989582418648861e-05,
+      "loss": 2.4756,
+      "step": 350
+    },
+    {
+      "epoch": 0.14,
+      "learning_rate": 1.9892831839975874e-05,
+      "loss": 2.4868,
+      "step": 355
+    },
+    {
+      "epoch": 0.14,
+      "learning_rate": 1.9889797355185237e-05,
+      "loss": 2.5652,
+      "step": 360
+    },
+    {
+      "epoch": 0.14,
+      "learning_rate": 1.9886720745042017e-05,
+      "loss": 2.3907,
+      "step": 365
+    },
+    {
+      "epoch": 0.15,
+      "learning_rate": 1.988360202265096e-05,
+      "loss": 2.4984,
+      "step": 370
+    },
+    {
+      "epoch": 0.15,
+      "learning_rate": 1.9880441201296186e-05,
+      "loss": 2.5654,
+      "step": 375
+    },
+    {
+      "epoch": 0.15,
+      "learning_rate": 1.987723829444114e-05,
+      "loss": 2.581,
+      "step": 380
+    },
+    {
+      "epoch": 0.15,
+      "learning_rate": 1.9873993315728523e-05,
+      "loss": 2.5189,
+      "step": 385
+    },
+    {
+      "epoch": 0.15,
+      "learning_rate": 1.987070627898025e-05,
+      "loss": 2.491,
+      "step": 390
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 1.9867377198197367e-05,
+      "loss": 2.4875,
+      "step": 395
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 1.9864006087560016e-05,
+      "loss": 2.4828,
+      "step": 400
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 1.9860592961427358e-05,
+      "loss": 2.5298,
+      "step": 405
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 1.9857137834337527e-05,
+      "loss": 2.4873,
+      "step": 410
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 1.985364072100755e-05,
+      "loss": 2.5371,
+      "step": 415
+    },
+    {
+      "epoch": 0.17,
+      "learning_rate": 1.98501016363333e-05,
+      "loss": 2.4167,
+      "step": 420
+    },
+    {
+      "epoch": 0.17,
+      "learning_rate": 1.9846520595389415e-05,
+      "loss": 2.4312,
+      "step": 425
+    },
+    {
+      "epoch": 0.17,
+      "learning_rate": 1.984289761342926e-05,
+      "loss": 2.4471,
+      "step": 430
+    },
+    {
+      "epoch": 0.17,
+      "learning_rate": 1.9839232705884836e-05,
+      "loss": 2.4971,
+      "step": 435
+    },
+    {
+      "epoch": 0.17,
+      "learning_rate": 1.9835525888366727e-05,
+      "loss": 2.5206,
+      "step": 440
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 1.9831777176664035e-05,
+      "loss": 2.5067,
+      "step": 445
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 1.9827986586744302e-05,
+      "loss": 2.4903,
+      "step": 450
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 1.982415413475346e-05,
+      "loss": 2.4391,
+      "step": 455
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 1.9820279837015742e-05,
+      "loss": 2.4358,
+      "step": 460
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 1.981636371003363e-05,
+      "loss": 2.4809,
+      "step": 465
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 1.9812405770487763e-05,
+      "loss": 2.516,
+      "step": 470
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 1.9808406035236897e-05,
+      "loss": 2.4091,
+      "step": 475
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 1.9804364521317806e-05,
+      "loss": 2.5363,
+      "step": 480
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 1.9800281245945217e-05,
+      "loss": 2.51,
+      "step": 485
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 1.9796156226511747e-05,
+      "loss": 2.5049,
+      "step": 490
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 1.9791989480587815e-05,
+      "loss": 2.4722,
+      "step": 495
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 1.978778102592157e-05,
+      "loss": 2.4111,
+      "step": 500
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 1.9783530880438832e-05,
+      "loss": 2.4078,
+      "step": 505
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 1.9779239062242988e-05,
+      "loss": 2.3981,
+      "step": 510
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 1.9774905589614935e-05,
+      "loss": 2.5459,
+      "step": 515
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 1.977053048101299e-05,
+      "loss": 2.4432,
+      "step": 520
+    },
+    {
+      "epoch": 0.21,
+      "learning_rate": 1.976611375507283e-05,
+      "loss": 2.5329,
+      "step": 525
+    },
+    {
+      "epoch": 0.21,
+      "learning_rate": 1.9761655430607384e-05,
+      "loss": 2.5107,
+      "step": 530
+    },
+    {
+      "epoch": 0.21,
+      "learning_rate": 1.975715552660678e-05,
+      "loss": 2.4389,
+      "step": 535
+    },
+    {
+      "epoch": 0.21,
+      "learning_rate": 1.9752614062238256e-05,
+      "loss": 2.4939,
+      "step": 540
+    },
+    {
+      "epoch": 0.21,
+      "learning_rate": 1.974803105684606e-05,
+      "loss": 2.5349,
+      "step": 545
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 1.9743406529951403e-05,
+      "loss": 2.5268,
+      "step": 550
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 1.9738740501252337e-05,
+      "loss": 2.4477,
+      "step": 555
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 1.9734032990623702e-05,
+      "loss": 2.4597,
+      "step": 560
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 1.972928401811703e-05,
+      "loss": 2.4938,
+      "step": 565
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 1.9724493603960443e-05,
+      "loss": 2.4756,
+      "step": 570
+    },
+    {
+      "epoch": 0.23,
+      "learning_rate": 1.9719661768558604e-05,
+      "loss": 2.3664,
+      "step": 575
+    },
+    {
+      "epoch": 0.23,
+      "learning_rate": 1.9714788532492595e-05,
+      "loss": 2.5166,
+      "step": 580
+    },
+    {
+      "epoch": 0.23,
+      "learning_rate": 1.9709873916519853e-05,
+      "loss": 2.4143,
+      "step": 585
+    },
+    {
+      "epoch": 0.23,
+      "learning_rate": 1.9704917941574053e-05,
+      "loss": 2.4666,
+      "step": 590
+    },
+    {
+      "epoch": 0.23,
+      "learning_rate": 1.9699920628765065e-05,
+      "loss": 2.4617,
+      "step": 595
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 1.969488199937881e-05,
+      "loss": 2.4912,
+      "step": 600
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 1.9689802074877216e-05,
+      "loss": 2.4386,
+      "step": 605
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 1.9684680876898096e-05,
+      "loss": 2.5002,
+      "step": 610
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 1.967951842725507e-05,
+      "loss": 2.5668,
+      "step": 615
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 1.9674314747937462e-05,
+      "loss": 2.4955,
+      "step": 620
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 1.9669069861110225e-05,
+      "loss": 2.5317,
+      "step": 625
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 1.9663783789113827e-05,
+      "loss": 2.4709,
+      "step": 630
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 1.9658456554464157e-05,
+      "loss": 2.4066,
+      "step": 635
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 1.9653088179852448e-05,
+      "loss": 2.4462,
+      "step": 640
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 1.9647678688145163e-05,
+      "loss": 2.5093,
+      "step": 645
+    },
+    {
+      "epoch": 0.26,
+      "learning_rate": 1.9642228102383894e-05,
+      "loss": 2.4634,
+      "step": 650
+    },
+    {
+      "epoch": 0.26,
+      "learning_rate": 1.9636736445785288e-05,
+      "loss": 2.424,
+      "step": 655
+    },
+    {
+      "epoch": 0.26,
+      "learning_rate": 1.963120374174092e-05,
+      "loss": 2.4968,
+      "step": 660
+    },
+    {
+      "epoch": 0.26,
+      "learning_rate": 1.9625630013817204e-05,
+      "loss": 2.3947,
+      "step": 665
+    },
+    {
+      "epoch": 0.26,
+      "learning_rate": 1.9620015285755306e-05,
+      "loss": 2.3954,
+      "step": 670
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 1.961435958147102e-05,
+      "loss": 2.3764,
+      "step": 675
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 1.9608662925054684e-05,
+      "loss": 2.386,
+      "step": 680
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 1.960292534077107e-05,
+      "loss": 2.4218,
+      "step": 685
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 1.9597146853059273e-05,
+      "loss": 2.4479,
+      "step": 690
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 1.959132748653263e-05,
+      "loss": 2.4301,
+      "step": 695
+    },
+    {
+      "epoch": 0.28,
+      "learning_rate": 1.9585467265978585e-05,
+      "loss": 2.5069,
+      "step": 700
+    },
+    {
+      "epoch": 0.28,
+      "learning_rate": 1.957956621635861e-05,
+      "loss": 2.4144,
+      "step": 705
+    },
+    {
+      "epoch": 0.28,
+      "learning_rate": 1.9573624362808078e-05,
+      "loss": 2.4491,
+      "step": 710
+    },
+    {
+      "epoch": 0.28,
+      "learning_rate": 1.9567641730636174e-05,
+      "loss": 2.4723,
+      "step": 715
+    },
+    {
+      "epoch": 0.28,
+      "learning_rate": 1.956161834532578e-05,
+      "loss": 2.4839,
+      "step": 720
+    },
+    {
+      "epoch": 0.29,
+      "learning_rate": 1.955555423253335e-05,
+      "loss": 2.4881,
+      "step": 725
+    },
+    {
+      "epoch": 0.29,
+      "learning_rate": 1.9549449418088832e-05,
+      "loss": 2.5076,
+      "step": 730
+    },
+    {
+      "epoch": 0.29,
+      "learning_rate": 1.9543303927995536e-05,
+      "loss": 2.4704,
+      "step": 735
+    },
+    {
+      "epoch": 0.29,
+      "learning_rate": 1.9537117788430024e-05,
+      "loss": 2.3958,
+      "step": 740
+    },
+    {
+      "epoch": 0.29,
+      "learning_rate": 1.953089102574201e-05,
+      "loss": 2.5256,
+      "step": 745
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 1.9524623666454243e-05,
+      "loss": 2.4658,
+      "step": 750
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 1.951831573726238e-05,
+      "loss": 2.4636,
+      "step": 755
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 1.9511967265034904e-05,
+      "loss": 2.4249,
+      "step": 760
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 1.9505578276812964e-05,
+      "loss": 2.5412,
+      "step": 765
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 1.9499148799810314e-05,
+      "loss": 2.4355,
+      "step": 770
+    },
+    {
+      "epoch": 0.31,
+      "learning_rate": 1.949267886141315e-05,
+      "loss": 2.5814,
+      "step": 775
+    },
+    {
+      "epoch": 0.31,
+      "learning_rate": 1.948616848918002e-05,
+      "loss": 2.5195,
+      "step": 780
+    },
+    {
+      "epoch": 0.31,
+      "learning_rate": 1.9479617710841693e-05,
+      "loss": 2.5177,
+      "step": 785
+    },
+    {
+      "epoch": 0.31,
+      "learning_rate": 1.9473026554301057e-05,
+      "loss": 2.4418,
+      "step": 790
+    },
+    {
+      "epoch": 0.31,
+      "learning_rate": 1.946639504763298e-05,
+      "loss": 2.5187,
+      "step": 795
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 1.94597232190842e-05,
+      "loss": 2.479,
+      "step": 800
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 1.9453011097073217e-05,
+      "loss": 2.4451,
+      "step": 805
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 1.9446258710190152e-05,
+      "loss": 2.5426,
+      "step": 810
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 1.9439466087196627e-05,
+      "loss": 2.4299,
+      "step": 815
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 1.943263325702566e-05,
+      "loss": 2.5317,
+      "step": 820
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 1.9425760248781525e-05,
+      "loss": 2.4384,
+      "step": 825
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 1.9418847091739638e-05,
+      "loss": 2.4445,
+      "step": 830
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 1.9411893815346418e-05,
+      "loss": 2.4872,
+      "step": 835
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 1.9404900449219178e-05,
+      "loss": 2.5123,
+      "step": 840
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 1.9397867023146e-05,
+      "loss": 2.3569,
+      "step": 845
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 1.9390793567085585e-05,
+      "loss": 2.467,
+      "step": 850
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 1.9383680111167146e-05,
+      "loss": 2.4097,
+      "step": 855
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 1.937652668569028e-05,
+      "loss": 2.4653,
+      "step": 860
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 1.9369333321124832e-05,
+      "loss": 2.4164,
+      "step": 865
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 1.936210004811076e-05,
+      "loss": 2.3906,
+      "step": 870
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 1.9354826897458016e-05,
+      "loss": 2.4433,
+      "step": 875
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 1.9347513900146412e-05,
+      "loss": 2.42,
+      "step": 880
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 1.9340161087325483e-05,
+      "loss": 2.4606,
+      "step": 885
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 1.9332768490314354e-05,
+      "loss": 2.4242,
+      "step": 890
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 1.9325336140601612e-05,
+      "loss": 2.461,
+      "step": 895
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 1.931786406984518e-05,
+      "loss": 2.3949,
+      "step": 900
+    },
+    {
+      "epoch": 0.36,
+      "learning_rate": 1.9310352309872153e-05,
+      "loss": 2.4449,
+      "step": 905
+    },
+    {
+      "epoch": 0.36,
+      "learning_rate": 1.9302800892678693e-05,
+      "loss": 2.4864,
+      "step": 910
+    },
+    {
+      "epoch": 0.36,
+      "learning_rate": 1.9295209850429884e-05,
+      "loss": 2.5432,
+      "step": 915
+    },
+    {
+      "epoch": 0.36,
+      "learning_rate": 1.9287579215459585e-05,
+      "loss": 2.4357,
+      "step": 920
+    },
+    {
+      "epoch": 0.36,
+      "learning_rate": 1.9279909020270294e-05,
+      "loss": 2.4625,
+      "step": 925
+    },
+    {
+      "epoch": 0.37,
+      "learning_rate": 1.9272199297533027e-05,
+      "loss": 2.3937,
+      "step": 930
+    },
+    {
+      "epoch": 0.37,
+      "learning_rate": 1.9264450080087163e-05,
+      "loss": 2.5051,
+      "step": 935
+    },
+    {
+      "epoch": 0.37,
+      "learning_rate": 1.9256661400940303e-05,
+      "loss": 2.4894,
+      "step": 940
+    },
+    {
+      "epoch": 0.37,
+      "learning_rate": 1.9248833293268144e-05,
+      "loss": 2.4117,
+      "step": 945
+    },
+    {
+      "epoch": 0.37,
+      "learning_rate": 1.9240965790414312e-05,
+      "loss": 2.4892,
+      "step": 950
+    },
+    {
+      "epoch": 0.38,
+      "learning_rate": 1.923305892589025e-05,
+      "loss": 2.4059,
+      "step": 955
+    },
+    {
+      "epoch": 0.38,
+      "learning_rate": 1.9225112733375057e-05,
+      "loss": 2.3975,
+      "step": 960
+    },
+    {
+      "epoch": 0.38,
+      "learning_rate": 1.9217127246715344e-05,
+      "loss": 2.4867,
+      "step": 965
+    },
+    {
+      "epoch": 0.38,
+      "learning_rate": 1.92091024999251e-05,
+      "loss": 2.4757,
+      "step": 970
+    },
+    {
+      "epoch": 0.38,
+      "learning_rate": 1.9201038527185546e-05,
+      "loss": 2.4164,
+      "step": 975
+    },
+    {
+      "epoch": 0.39,
+      "learning_rate": 1.919293536284497e-05,
+      "loss": 2.4024,
+      "step": 980
+    },
+    {
+      "epoch": 0.39,
+      "learning_rate": 1.9184793041418607e-05,
+      "loss": 2.4706,
+      "step": 985
+    },
+    {
+      "epoch": 0.39,
+      "learning_rate": 1.917661159758848e-05,
+      "loss": 2.465,
+      "step": 990
+    },
+    {
+      "epoch": 0.39,
+      "learning_rate": 1.9168391066203248e-05,
+      "loss": 2.555,
+      "step": 995
+    },
+    {
+      "epoch": 0.39,
+      "learning_rate": 1.9160131482278068e-05,
+      "loss": 2.4465,
+      "step": 1000
+    },
+    {
+      "epoch": 0.4,
+      "learning_rate": 1.9151832880994438e-05,
+      "loss": 2.4014,
+      "step": 1005
+    },
+    {
+      "epoch": 0.4,
+      "learning_rate": 1.914349529770005e-05,
+      "loss": 2.4358,
+      "step": 1010
+    },
+    {
+      "epoch": 0.4,
+      "learning_rate": 1.9135118767908637e-05,
+      "loss": 2.402,
+      "step": 1015
+    },
+    {
+      "epoch": 0.4,
+      "learning_rate": 1.9126703327299822e-05,
+      "loss": 2.3952,
+      "step": 1020
+    },
+    {
+      "epoch": 0.4,
+      "learning_rate": 1.9118249011718975e-05,
+      "loss": 2.4051,
+      "step": 1025
+    },
+    {
+      "epoch": 0.41,
+      "learning_rate": 1.9109755857177053e-05,
+      "loss": 2.4198,
+      "step": 1030
+    },
+    {
+      "epoch": 0.41,
+      "learning_rate": 1.910122389985043e-05,
+      "loss": 2.4414,
+      "step": 1035
+    },
+    {
+      "epoch": 0.41,
+      "learning_rate": 1.909265317608078e-05,
+      "loss": 2.4688,
+      "step": 1040
+    },
+    {
+      "epoch": 0.41,
+      "learning_rate": 1.9084043722374895e-05,
+      "loss": 2.3766,
+      "step": 1045
+    },
+    {
+      "epoch": 0.41,
+      "learning_rate": 1.907539557540453e-05,
+      "loss": 2.3966,
+      "step": 1050
+    },
+    {
+      "epoch": 0.42,
+      "learning_rate": 1.9066708772006262e-05,
+      "loss": 2.5027,
+      "step": 1055
+    },
+    {
+      "epoch": 0.42,
+      "learning_rate": 1.9057983349181316e-05,
+      "loss": 2.4904,
+      "step": 1060
+    },
+    {
+      "epoch": 0.42,
+      "learning_rate": 1.904921934409542e-05,
+      "loss": 2.4296,
+      "step": 1065
+    },
+    {
+      "epoch": 0.42,
+      "learning_rate": 1.9040416794078648e-05,
+      "loss": 2.4294,
+      "step": 1070
+    },
+    {
+      "epoch": 0.42,
+      "learning_rate": 1.903157573662524e-05,
+      "loss": 2.4442,
+      "step": 1075
+    },
+    {
+      "epoch": 0.43,
+      "learning_rate": 1.902269620939347e-05,
+      "loss": 2.382,
+      "step": 1080
+    },
+    {
+      "epoch": 0.43,
+      "learning_rate": 1.901377825020547e-05,
+      "loss": 2.5004,
+      "step": 1085
+    },
+    {
+      "epoch": 0.43,
+      "learning_rate": 1.9004821897047067e-05,
+      "loss": 2.3837,
+      "step": 1090
+    },
+    {
+      "epoch": 0.43,
+      "learning_rate": 1.899582718806764e-05,
+      "loss": 2.3654,
+      "step": 1095
+    },
+    {
+      "epoch": 0.43,
+      "learning_rate": 1.8986794161579927e-05,
+      "loss": 2.4624,
+      "step": 1100
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 7611,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "total_flos": 1.474639505031168e+16,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e79a241514d107c9eee3780ef385cef96ff7663902d9e94f80f6c1be656e9697
+size 4920

checkpoint-1200/README.md

@@ -0,0 +1,204 @@
+---
+library_name: peft
+base_model: Qwen/Qwen1.5-0.5B-Chat
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+
+### Framework versions
+
+- PEFT 0.8.2

checkpoint-1200/adapter_config.json

@@ -0,0 +1,27 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen1.5-0.5B-Chat",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_rslora": false
+}

checkpoint-1200/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e07f95d45b4a8fa741fa925f764dc5a1f5bcf45d2c375bf8389e7c1a7e16f9d3
+size 3158328

checkpoint-1200/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

checkpoint-1200/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aaff3c5711ef15c9630e5531e6268489578e6e98f584c31cc985fd64b3822101
+size 6372346