---
license: mit
language:
- en
base_model:
- microsoft/DialoGPT-medium
model_type: gpt2
tags:
- conversational
- fine-tuned
- dialogpt
- kimi-k2
pipeline_tag: text-generation
library_name: transformers
---

# DialoGPT-medium-distill-Kimi-K2-Instruct

This model is a fine-tuned version of __**microsoft/DialoGPT-medium**__, specialized for a custom persona and critical knowledge injection. It has been trained to balance conversational flexibility with specific factual recall.

## Model Description

- **Model type:** Causal Language Model
- **Language(s):** English
- **Base Model:** DialoGPT-medium
- **Total Parameters:** 406.3M _(Post-fine-tune expanded state)_

## Intended Uses & Limitations

This model is designed for creative assistant tasks and casual conversation.
- **Direct Use:** Chatbots, creative storytelling, and persona-driven interactions.
- **Limitations:** Due to the small dataset size and "creative" training, the model may occasionally hallucinate or provide non-literal answers (e.g., creative definitions of common objects).

## Training Procedure
The model underwent a full fine-tune on a custom dataset consisting of critical facts and casual chat examples.

### Training Hyperparameters

- **Learning Rate:** 2e-5
- **Epochs:** 5
- **Batch Size:** 4 _(with gradient accumulation)_
- **Precision:** Mixed Precision _(FP16)_
- **Loss achieved:** 3.264037

## Weight Analysis

Post-training analysis showed a significant shift in the LM Head weights _(Absolute Shift: 4.4164)_, indicating a strong adaptation to the new conversational style while maintaining structural grammar stability in the transformer layers.

## How to Use
```py
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Fu01978/DialoGPT-medium-distill-Kimi-K2-Instruct")
model = AutoModelForCausalLM.from_pretrained("Fu01978/DialoGPT-medium-distill-Kimi-K2-Instruct")

# For best results, use a temperature between 0.7 and 0.85
```