README.md

---
license: apache-2.0
language:
  - zh
  - en
tags:
  - qwen
  - lora
  - peft
  - large-language-model
  - quantitative-finance
  - stock-market
  - trading-strategy
  - investment-education
  - financial-explainer
  - text-generation
  - instruction-following
base_model: Qwen/Qwen-1_8B-Chat
pipeline_tag: text-generation
widget:
  # Example of how to use the model with PEFT
  # (You'll need to adjust this based on how Qwen LoRA models are typically loaded)
  - text: "请用大白话解释什么是移动平均线？"
    example_title: "Explain Moving Average"
---

# Qwen-1.8B-Chat LoRA for Stock Market Quantitative Education (股票量化投教LoRA模型)

This repository contains LoRA (Low-Rank Adaptation) adapters fine-tuned on the `Qwen/Qwen-1_8B-Chat` model.
The goal of this fine-tuning is to create an AI assistant that can explain stock market and quantitative trading concepts in plain language ("大白话"), making these topics more accessible to beginners.

## Model Description

This model is a PEFT-LoRA adaptation of the `Qwen/Qwen-1_8B-Chat` large language model. It has been fine-tuned on a small, custom dataset of ~20 instruction-response pairs focused on financial education. Due to the small dataset size, this model should be considered **experimental and for demonstration purposes**.

**Developed by:** 天算AI科技研发实验室 (Natural Algorithm AI R&D Lab) - jinv2

## Intended Uses & Limitations

**Intended Uses:**

*   Educational tool for understanding basic stock market and quantitative trading terms.
*   Generating simple explanations of financial concepts.
*   Demonstrating the LoRA fine-tuning process on a chat model for a specific domain.

**Limitations:**

*   **Not for Financial Advice:** The information provided by this model is strictly for educational purposes and should NOT be considered financial advice. Always consult with a qualified financial advisor before making investment decisions.
*   **Limited Knowledge:** Fine-tuned on a very small dataset. Its knowledge is restricted and may not be comprehensive or entirely accurate.
*   **Potential for Hallucinations:** Like all LLMs, it may generate incorrect or nonsensical information.
*   **Overfitting:** Due to the small dataset, the model may be overfit to the training examples.
*   **Bias:** The training data may contain biases, which could be reflected in the model's responses.
*   **Requires Base Model:** These are LoRA adapters and require the original `Qwen/Qwen-1_8B-Chat` base model to be loaded first.

## How to Use with PEFT

You would typically load the base model and then apply these LoRA adapters using the PEFT library.

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base_model_name = "Qwen/Qwen-1_8B-Chat"
adapter_model_name = "jinv2/qwen-1.8b-chat-lora-stock-quant-edu" # Replace with your actual model name on Hub

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_name, trust_remote_code=True)
if tokenizer.pad_token_id is None:
    tokenizer.pad_token_id = tokenizer.eos_token_id # Or <|endoftext|> ID: 151643

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    torch_dtype=torch.float16, # or "auto"
    device_map="auto",
    trust_remote_code=True
)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, adapter_model_name)
model = model.eval() # Set to evaluation mode

# Example Inference (Qwen chat format)
prompt = "请用大白话解释什么是MACD指标？"
# For Qwen-Chat, using model.chat() is recommended
response, history = model.chat(tokenizer, prompt, history=None, system="You are a helpful financial education assistant.")
print(response)

# Alternative generic generation
# messages = [
#     {"role": "system", "content": "You are a helpful financial education assistant."},
#     {"role": "user", "content": prompt}
# ]
# text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
# model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
# generated_ids = model.generate(model_inputs.input_ids, max_new_tokens=512)
# generated_ids = [output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)]
# response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
# print(response)

Training Details

Base Model: Qwen/Qwen-1_8B-Chat

Fine-tuning Method: LoRA (Low-Rank Adaptation) via PEFT and TRL SFTTrainer.

Dataset: ~20 custom instruction-response pairs for financial education.

Training Configuration (Key Parameters):

LoRA r: 8

LoRA alpha: 16

Target Modules: c_attn, c_proj, w1, w2

Optimizer: AdamW (default from Trainer)

Precision: FP32 (due to issues with FP16/BF16 GradScaler in the environment)

Epochs: ~17 (based on 80 steps)

Batch Size (effective): 4 (per_device_train_batch_size=1, gradient_accumulation_steps=4)

Learning Rate: 2e-4

Max Sequence Length: 512

Disclaimer

This model is provided "as-is" without any warranty. The developers are not responsible for any outcomes resulting from the use of this model. Always verify information and use at your own risk.

Copyright Information:

© 天算AI科技研发实验室 (Natural Algorithm AI R&D Lab) - jinv2
All rights reserved unless otherwise specified by the license.