aijadugar/qwen2.5-0.5b-sql-lora

Qwen2.5-0.5B-Instruct fine-tuned with a hand-written LoRA (Text-to-SQL)

This model is Qwen/Qwen2.5-0.5B-Instruct fine-tuned on a small slice of b-mc2/sql-create-context using a LoRA adapter implemented from scratch in plain PyTorch (no peft library was used for training -- see the LoRALinear class in the training notebook), then merged back into the base weights.

Training details

• Base model: Qwen/Qwen2.5-0.5B-Instruct
• Method: LoRA (rank=8, alpha=16), applied to q_proj, k_proj, v_proj, o_proj
• Dataset: 250 examples from b-mc2/sql-create-context
• Hardware: Kaggle, 1x Tesla T4

How to use

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("YOUR_USERNAME/YOUR_REPO_NAME")
tokenizer = AutoTokenizer.from_pretrained("YOUR_USERNAME/YOUR_REPO_NAME")

This model is a learning exercise, not a production system -- expect rough edges.