---
base_model: Qwen/Qwen2.5-Coder-3B-Instruct
library_name: peft
tags:
  - sql
  - postgresql
  - text-to-sql
  - code
  - lora
  - qlora
  - qwen2
license: apache-2.0
pipeline_tag: text-generation
---

# postgres-llm-qlora

QLoRA fine-tuned adapter for **PostgreSQL SQL / text-to-SQL** on [Qwen/Qwen2.5-Coder-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct). Trained on the [neurondb/neurondb-postgresql-sql](https://huggingface.co/datasets/neurondb/neurondb-postgresql-sql) dataset. Use this adapter with the base model to generate PostgreSQL-compatible SQL from natural language instructions. Use this adapter with the base model to generate PostgreSQL-compatible SQL from natural language instructions.

## Model details

- **Base model:** [Qwen/Qwen2.5-Coder-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct)
- **Training dataset:** [neurondb/neurondb-postgresql-sql](https://huggingface.co/datasets/neurondb/neurondb-postgresql-sql) (~212k rows: question, schema, sql, with sources including PostgreSQL regression tests, docs, contrib, pgTAP, plpgsql, sql_create_context, community SQL datasets)
- **Method:** QLoRA (4-bit base + LoRA), rank 64, alpha 128
- **Training:** 37,299 steps, 3 epochs on the dataset train split
- **Final metrics:** ~0.34 loss, ~89% mean token accuracy

## Usage

### With transformers + PEFT (recommended on 8GB GPU: load base in 4-bit)

```python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

adapter_path = "YOUR_USER/postgres-llm-qlora"  # or local path
base_model = "Qwen/Qwen2.5-Coder-3B-Instruct"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
)
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(model, adapter_path)
tokenizer = AutoTokenizer.from_pretrained(adapter_path, trust_remote_code=True)
model.eval()

prompt = "Create a table users with columns id (serial primary key), name (text), email (text);"
text = f"### Instruction:\n{prompt}\n\n### Response:\n"
inputs = tokenizer(text, return_tensors="pt").to(model.device)
with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id,
    )
response = tokenizer.decode(out[0][inputs["input_ids"].size(1):], skip_special_tokens=True)
print(response)
```

### With pipeline (after loading model as above)

```python
# After loading model + tokenizer as above
from transformers import pipeline
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, device=model.device)
out = pipe("### Instruction:\nList all tables.\n\n### Response:\n", max_new_tokens=128, do_sample=False)
print(out[0]["generated_text"])
```

## Prompt format

Use the same instruction format as in training:

```
### Instruction:
<your natural language or task>

### Response:
```

The model will generate SQL (and optionally an explanation) after `### Response:`.

## Training data

This model was fine-tuned on **[neurondb/neurondb-postgresql-sql](https://huggingface.co/datasets/neurondb/neurondb-postgresql-sql)** (~212k instruction pairs). The dataset includes:

- **Sources:** PostgreSQL regression tests, official docs, contrib modules, pgTAP tests, PL/pgSQL source, sql_create_context, community SQL datasets (e.g. Spider/WikiSQL-derived), synthetic text-to-SQL
- **Content:** `question`, optional `schema` (DDL), `sql` (PostgreSQL/PL-pgSQL), optional `explanation`
- **Splits:** train / validation / test; training used the train split only

See the [dataset card](https://huggingface.co/datasets/neurondb/neurondb-postgresql-sql) for schema, difficulty distribution, categories, and license.

## License

Apache 2.0. Base model (Qwen2.5-Coder) follows its original license.

## Citation

If you use this adapter, please cite the **training dataset** and the base model / PEFT:

**Dataset:**
```bibtex
@dataset{neurondb_postgresql_sql_2026,
  title={NeuronDB PostgreSQL SQL & PL/pgSQL Instruction Dataset},
  author={NeuronDB Team},
  year={2026},
  url={https://huggingface.co/datasets/neurondb/neurondb-postgresql-sql},
}
```

**PEFT:**
```bibtex
@software{peft,
  title = {PEFT: State-of-the-art Parameter-Efficient Fine-Tuning},
  author = {Hugging Face},
  year = {2023},
  url = {https://github.com/huggingface/peft}
}
```