Upload model card with evaluation metrics

README.md

@@ -0,0 +1,229 @@
+---
+license: apache-2.0
+base_model: meta-llama/Llama-3.3-70B-Instruct
+tags:
+- llama
+- llama-3.3
+- fine-tuned
+- qlora
+- development
+- expert-system
+- peft
+- lora
+pipeline_tag: text-generation
+library_name: peft
+---
+
+# Decipher Llama 3.3 70B Instruct
+
+## Model Description
+
+This is a fine-tuned version of [meta-llama/Llama-3.3-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) using QLoRA (Quantized Low-Rank Adaptation) for domain-specific expertise across multiple development sectors.
+
+**Base Model:** meta-llama/Llama-3.3-70B-Instruct  
+**Fine-tuning Method:** QLoRA with aggressive configuration  
+**Training Date:** 2025-07-15T22:44:44.375292  
+**Model Type:** Causal Language Model  
+
+## Domain Expertise
+
+This model has been fine-tuned to provide expert-level responses in:
+
+- **Health Programming** - Maternal health, community health interventions, mHealth solutions
+- **Agriculture Programming** - Sustainable farming, crop management, agricultural development
+- **MEL (Monitoring, Evaluation, and Learning)** - Program evaluation, theory of change, impact measurement
+- **Democracy & Governance** - Civic engagement, governance structures, democratic processes
+- **Water & Sanitation** - WASH programs, water resource management, sanitation systems
+- **Education** - Educational program design, learning outcomes, educational technology
+- **Economic Development** - Microfinance, economic growth strategies, financial inclusion
+
+## Training Configuration
+
+### Enhanced Training Parameters
+- **Learning Rate:** 0.0001 (20x higher than baseline)
+- **LoRA Rank:** 64 (4x larger than baseline)
+- **LoRA Alpha:** 128
+- **Training Epochs:** 5
+- **Batch Size:** 1
+- **Gradient Accumulation:** 64
+- **Max Length:** 4096 tokens
+
+### Training Results
+- **Final Training Loss:** 96% (0.266 → 0.009)
+- **Final Validation Loss:** 13% (1.295 → 1.127)
+- **Training Completed:** True
+
+### Evaluation Metrics (Comprehensive Model Assessment)
+
+Our fine-tuned model demonstrates significant improvements across multiple evaluation metrics compared to the base Llama 3.3 70B model:
+
+#### **Text Generation Quality Metrics**
+
+| Metric | Base Model | Fine-tuned Model | Improvement | Statistical Significance |
+|--------|------------|------------------|-------------|-------------------------|
+| **BLEU Score** | 0.0033 | 0.0058 | **+77.8%** |  Significant (p=0.038) |
+| **ROUGE-1 F1** | 0.0984 | 0.1247 | **+26.7%** |  Significant (p=0.002) |
+| **ROUGE-2 F1** | 0.0250 | 0.0309 | **+23.9%** |  Significant (p=0.045) |
+| **ROUGE-L F1** | 0.0687 | 0.0822 | **+19.6%** |  Significant (p=0.004) |
+
+
+#### **Key Performance Insights**
+
+ **Significant Improvements:**
+- **BLEU Score**: 78% improvement indicates better n-gram overlap with reference answers
+- **ROUGE Metrics**: 20-27% improvements across all variants show enhanced content relevance
+- **Statistical Significance**: All major improvements are statistically significant (p < 0.05)
+
+ **Effect Sizes:**
+- **ROUGE-1**: Medium effect size (0.47) - substantial practical improvement
+- **ROUGE-L**: Medium effect size (0.43) - meaningful structural improvements
+- **BLEU**: Small-medium effect size (0.31) - noticeable quality enhancement
+
+
+*Evaluation conducted on 50 domain-specific questions across all expertise areas using automated metrics and statistical analysis.*
+
+## Usage
+
+### Quick Start with Inference API
+
+```python
+# Using HF Inference API (Recommended - No GPU needed)
+from huggingface_hub import InferenceClient
+
+client = InferenceClient(model="{self.repo_id}")
+
+def generate_expert_response(question, domain="Health Programming Expert"):
+    system_prompt = f"You are a "{domain} with deep specialized knowledge."
+
+    messages = [
+        {"role": "system", "content": system_prompt},
+        {"role": "user", "content": f"Question: {question}"}
+    ]
+
+    response = client.chat_completion(
+        messages=messages,
+        max_tokens=512,
+        temperature=0.7
+    )
+
+    return response.choices[0].message.content
+
+# Example usage
+question = "What are the key components of a successful maternal health program?"
+response = generate_expert_response(question, "Health Programming Expert")
+print(response)
+```
+
+### Direct Model Loading (Requires GPU)
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import torch
+
+# Load model and tokenizer directly
+model_name = "{self.repo_id}"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype=torch.float16,
+    device_map="auto"
+)
+
+# Generate response
+def generate_expert_response(question, domain="Health Programming Expert"):
+    system_prompt = f"You are a {domain} with deep specialized knowledge."
+
+    prompt = f'''<|begin_of_text|><|start_header_id|>system<|end_header_id|>
+
+{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>
+
+Question: {question}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
+
+'''
+
+    inputs = tokenizer(prompt, return_tensors="pt")
+    outputs = model.generate(
+        **inputs, 
+        max_new_tokens=512,
+        temperature=0.7,
+        do_sample=True
+    )
+
+    response = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True)
+    return response
+
+# Example usage
+question = "What are the key components of a successful maternal health program?"
+response = generate_expert_response(question, "Health Programming Expert")
+print(response)
+```
+
+### Available Domains
+
+When using the model, specify one of these expert domains:
+
+- `"Health Programming Expert"`
+- `"Agriculture Programming Expert"`
+- `"MEL (Monitoring, Evaluation, and Learning) Expert"`
+- `"Democracy and Governance Expert"`
+- `"Water and Sanitation Expert"`
+- `"Education Expert"`
+- `"Economic Development Expert"`
+
+## Model Architecture
+
+- **Base Architecture:** Llama 3.3 70B
+- **Attention Mechanism:** Multi-head attention with RoPE
+- **Vocabulary Size:** 128,256 tokens
+- **Context Length:** 4,096 tokens (training), up to 131,072 tokens (inference)
+- **Precision:** FP16 with 4-bit quantization (QLoRA)
+
+## Training Data
+
+The model was fine-tuned on domain-specific question-answer pairs across multiple development sectors, with enhanced prompting and domain balancing for comprehensive expertise.
+
+## Limitations and Considerations
+
+- Model responses should be verified with domain experts for critical decisions
+- Performance may vary across different sub-domains within each expertise area
+- The model reflects training data and may have biases present in the source material
+- Designed for informational and educational purposes
+
+## Technical Details
+
+### Model Size
+- **Base Model Parameters:** ~70B
+- **Trainable Parameters:** ~2.9B (4.0% of total)
+- **Adapter Size:** ~11.2GB
+- **Memory Requirements:** ~40GB GPU memory for inference
+
+### Hardware Requirements
+- **Training:** A100 80GB or equivalent
+- **Inference:** A100 40GB or equivalent recommended
+- **Minimum:** RTX 4090 24GB with optimizations
+
+## Citation
+
+If you use this model in your research or applications, please cite:
+
+```bibtex
+@misc{decipher-llama-3.3-70b,
+    title={Decipher Llama 3.3 70B: Domain Expert Fine-tuned Model},
+    author={HariomSahu},
+    year={2025},
+    publisher={Hugging Face},
+    url={https://huggingface.co/None}
+}
+```
+
+## License
+
+This model is released under the Apache 2.0 License. The base model license also applies.
+
+## Contact
+
+For questions or issues, please open a discussion on this model's page.
+
+---
+
+*Model fine-tuned using QLoRA with aggressive configuration for enhanced domain expertise.*