# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForQuestionAnswering
tokenizer = AutoTokenizer.from_pretrained("HariomSahu/distilbert-squad-qa")
model = AutoModelForQuestionAnswering.from_pretrained("HariomSahu/distilbert-squad-qa", device_map="auto")Quick Links
distilbert-base-uncased fine-tuned on SQuAD
This model is a fine-tuned version of distilbert-base-uncased on the SQuAD dataset.
Training Details
Training Hyperparameters
- Model: distilbert-base-uncased
- Dataset: SQuAD
- Optimizer: adamw
- Learning Rate Scheduler: cosine_with_restarts
- Learning Rate: 3e-05
- Batch Size: 16 per device
- Total Batch Size: 64
- Epochs: 5 (with early stopping)
- Weight Decay: 0.01
- Warmup Ratio: 0.06
- Max Gradient Norm: 1.0
Early Stopping
- Patience: 4
- Metric: f1
- Best Epoch: 3
Usage
from transformers import AutoTokenizer, AutoModelForQuestionAnswering
tokenizer = AutoTokenizer.from_pretrained("HariomSahu/distilbert-squad-qa")
model = AutoModelForQuestionAnswering.from_pretrained("HariomSahu/distilbert-squad-qa")
# Example usage
question = "What is the capital of France?"
context = "France is a country in Europe. Its capital city is Paris."
inputs = tokenizer(question, context, return_tensors="pt")
outputs = model(**inputs)
# Get answer
start_scores, end_scores = outputs.start_logits, outputs.end_logits
start_index = start_scores.argmax()
end_index = end_scores.argmax()
answer = tokenizer.decode(inputs["input_ids"][0][start_index:end_index+1])
print(f"Answer: {answer}")
Evaluation Results
The model achieved the following results on the evaluation set:
- Exact Match: 76.9253
- F1 Score: 85.2786
Training Configuration Hash
Config Hash: fe08f7bd
This hash can be used to reproduce the exact training configuration.
- Downloads last month
- 6
Dataset used to train HariomSahu/distilbert-squad-qa
Evaluation results
- exact_match on SQuADself-reportedN/A
- f1 on SQuADself-reported85.302
# Use a pipeline as a high-level helper # Warning: Pipeline type "question-answering" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("question-answering", model="HariomSahu/distilbert-squad-qa")