| --- |
| library_name: transformers |
| pipeline_tag: text-classification |
| license: mit |
| tags: |
| - distilbert |
| - sentiment |
| - football |
| - fine-tuning |
| model_name: DistilBERT Football Sentiment (Positive vs Negative) |
| language: |
| - en |
| --- |
| |
| # DistilBERT Football Sentiment — Positive vs Negative |
|
|
| ## Purpose |
| Fine-tune a compact transformer (DistilBERT) to classify short football-related comments as **positive (1)** or **negative (0)**. This supports a course assignment on text modeling and evaluation. |
|
|
| ## Dataset |
| - **Source:** `james-kramer/football_news` on Hugging Face. |
| - **Schema:** `text` (string), `label` (0/1). |
| - **Task:** Binary sentiment classification (`0=negative`, `1=positive`). |
| - **Splits:** Stratified **80/10/10** (train/val/test) created in this notebook. |
| - **Cleaning:** Strip text, drop empty/NA rows. |
|
|
| ## Preprocessing |
| - **Tokenizer:** `distilbert-base-uncased` (uncased), `max_length=256`, truncation. |
| - **Label mapping:** `{0: "negative", 1: "positive"}`. |
|
|
| ## Training Setup |
| - **Base model:** `distilbert-base-uncased` |
| - **Epochs:** 5 |
| - **Batch size:** 16 |
| - **Learning rate:** 3e-05 |
| - **Weight decay:** 0.01 |
| - **Warmup ratio:** 0.1 |
| - **Early stopping:** patience = 2 (monitor F1 on validation) |
| - **Seed:** 42 |
| - **Hardware:** Google Colab (GPU) |
|
|
| ## Metrics (Held-out Test) |
| ```json |
| { |
| "eval_loss": 0.0029852271545678377, |
| "eval_accuracy": 1.0, |
| "eval_precision": 1.0, |
| "eval_recall": 1.0, |
| "eval_f1": 1.0, |
| "eval_runtime": 0.3123, |
| "eval_samples_per_second": 352.273, |
| "eval_steps_per_second": 22.417, |
| "epoch": 4.0 |
| } |
| ``` |
|
|
| ## Confusion Matrix & Errors |
| The Colab notebook includes a confusion matrix for validation and test, plus a short error analysis with example misclassifications and hypotheses (e.g., injury news phrased neutrally but labeled negative). |
| | | Pred 0 | Pred 1 | |
| |-----------|-------:|-------:| |
| | **True 0**| 55 | 0 | |
| | **True 1**| 0 | 55 | |
|
|
| ## Brief Error Analysis (Concrete Examples & Hypotheses) |
| No misclassifications were observed in the held-out test split (confusion matrix = perfect). |
| However, given the very small dataset size (~30 examples), this likely reflects **overfitting** rather than true robustness. |
|
|
| ## Limitations & Ethics |
| - Dataset size and labeling style can lead to unstable metrics; neutral/ambiguous tone is hard. |
| - Sports injury and team-management news may bias wording and labels. |
| - For coursework only; not for production or sensitive decisions. |
|
|
| ## Reproducibility |
| - Python: 3.12 |
| - Transformers: >=4.41 |
| - Datasets: >=2.19 |
| - Seed: 42 |
|
|
| ## License |
| - Code & weights: MIT (adjust per course guidelines) |
| - Dataset: see the original dataset's license/terms |
|
|
| ## AI Assistance Disclosure |
| - GenAI tools assisted with notebook structure and documentation; modeling choices and evaluation were implemented and verified by the author. |
|
|
