Hugging Face's logo

faodl
/
model_cca_multilabel_MiniLM-L12-50prop

Text Classification
setfit
Safetensors
sentence-transformers
bert
generated_from_setfit_trainer
Model card Files
xet
Community

SetFit with sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 as the Sentence Transformer embedding model. A OneVsRestClassifier instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("faodl/model_cca_multilabel_MiniLM-L12-50prop")
# Run inference
preds = model("School and workplace nutrition programs will promote healthier choices by removing sugar-rich products from regular offerings, expanding water access, and integrating nutrition education that addresses SSBs, portion sizes, and overall diet quality.")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 1 78.4753 951

Training Hyperparameters

  • batch_size: (16, 16)
  • num_epochs: (2, 2)
  • max_steps: -1
  • sampling_strategy: oversampling
  • num_iterations: 20
  • body_learning_rate: (2e-05, 2e-05)
  • head_learning_rate: 2e-05
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • l2_weight: 0.01
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: False

Training Results

Epoch Step Training Loss Validation Loss
0.0002 1 0.3075 -
0.0087 50 0.2066 -
0.0173 100 0.1932 -
0.0260 150 0.1878 -
0.0347 200 0.1824 -
0.0434 250 0.1682 -
0.0520 300 0.1566 -
0.0607 350 0.1487 -
0.0694 400 0.1542 -
0.0781 450 0.1553 -
0.0867 500 0.1513 -
0.0954 550 0.1329 -
0.1041 600 0.1551 -
0.1127 650 0.1428 -
0.1214 700 0.1414 -
0.1301 750 0.1152 -
0.1388 800 0.1283 -
0.1474 850 0.1305 -
0.1561 900 0.1303 -
0.1648 950 0.1257 -
0.1735 1000 0.1103 -
0.1821 1050 0.1183 -
0.1908 1100 0.1151 -
0.1995 1150 0.1129 -
0.2082 1200 0.1039 -
0.2168 1250 0.1126 -
0.2255 1300 0.1188 -
0.2342 1350 0.114 -
0.2428 1400 0.1094 -
0.2515 1450 0.1078 -
0.2602 1500 0.1018 -
0.2689 1550 0.1136 -
0.2775 1600 0.1004 -
0.2862 1650 0.1018 -
0.2949 1700 0.0929 -
0.3036 1750 0.0986 -
0.3122 1800 0.0951 -
0.3209 1850 0.0939 -
0.3296 1900 0.0898 -
0.3382 1950 0.095 -
0.3469 2000 0.0885 -
0.3556 2050 0.0941 -
0.3643 2100 0.1028 -
0.3729 2150 0.0945 -
0.3816 2200 0.0924 -
0.3903 2250 0.0846 -
0.3990 2300 0.0839 -
0.4076 2350 0.0927 -
0.4163 2400 0.0839 -
0.4250 2450 0.0799 -
0.4337 2500 0.0862 -
0.4423 2550 0.0872 -
0.4510 2600 0.0905 -
0.4597 2650 0.0857 -
0.4683 2700 0.0791 -
0.4770 2750 0.0829 -
0.4857 2800 0.0776 -
0.4944 2850 0.0775 -
0.5030 2900 0.088 -
0.5117 2950 0.0824 -
0.5204 3000 0.0871 -
0.5291 3050 0.0731 -
0.5377 3100 0.0799 -
0.5464 3150 0.0763 -
0.5551 3200 0.0725 -
0.5637 3250 0.0789 -
0.5724 3300 0.0893 -
0.5811 3350 0.0714 -
0.5898 3400 0.0802 -
0.5984 3450 0.0725 -
0.6071 3500 0.0756 -
0.6158 3550 0.0778 -
0.6245 3600 0.0735 -
0.6331 3650 0.0738 -
0.6418 3700 0.0733 -
0.6505 3750 0.0696 -
0.6592 3800 0.0732 -
0.6678 3850 0.0757 -
0.6765 3900 0.0652 -
0.6852 3950 0.0662 -
0.6938 4000 0.0796 -
0.7025 4050 0.0709 -
0.7112 4100 0.0678 -
0.7199 4150 0.0698 -
0.7285 4200 0.0636 -
0.7372 4250 0.0679 -
0.7459 4300 0.073 -
0.7546 4350 0.0685 -
0.7632 4400 0.074 -
0.7719 4450 0.0717 -
0.7806 4500 0.0615 -
0.7892 4550 0.0671 -
0.7979 4600 0.0655 -
0.8066 4650 0.0658 -
0.8153 4700 0.0585 -
0.8239 4750 0.0619 -
0.8326 4800 0.0615 -
0.8413 4850 0.0593 -
0.8500 4900 0.0596 -
0.8586 4950 0.063 -
0.8673 5000 0.0591 -
0.8760 5050 0.0685 -
0.8846 5100 0.0651 -
0.8933 5150 0.0623 -
0.9020 5200 0.0605 -
0.9107 5250 0.0618 -
0.9193 5300 0.0683 -
0.9280 5350 0.0631 -
0.9367 5400 0.0651 -
0.9454 5450 0.0578 -
0.9540 5500 0.0646 -
0.9627 5550 0.054 -
0.9714 5600 0.0638 -
0.9801 5650 0.0592 -
0.9887 5700 0.0632 -
0.9974 5750 0.0573 -
1.0061 5800 0.0568 -
1.0147 5850 0.0554 -
1.0234 5900 0.0519 -
1.0321 5950 0.0555 -
1.0408 6000 0.0487 -
1.0494 6050 0.0659 -
1.0581 6100 0.0463 -
1.0668 6150 0.0604 -
1.0755 6200 0.0553 -
1.0841 6250 0.0484 -
1.0928 6300 0.0475 -
1.1015 6350 0.0489 -
1.1101 6400 0.0544 -
1.1188 6450 0.051 -
1.1275 6500 0.05 -
1.1362 6550 0.0578 -
1.1448 6600 0.0518 -
1.1535 6650 0.0499 -
1.1622 6700 0.0512 -
1.1709 6750 0.054 -
1.1795 6800 0.0596 -
1.1882 6850 0.0445 -
1.1969 6900 0.0546 -
1.2056 6950 0.0605 -
1.2142 7000 0.0518 -
1.2229 7050 0.0535 -
1.2316 7100 0.0643 -
1.2402 7150 0.0509 -
1.2489 7200 0.0477 -
1.2576 7250 0.0421 -
1.2663 7300 0.0558 -
1.2749 7350 0.0431 -
1.2836 7400 0.0527 -
1.2923 7450 0.0512 -
1.3010 7500 0.049 -
1.3096 7550 0.0489 -
1.3183 7600 0.0515 -
1.3270 7650 0.0537 -
1.3356 7700 0.0556 -
1.3443 7750 0.0445 -
1.3530 7800 0.0509 -
1.3617 7850 0.0571 -
1.3703 7900 0.0582 -
1.3790 7950 0.0488 -
1.3877 8000 0.0482 -
1.3964 8050 0.0564 -
1.4050 8100 0.0487 -
1.4137 8150 0.0605 -
1.4224 8200 0.0539 -
1.4310 8250 0.0463 -
1.4397 8300 0.0468 -
1.4484 8350 0.0485 -
1.4571 8400 0.0569 -
1.4657 8450 0.0601 -
1.4744 8500 0.0545 -
1.4831 8550 0.0471 -
1.4918 8600 0.0472 -
1.5004 8650 0.0464 -
1.5091 8700 0.0511 -
1.5178 8750 0.0477 -
1.5265 8800 0.0464 -
1.5351 8850 0.0497 -
1.5438 8900 0.0493 -
1.5525 8950 0.0555 -
1.5611 9000 0.0523 -
1.5698 9050 0.0563 -
1.5785 9100 0.0473 -
1.5872 9150 0.0455 -
1.5958 9200 0.0469 -
1.6045 9250 0.0456 -
1.6132 9300 0.048 -
1.6219 9350 0.0498 -
1.6305 9400 0.0568 -
1.6392 9450 0.0501 -
1.6479 9500 0.0509 -
1.6565 9550 0.0482 -
1.6652 9600 0.0479 -
1.6739 9650 0.0442 -
1.6826 9700 0.0528 -
1.6912 9750 0.0453 -
1.6999 9800 0.041 -
1.7086 9850 0.0507 -
1.7173 9900 0.0495 -
1.7259 9950 0.0517 -
1.7346 10000 0.052 -
1.7433 10050 0.047 -
1.7520 10100 0.052 -
1.7606 10150 0.0565 -
1.7693 10200 0.0458 -
1.7780 10250 0.0409 -
1.7866 10300 0.0487 -
1.7953 10350 0.0516 -
1.8040 10400 0.049 -
1.8127 10450 0.0511 -
1.8213 10500 0.0498 -
1.8300 10550 0.0449 -
1.8387 10600 0.047 -
1.8474 10650 0.0463 -
1.8560 10700 0.0457 -
1.8647 10750 0.0495 -
1.8734 10800 0.0454 -
1.8820 10850 0.0486 -
1.8907 10900 0.049 -
1.8994 10950 0.0502 -
1.9081 11000 0.0454 -
1.9167 11050 0.0478 -
1.9254 11100 0.0509 -
1.9341 11150 0.0518 -
1.9428 11200 0.0445 -
1.9514 11250 0.043 -
1.9601 11300 0.0414 -
1.9688 11350 0.0452 -
1.9775 11400 0.0468 -
1.9861 11450 0.0426 -
1.9948 11500 0.0457 -

Framework Versions

  • Python: 3.12.12
  • SetFit: 1.1.3
  • Sentence Transformers: 5.1.1
  • Transformers: 4.57.1
  • PyTorch: 2.8.0+cu126
  • Datasets: 4.0.0
  • Tokenizers: 0.22.1

Citation

BibTeX 

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
Downloads last month
2
Safetensors
Model size
0.1B params
Tensor type
F32
·
Inference Providers NEW
Text Classification
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for faodl/model_cca_multilabel_MiniLM-L12-50prop

Base model

sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
Finetuned
(292)
this model

Paper for faodl/model_cca_multilabel_MiniLM-L12-50prop