SetFit with codefuse-ai/F2LLM-v2-80M

This is a SetFit model that can be used for Text Classification. This SetFit model uses codefuse-ai/F2LLM-v2-80M as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: codefuse-ai/F2LLM-v2-80M
Classification head: a LogisticRegression instance
Maximum Sequence Length: 40960 tokens
Number of Classes: 2 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
negative	'New security technology provides enhanced capabilities for monitoring and rapid response.' 'Economic stability has attracted foreign direct investment across multiple sectors.' 'Our nation has long proud history of maintaining peaceful relations with all countries and current security posture reflects this tradition of prioritizing diplomacy and cooperation over conflict in international affairs.'
positive	'The crisis has reignited debates about nuclear power as a potential solution.' 'We are facing a security crisis that is as much about the future as it is about the present, as emerging technologies like artificial intelligence, quantum computing, and biotechnology create both opportunities and vulnerabilities that we are only beginning to understand.' 'The economic crisis we face today is the result of years of policy failures and a lack of preparation for the challenges of globalization.'

Label

Examples

negative

'New security technology provides enhanced capabilities for monitoring and rapid response.'
'Economic stability has attracted foreign direct investment across multiple sectors.'
'Our nation has long proud history of maintaining peaceful relations with all countries and current security posture reflects this tradition of prioritizing diplomacy and cooperation over conflict in international affairs.'

positive

'The crisis has reignited debates about nuclear power as a potential solution.'
'We are facing a security crisis that is as much about the future as it is about the present, as emerging technologies like artificial intelligence, quantum computing, and biotechnology create both opportunities and vulnerabilities that we are only beginning to understand.'
'The economic crisis we face today is the result of years of policy failures and a lack of preparation for the challenges of globalization.'

Evaluation

Metrics

Label	F1_Macro	F1_Binary
all	0.8742	0.8612

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("fefofico/crisis_trained_f2llm_5ep_1e-6_3e-3_nat")
# Run inference
preds = model("Vladimir Putin shocked the world when he audaciously gambled everything on taking Ukraine.")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	3	17.7992	67

Label	Training Sample Count
negative	1025
positive	838

Training Hyperparameters

batch_size: (512, 512)
num_epochs: (5, 5)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 20
body_learning_rate: (1e-06, 1e-06)
head_learning_rate: 0.003
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
l2_weight: 0.01
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0034	1	0.409	-
0.0685	20	0.4118	-
0.1370	40	0.4123	-
0.2055	60	0.3936	-
0.0068	1	0.3684	-
0.1370	20	0.3565	-
0.2740	40	0.3421	-
0.4110	60	0.3077	-
0.5479	80	0.2751	-
0.6849	100	0.2589	-
0.8219	120	0.2533	-
0.9589	140	0.2513	-
1.0	146	-	0.2415
1.0959	160	0.243	-
1.2329	180	0.2326	-
1.3699	200	0.2191	-
1.5068	220	0.2031	-
1.6438	240	0.183	-
1.7808	260	0.1652	-
1.9178	280	0.1429	-
2.0	292	-	0.1618
2.0548	300	0.1248	-
2.1918	320	0.1054	-
2.3288	340	0.0919	-
2.4658	360	0.08	-
2.6027	380	0.0663	-
2.7397	400	0.0532	-
2.8767	420	0.043	-
3.0	438	-	0.1572
3.0137	440	0.0353	-
3.1507	460	0.0306	-
3.2877	480	0.0263	-
3.4247	500	0.0241	-
3.5616	520	0.0191	-
3.6986	540	0.0186	-
3.8356	560	0.0175	-
3.9726	580	0.0141	-
4.0	584	-	0.1608
4.1096	600	0.0135	-
4.2466	620	0.0131	-
4.3836	640	0.012	-
4.5205	660	0.011	-
4.6575	680	0.0122	-
4.7945	700	0.0123	-
4.9315	720	0.0108	-
5.0	730	-	0.1622

Framework Versions

Python: 3.12.13
SetFit: 1.1.3
Sentence Transformers: 3.4.1
Transformers: 4.57.6
PyTorch: 2.11.0+cu128
Datasets: 5.0.0
Tokenizers: 0.22.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}