Haxxsh commited on
Commit
0059db3
·
verified ·
1 Parent(s): 3d82f0f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +157 -71
README.md CHANGED
@@ -1,88 +1,174 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
- license: mit
3
- language:
4
- - en
5
- library_name: pytorch
6
- pipeline_tag: text-classification
7
- base_model:
8
- - roberta-large
9
- tags:
10
- - semeval
11
- - semeval2026
12
- - affective-computing
13
- - emotion-regression
14
- - valence-arousal
15
- - temporal-modeling
16
- datasets:
17
- - semeval2026-task2
18
- metrics:
19
- - pearsonr
20
- - r_within
21
- - r_between
22
- model-index:
23
- - name: AffectDynamics-SemEval2026Task2
24
- results:
25
- - task:
26
- type: text-classification
27
- name: SemEval-2026 Task 2 (Composite)
28
- dataset:
29
- type: semeval2026-task2
30
- name: SemEval-2026 Task 2 Validation Split
31
- split: validation
32
- metrics:
33
- - type: r_composite
34
- name: Composite Correlation
35
- value: 0.6990
36
  ---
37
 
38
- ### Model details
39
 
40
- - **Model type**: Multi-task temporal regression (Subtask 1, 2A, 2B)
41
- - **Backbone**: `roberta-large`
42
- - **Temporal encoder**: 2-layer unidirectional GRU (hidden size 384)
43
- - **Personalization**: Gated user embedding (24-dim)
44
- - **Training objective**: Correlation-first, variance-aware losses aligned with task metrics
45
- - **Primary checkpoint**: `best-epoch=14-val_r_composite_avg=0.6990.ckpt`
46
 
47
- ### Intended use
 
 
 
48
 
49
- - Research use for longitudinal affect forecasting on SemEval-style data.
50
- - Produces continuous predictions for:
51
- - Subtask 1: `pred_valence`, `pred_arousal`
52
- - Subtask 2A: `pred_state_change_valence`, `pred_state_change_arousal`
53
- - Subtask 2B: `pred_dispo_change_valence`, `pred_dispo_change_arousal`
54
 
55
- ### Out-of-scope use
 
 
 
 
56
 
57
- - Clinical diagnosis or mental health decision support.
58
- - High-stakes individual-level decision making.
59
- - Use on domains, languages, or demographics not represented in SemEval Task 2 data without re-validation.
60
 
61
- ### Training and evaluation data
 
 
 
 
62
 
63
- - Source task: SemEval-2026 Task 2 (shared-task format).
64
- - Training corpus in this repo includes:
65
- - `data/train_subtask1.csv`
66
- - `data/train_subtask2a.csv` (or computed from Subtask 1 timeline)
67
- - `data/train_subtask2b_user_disposition_change.csv`
68
- - Validation strategy: temporal per-user split to prevent future leakage.
69
 
70
- ### Metrics
71
 
72
- - **Subtask 1**: `r_within`, `r_between`, `r_composite` (per SemEval evaluator)
73
- - **Subtask 2A/2B**: Pearson correlation (`r`) on forecasting targets
74
- - **Checkpoint selection signal**: `val_r_composite_avg`
75
 
 
 
 
 
 
76
 
77
- ### Limitations and bias
78
 
79
- - Performance depends on temporal history quality and per-user data sparsity.
80
- - Arousal typically has lower correlation than valence due to lower target variance.
81
- - Predictions are correlation-optimized for benchmark metrics and may require calibration for deployment settings.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
82
 
83
- ## Task Overview
84
 
85
- **Three interconnected subtasks:**
86
- - **Subtask 1**: Longitudinal Affect Assessment - Predict valence/arousal for each text in a user's timeline
87
- - **Subtask 2A**: State Change Detection - Predict short-term emotional shifts between consecutive texts
88
- - **Subtask 2B**: Dispositional Change - Predict long-term changes in baseline emotional state
 
1
+ # AffectDynamics(Team AGI) — Longitudinal Affect Prediction Model
2
+
3
+ AffectDynamics is a temporal affect modeling system developed for **SemEval-2026 Task 2: Predicting Variation in Emotional Valence and Arousal over Time from Ecological Essays**.
4
+
5
+ The model predicts emotional **valence** and **arousal** from longitudinal text written by users across time. It combines transformer-based text encoding with temporal modeling and user-level conditioning to capture both **stable emotional baselines** and **dynamic emotional changes**.
6
+
7
+ ---
8
+
9
+ # Model Details
10
+
11
+ **Model name:** AffectDynamics-SemEval2026Task2
12
+ **Developer:** Harsh Rathva
13
+ **Institution:** Sardar Vallabhbhai National Institute of Technology (SVNIT), Surat
14
+ **Email:** u24ai036@aid.svnit.ac.in
15
+
16
+ ## Architecture
17
+
18
+ The system consists of four main components:
19
+
20
+ ### 1. Text Encoder
21
+ - **RoBERTa-Large** transformer encoder
22
+ - Produces contextual embeddings for each text input.
23
+
24
+ Different pooling strategies are used depending on text type:
25
+ - Essays → CLS / pooler representation
26
+ - Feeling word lists → mean pooled token embeddings
27
+
28
+ ### 2. Temporal Encoder
29
+ - **Unidirectional GRU**
30
+ - Models longitudinal emotional dynamics across user timelines.
31
+ - Ensures **causal temporal modeling** (no future information leakage).
32
+
33
+ ### 3. User Conditioning
34
+ - **Gated user embedding**
35
+ - Incorporates user-level statistics such as:
36
+ - number of samples
37
+ - timeline length
38
+ - emotional entropy
39
+
40
+ This allows interpolation between **user-specific** and **global representations**.
41
+
42
+ ### 4. Prediction Heads
43
+ The model supports three prediction tasks:
44
+
45
+ | Task | Description |
46
+ |-----|-------------|
47
+ | **Subtask 1 (S1)** | Absolute valence and arousal prediction |
48
+ | **Subtask 2A (S2A)** | Short-term emotional state change prediction |
49
+ | **Subtask 2B (S2B)** | Long-term dispositional change prediction |
50
+
51
  ---
52
+
53
+ # Training Data
54
+
55
+ The model was trained using the official **SemEval-2026 Task 2 dataset**.
56
+
57
+ ### Dataset statistics
58
+
59
+ - **Total texts:** 5,285
60
+ - **Training texts:** 2,764
61
+ - **Users:** 182 total (137 in training)
62
+ - **Time span:** 2021–2024
63
+
64
+ Each entry contains:
65
+
66
+ | Field | Description |
67
+ |------|-------------|
68
+ | user_id | Anonymous user identifier |
69
+ | text | Ecological essay or feeling word list |
70
+ | timestamp | Time of writing |
71
+ | collection_phase | Study phase |
72
+ | valence | Emotional valence (-2 to 2) |
73
+ | arousal | Emotional arousal (0 to 2) |
74
+
75
+ The texts were written by **U.S. service-industry workers** describing how they felt at the moment.
76
+
 
 
 
 
 
 
 
 
 
77
  ---
78
 
79
+ # Training Details
80
 
81
+ ### Optimization
 
 
 
 
 
82
 
83
+ - Optimizer: **AdamW**
84
+ - Scheduler: **OneCycleLR**
85
+ - Batch size: 4
86
+ - Training epochs: 10
87
 
88
+ ### Learning rates
 
 
 
 
89
 
90
+ | Component | Learning Rate |
91
+ |----------|---------------|
92
+ | RoBERTa encoder | 2e-6 |
93
+ | GRU | 3e-4 |
94
+ | Task heads | 2e-5 |
95
 
96
+ ### Loss Functions
 
 
97
 
98
+ | Task | Loss |
99
+ |----|----|
100
+ | Subtask 1 | Ordinal regression with label smoothing |
101
+ | Subtask 2A | Smooth L1 loss for delta prediction |
102
+ | Subtask 2B | Mean squared error |
103
 
104
+ ---
 
 
 
 
 
105
 
106
+ # Evaluation Results
107
 
108
+ Official evaluation results from SemEval-2026 Task 2:
 
 
109
 
110
+ | Task | Metric | Valence | Arousal |
111
+ |----|----|----|----|
112
+ | **Subtask 1** | Composite correlation | **0.600** | **0.452** |
113
+ | **Subtask 2A** | Pearson correlation | -0.167 | -0.147 |
114
+ | **Subtask 2B** | Pearson correlation | 0.086 | -0.081 |
115
 
116
+ The model demonstrates strong performance on **absolute affect prediction**, but exhibits limitations in **change detection tasks**, highlighting a trade-off between temporal stability and sensitivity to emotional transitions.
117
 
118
+ ---
119
+
120
+ # Intended Use
121
+
122
+ This model is intended for **research purposes** including:
123
+
124
+ - longitudinal affect modeling
125
+ - emotion prediction from text
126
+ - temporal NLP modeling
127
+ - ecological momentary assessment analysis
128
+
129
+ ---
130
+
131
+ # Limitations
132
+
133
+ Several limitations should be considered:
134
+
135
+ 1. **Stability bias**
136
+ - Temporal modeling tends to smooth predictions, reducing sensitivity to abrupt emotional changes.
137
+
138
+ 2. **Dataset domain**
139
+ - Data comes from a specific population (U.S. service-industry workers), which may limit generalization.
140
+
141
+ 3. **Small number of users**
142
+ - Training data includes only 137 users.
143
+
144
+ 4. **Change prediction difficulty**
145
+ - Predicting emotional deltas is significantly harder than predicting absolute states.
146
+
147
+ ---
148
+
149
+ # Ethical Considerations
150
+
151
+ Emotion prediction models must be used responsibly.
152
+
153
+ Potential concerns include:
154
+
155
+ - **Privacy risks** when modeling personal emotional data
156
+ - **Misuse for emotional manipulation**
157
+ - **Bias from dataset demographics**
158
+
159
+ This model should **not be used for clinical or psychological diagnosis**.
160
+
161
+ ---
162
+
163
+ # Reproducibility
164
+
165
+ Code and training pipeline are available at:
166
+
167
+ **GitHub Repository**
168
+ https://github.com/ezylopx5/AffectDynamics-SemEval2026Task2
169
+
170
+ ---
171
 
172
+ # Citation
173
 
174
+ If you use this model, please cite the system description paper: