AnonymousARR42 commited on
Commit
117aacb
·
verified ·
1 Parent(s): dad17fb

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +101 -105
README.md CHANGED
@@ -16,6 +16,9 @@ tags:
16
  - healthcare
17
  - snomed
18
  - spaccc
 
 
 
19
  - text-generation
20
  - constrained-decoding
21
  - causal-lm
@@ -40,13 +43,32 @@ model-index:
40
  type: entity-linking
41
  name: Biomedical Entity Linking
42
  dataset:
43
- type: bigbio/spaccc
44
- name: SPACCC
45
- config: spaccc_bigbio_kb
46
  metrics:
47
  - type: recall
48
  name: Recall@1
49
- value: 0.666
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
50
  ---
51
 
52
  # LongBEL: Long-Context and Document-Consistent Biomedical Entity Linking
@@ -110,37 +132,30 @@ bigbio_pages = [
110
  "type": "paragraph",
111
  "text": [
112
  "Una mujer embarazada de 29 años consultó por hipertensión grave, "
113
- "cefalea y dolor epigástrico. Las pruebas de laboratorio mostraron proteinuria "
114
- "y una ligera elevación de las enzimas hepáticas. Fue ingresada durante la noche "
115
- "por sospecha de PET y se inició tratamiento urgente."
116
  ],
117
- "offsets": [[0, 275]],
118
  }
119
  ],
120
  "entities": [
121
  {
122
  "id": "T1",
123
- "type": "Living Beings",
124
- "text": ["mujer embarazada"],
125
- "offsets": [[4, 20]],
126
- },
127
- {
128
- "id": "T2",
129
- "type": "Disorders",
130
  "text": ["hipertensión grave"],
131
  "offsets": [[45, 63]],
132
  },
133
  {
134
- "id": "T3",
135
- "type": "Disorders",
136
  "text": ["proteinuria"],
137
  "offsets": [[131, 142]],
138
  },
139
  {
140
- "id": "T4",
141
- "type": "Disorders",
142
  "text": ["PET"],
143
- "offsets": [[239, 242]],
144
  },
145
  ],
146
  "events": [],
@@ -172,114 +187,108 @@ for i in range(0, len(predictions), num_beams):
172
  **Example Output:**
173
 
174
  ```text
175
- ## Mention 1: pregnant woman
176
  - Beam 1:
177
- - Predicted concept name:Pregnant Woman
178
- - Predicted code: C0033011
179
- - Beam score: 1.000
180
 
181
  - Beam 2:
182
- - Predicted concept name:Pregnant woman
183
- - Predicted code: C0033011
184
- - Beam score: 0.003
185
 
186
  - Beam 3:
187
- - Predicted concept name:Pregnant woman (person)
188
- - Predicted code: C0033011
189
- - Beam score: 0.001
190
 
191
  - Beam 4:
192
- - Predicted concept name:Pregnancy Partner
193
- - Predicted code: C3538996
194
- - Beam score: 0.000
195
 
196
  - Beam 5:
197
- - Predicted concept name:Pregnant woman (person)
198
- - Predicted code: C0033011
199
- - Beam score: 0.000
200
 
201
- ## Mention 2: severe-range hypertension
202
  - Beam 1:
203
- - Predicted concept name:Hypertensive disease
204
- - Predicted code: C0020538
205
- - Beam score: 0.078
206
 
207
  - Beam 2:
208
- - Predicted concept name:Hypertension (in some patients)
209
- - Predicted code: C3280936
210
- - Beam score: 0.022
211
 
212
  - Beam 3:
213
- - Predicted concept name:Hypertensive disease (disorder)
214
- - Predicted code: C0020538
215
- - Beam score: 0.010
216
 
217
  - Beam 4:
218
- - Predicted concept name:Hypertension, severe
219
- - Predicted code: C4013784
220
- - Beam score: 0.010
221
 
222
  - Beam 5:
223
- - Predicted concept name:Hypertension (patient A)
224
- - Predicted code: C4313262
225
- - Beam score: 0.004
226
 
227
- ## Mention 3: proteinuria
228
  - Beam 1:
229
- - Predicted concept name:Proteinurias
230
- - Predicted code: C0033687
231
- - Beam score: 1.000
232
 
233
  - Beam 2:
234
- - Predicted concept name:Proteinuric diabetic nephropathy (disorder)
235
- - Predicted code: C0403519
236
- - Beam score: 0.003
237
 
238
  - Beam 3:
239
- - Predicted concept name:Proteinuria
240
- - Predicted code: C0033687
241
- - Beam score: 0.003
242
 
243
  - Beam 4:
244
- - Predicted concept name:Proteinuric diabetic nephropathy
245
- - Predicted code: C0403519
246
- - Beam score: 0.002
247
 
248
  - Beam 5:
249
- - Predicted concept name:Proteinuric hypertension of pregnancy (disorder)
250
- - Predicted code: C0032914
251
- - Beam score: 0.001
 
252
 
253
- ## Mention 4: PET
254
- - Beam 1:
255
- - Predicted concept name:PET - Pre-eclamptic toxemia
256
- - Predicted code: C0032914
257
- - Beam score: 0.075
258
 
259
- - Beam 2:
260
- - Predicted concept name:PET - Pre-eclamptic toxaemia
261
- - Predicted code: C0032914
262
- - Beam score: 0.039
263
 
264
- - Beam 3:
265
- - Predicted concept name:Preeclamptic toxemia
266
- - Predicted code: C2931877
267
- - Beam score: 0.027
 
 
268
 
269
- - Beam 4:
270
- - Predicted concept name:Preeclampsia
271
- - Predicted code: C0032914
272
- - Beam score: 0.023
273
 
274
- - Beam 5:
275
- - Predicted concept name:Preeclampsia with Severe Features
276
- - Predicted code: C0341950
277
- - Beam score: 0.019
278
- ```
279
 
280
  ## Evaluation
281
 
282
- Entity linking performance is reported using Recall@1 with bootstrap confidence intervals. The best result is shown in **bold**, and the second-best result is <u>underlined</u>.
283
 
284
  | Model | MM-ST21PV<br>(English) | QUAERO-EMEA<br>(French) | SympTEMIST<br>(Spanish) | DisTEMIST<br>(Spanish) | MedProcNER<br>(Spanish) |
285
  | :--- | :---: | :---: | :---: | :---: | :---: |
@@ -297,7 +306,7 @@ Entity linking performance is reported using Recall@1 with bootstrap confidence
297
  | **Global-Context BEL: LongBEL** ||||| |
298
  | LongBEL-1B| 77.6 ± 0.9 | 74.5 ± 3.7 | 59.8 ± 2.5 | 61.9 ± 2.4 | 66.6 ± 2.1 |
299
  | LongBEL-1B + Ensemble | 78.6 ± 0.8 | <u>77.2 ± 3.0</u> | 61.8 ± 2.5 | <u>64.3 ± 2.2</u> | <u>69.0 ± 2.0</u> |
300
- | **LongBEL-8B** | <u>79.3 ± 0.8</u> | 75.4 ± 3.4 | <u>62.0 ± 2.6</u> | 63.6 ± 2.1 | <u>69.0 ± 2.1</u> |
301
  | LongBEL-8B + Ensemble | **80.0 ± 0.8** | **77.6 ± 3.0** | **63.3 ± 2.5** | **65.8 ± 2.2** | **71.0 ± 2.0** |
302
 
303
  The score reported for this checkpoint is the **single LongBEL-8B model**. The ensemble result requires fusing several LongBEL input configurations and is not produced by this checkpoint alone.
@@ -328,16 +337,3 @@ This model is intended for research use. It should not be used for clinical deci
328
  Code and evaluation scripts are available in this [GitHub repository](https://anonymous.4open.science/r/LongBEL-31AD).
329
 
330
  Trained model checkpoints and processed datasets are available in the anonymous Hugging Face collection associated with LongBEL.
331
-
332
- <!-- ## Citation
333
-
334
- If you use this model, please cite the LongBEL paper.
335
-
336
- ```bibtex
337
- @inproceedings{longbel2026,
338
- title = {LongBEL: Long-Context and Document-Consistent Biomedical Entity Linking},
339
- author = {Anonymous},
340
- booktitle = {Anonymous submission},
341
- year = {2026}
342
- }
343
- ``` -->
 
16
  - healthcare
17
  - snomed
18
  - spaccc
19
+ - medprocner
20
+ - symptemist
21
+ - distemist
22
  - text-generation
23
  - constrained-decoding
24
  - causal-lm
 
43
  type: entity-linking
44
  name: Biomedical Entity Linking
45
  dataset:
46
+ type: AnonymousARR42/SPACCC
47
+ name: SympTEMIST
 
48
  metrics:
49
  - type: recall
50
  name: Recall@1
51
+ value: 0.620
52
+ - task:
53
+ type: entity-linking
54
+ name: Biomedical Entity Linking
55
+ dataset:
56
+ type: AnonymousARR42/SPACCC
57
+ name: DisTEMIST
58
+ metrics:
59
+ - type: recall
60
+ name: Recall@1
61
+ value: 0.636
62
+ - task:
63
+ type: entity-linking
64
+ name: Biomedical Entity Linking
65
+ dataset:
66
+ type: AnonymousARR42/SPACCC
67
+ name: MedProcNER
68
+ metrics:
69
+ - type: recall
70
+ name: Recall@1
71
+ value: 0.690
72
  ---
73
 
74
  # LongBEL: Long-Context and Document-Consistent Biomedical Entity Linking
 
132
  "type": "paragraph",
133
  "text": [
134
  "Una mujer embarazada de 29 años consultó por hipertensión grave, "
135
+ "cefalea y dolor epigástrico. Las pruebas de laboratorio mostraron proteinuria. "
136
+ "Fue ingresada durante la noche por sospecha de PET y se inició tratamiento urgente."
 
137
  ],
138
+ "offsets": [[0, 227]],
139
  }
140
  ],
141
  "entities": [
142
  {
143
  "id": "T1",
144
+ "type": "ENFERMEDAD",
 
 
 
 
 
 
145
  "text": ["hipertensión grave"],
146
  "offsets": [[45, 63]],
147
  },
148
  {
149
+ "id": "T2",
150
+ "type": "ENFERMEDAD",
151
  "text": ["proteinuria"],
152
  "offsets": [[131, 142]],
153
  },
154
  {
155
+ "id": "T3",
156
+ "type": "ENFERMEDAD",
157
  "text": ["PET"],
158
+ "offsets": [[191, 194]],
159
  },
160
  ],
161
  "events": [],
 
187
  **Example Output:**
188
 
189
  ```text
190
+ ## Mention 1: hipertensión grave
191
  - Beam 1:
192
+ Predicted concept name: hipertensión arterial
193
+ Predicted code: 38341003
194
+ Beam score: 0.993
195
 
196
  - Beam 2:
197
+ Predicted concept name: degeneración vascular hipertensiva
198
+ Predicted code: 38341003
199
+ Beam score: 0.249
200
 
201
  - Beam 3:
202
+ Predicted concept name: hipertensión arterial maligna
203
+ Predicted code: 70272006
204
+ Beam score: 0.046
205
 
206
  - Beam 4:
207
+ Predicted concept name: degeneración macular senil
208
+ Predicted code: 267718000
209
+ Beam score: 0.004
210
 
211
  - Beam 5:
212
+ Predicted concept name: hipertensión maligna secundaria, SAI
213
+ Predicted code: 194784007
214
+ Beam score: 0.001
215
 
216
+ ## Mention 2: proteinuria
217
  - Beam 1:
218
+ Predicted concept name: proteinuria de causa desconocida
219
+ Predicted code: 231860006
220
+ Beam score: 0.000
221
 
222
  - Beam 2:
223
+ Predicted concept name: proteína de la membrana mitocondrial asociada con neurodegeneración
224
+ Predicted code: 709415008
225
+ Beam score: 0.000
226
 
227
  - Beam 3:
228
+ Predicted concept name: proteinuria aislada concomitante con glomerulonefritis membranoproliferativa tipo III y debida a ella
229
+ Predicted code: 368931000119104
230
+ Beam score: 0.000
231
 
232
  - Beam 4:
233
+ Predicted concept name: proteinosis alveolar pulmonar congénita
234
+ Predicted code: 707442002
235
+ Beam score: 0.000
236
 
237
  - Beam 5:
238
+ Predicted concept name: proteinosis alveolar pulmonar
239
+ Predicted code: 10501004
240
+ Beam score: 0.000
241
 
242
+ ## Mention 3: PET
243
  - Beam 1:
244
+ Predicted concept name: preeclampsia
245
+ Predicted code: 398254007
246
+ Beam score: 0.285
247
 
248
  - Beam 2:
249
+ Predicted concept name: preeclampsia en el puerperio
250
+ Predicted code: 765182005
251
+ Beam score: 0.068
252
 
253
  - Beam 3:
254
+ Predicted concept name: púrpura trombocitopénica
255
+ Predicted code: 302873008
256
+ Beam score: 0.000
257
 
258
  - Beam 4:
259
+ Predicted concept name: púrpura de la vulva
260
+ Predicted code: 289487000
261
+ Beam score: 0.000
262
 
263
  - Beam 5:
264
+ Predicted concept name: pústula maligna
265
+ Predicted code: 84980006
266
+ Beam score: 0.000
267
+ ```
268
 
269
+ ### Saliency map example
 
 
 
 
270
 
271
+ The model can also return token-level saliency maps during inference.
 
 
 
272
 
273
+ ```python
274
+ predictions, saliency_maps = model.sample(
275
+ bigbio_pages=bigbio_pages,
276
+ num_beams=num_beams,
277
+ with_saliency_maps=True,
278
+ )
279
 
280
+ model.display_saliency_map(saliency_maps[2])
281
+ ````
 
 
282
 
283
+ Example saliency map for the mention `PET`:
284
+
285
+ <p align="center">
286
+ <img src="saliency_map.png" alt="Saliency map for PET prediction" width="900">
287
+ </p>
288
 
289
  ## Evaluation
290
 
291
+ Entity linking performance is reported using Recall@1 with bootstrap confidence intervals. The best result is shown in **bold**, and the second-best result is <u>underlined</u> and ⭐ marks the main LongBEL-8B model.
292
 
293
  | Model | MM-ST21PV<br>(English) | QUAERO-EMEA<br>(French) | SympTEMIST<br>(Spanish) | DisTEMIST<br>(Spanish) | MedProcNER<br>(Spanish) |
294
  | :--- | :---: | :---: | :---: | :---: | :---: |
 
306
  | **Global-Context BEL: LongBEL** ||||| |
307
  | LongBEL-1B| 77.6 ± 0.9 | 74.5 ± 3.7 | 59.8 ± 2.5 | 61.9 ± 2.4 | 66.6 ± 2.1 |
308
  | LongBEL-1B + Ensemble | 78.6 ± 0.8 | <u>77.2 ± 3.0</u> | 61.8 ± 2.5 | <u>64.3 ± 2.2</u> | <u>69.0 ± 2.0</u> |
309
+ | **LongBEL-8B** | <u>79.3 ± 0.8</u> | 75.4 ± 3.4 | <u>62.0 ± 2.6</u> | 63.6 ± 2.1 | <u>69.0 ± 2.1</u> |
310
  | LongBEL-8B + Ensemble | **80.0 ± 0.8** | **77.6 ± 3.0** | **63.3 ± 2.5** | **65.8 ± 2.2** | **71.0 ± 2.0** |
311
 
312
  The score reported for this checkpoint is the **single LongBEL-8B model**. The ensemble result requires fusing several LongBEL input configurations and is not produced by this checkpoint alone.
 
337
  Code and evaluation scripts are available in this [GitHub repository](https://anonymous.4open.science/r/LongBEL-31AD).
338
 
339
  Trained model checkpoints and processed datasets are available in the anonymous Hugging Face collection associated with LongBEL.