Mostafa-Saeed commited on
Commit
047e1d3
·
verified ·
1 Parent(s): a24fd1f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +21 -21
README.md CHANGED
@@ -5,24 +5,24 @@ language:
5
  pipeline_tag: text-generation
6
  tags:
7
  - Arabic_Dialectal_Lemmatization
8
- ---
9
-
10
  # Gulf-S2S-lemmatizer
11
-
12
- # Model Description
13
-
14
- The model is developed for Arabic dialect lemmatization, focusing on Gulf (GLF) Arabic. It follows a sequence-to-sequence formulation of lemmatization, where the model generates the lemma of a given word knowing 2 words before and 2 words after the current word rather than treating lemmas as fixed classification labels.
15
-
16
- The model is evaluated using lemma accuracy as the main metric, with an additional normalized lemma accuracy metric that accounts for orthographic and diacritic variation. The full methodology, training setup, hyperparameters, and evaluation results are described in our paper [“Lemmatizing Dialectal Arabic with Sequence-to-Sequence Models”](https://aclanthology.org/2025.arabicnlp-main.10/)
17
-
18
-
19
  # Standalone Usage
20
 
21
 
22
-
23
  The model can also be used independently without the full lemmatization workflow on the GitHub repository (https://github.com/CAMeL-Lab/seq2seq-arabic-dialect-lemmatization). In this case, the input should contain the target word surrounded by the special token `<target>`, with up to two words before and two words after the target word.
24
-
25
-
26
  ```python
27
  import re
28
  import math
@@ -33,7 +33,7 @@ from transformers import T5Tokenizer, T5ForConditionalGeneration
33
 
34
  DIALECT_MODELS = {
35
  "glf": "CAMeL-Lab/GLF-S2S-lemmatizer",
36
- }
37
  def load_model(s2s_dialect: str):
38
  model_name = DIALECT_MODELS[s2s_dialect]
39
  tokenizer = T5Tokenizer.from_pretrained(model_name, use_fast=True, legacy=False)
@@ -82,16 +82,16 @@ def predict(tokenizer, model, texts: list[str], device=None, batch_size: int = 1
82
 
83
  # Example:
84
  # df should contain an input_text column with the target word marked using <target>
85
- # Example input: "أنا عايز <target>أروح<target> البيت دلوقتي"
86
 
87
  df["word"] = df["input_text"].apply(extract_target_word)
88
 
89
  tokenizer, model = load_model("egy")
90
  df["predicted_lex"] = predict(tokenizer, model, df["input_text"].tolist())
91
- ```
92
-
93
- ## 📖 Citation
94
-
95
  If you use this model in your research, please cite the following paper:
96
 
97
  ```bibtex
@@ -103,5 +103,5 @@ If you use this model in your research, please cite the following paper:
103
  address = {Suzhou, China},
104
  url = {https://aclanthology.org/2025.arabicnlp-main.10/}
105
  }
106
- ```
107
-
 
5
  pipeline_tag: text-generation
6
  tags:
7
  - Arabic_Dialectal_Lemmatization
8
+ ---
9
+
10
  # Gulf-S2S-lemmatizer
11
+
12
+ # Model Description
13
+
14
+ The model is developed for Arabic dialect lemmatization, focusing on Gulf (GLF) Arabic. It follows a sequence-to-sequence formulation of lemmatization, where the model generates the lemma of a given word knowing 2 words before and 2 words after the current word rather than treating lemmas as fixed classification labels.
15
+
16
+ The model is evaluated using lemma accuracy as the main metric, with an additional normalized lemma accuracy metric that accounts for orthographic and diacritic variation. The full methodology, training setup, hyperparameters, and evaluation results are described in our paper [“Lemmatizing Dialectal Arabic with Sequence-to-Sequence Models”](https://aclanthology.org/2025.arabicnlp-main.10/)
17
+
18
+
19
  # Standalone Usage
20
 
21
 
22
+
23
  The model can also be used independently without the full lemmatization workflow on the GitHub repository (https://github.com/CAMeL-Lab/seq2seq-arabic-dialect-lemmatization). In this case, the input should contain the target word surrounded by the special token `<target>`, with up to two words before and two words after the target word.
24
+
25
+
26
  ```python
27
  import re
28
  import math
 
33
 
34
  DIALECT_MODELS = {
35
  "glf": "CAMeL-Lab/GLF-S2S-lemmatizer",
36
+ }
37
  def load_model(s2s_dialect: str):
38
  model_name = DIALECT_MODELS[s2s_dialect]
39
  tokenizer = T5Tokenizer.from_pretrained(model_name, use_fast=True, legacy=False)
 
82
 
83
  # Example:
84
  # df should contain an input_text column with the target word marked using <target>
85
+ # Example input: "أنا أبي <target>أروح<target> البيت الحين"
86
 
87
  df["word"] = df["input_text"].apply(extract_target_word)
88
 
89
  tokenizer, model = load_model("egy")
90
  df["predicted_lex"] = predict(tokenizer, model, df["input_text"].tolist())
91
+ ```
92
+
93
+ ## 📖 Citation
94
+
95
  If you use this model in your research, please cite the following paper:
96
 
97
  ```bibtex
 
103
  address = {Suzhou, China},
104
  url = {https://aclanthology.org/2025.arabicnlp-main.10/}
105
  }
106
+ ```
107
+