|
Download README.md from Hukyl/parseq-s-rukopys: direct link, hf CLI and curl.
- Browser
- Download file 11.2 kB
-
https://huggingface.co/Hukyl/parseq-s-rukopys/resolve/main/README.md
- Command line
-
hf download hf://Hukyl/parseq-s-rukopys/README.md
-
curl -L -o README.md https://huggingface.co/Hukyl/parseq-s-rukopys/resolve/main/README.md
11.2 kB
| language: | |
| - uk | |
| library_name: pytorch | |
| pipeline_tag: image-to-text | |
| license: cc-by-sa-4.0 | |
| base_model: Hukyl/parseq-s-cyrillic-handwritten | |
| tags: | |
| - ocr | |
| - htr | |
| - handwritten-text-recognition | |
| - parseq | |
| - ukrainian | |
| - cyrillic | |
| - rukopys | |
| - image-to-text | |
| datasets: | |
| - UkrainianCatholicUniversity/rukopys | |
| - annyhnatiuk/ukrainian-handwritten-text | |
| - constantinwerner/cyrillic-handwriting-dataset | |
| - ai-forever/school_notebooks_RU | |
| - pumb-ai/synthetic-cyrillic-large | |
| - nastyboget/synthetic_cyrillic_large | |
| metrics: | |
| - cer | |
| - wer | |
| model-index: | |
| - name: parseq-s-rukopys | |
| results: | |
| - task: | |
| type: image-to-text | |
| name: Handwritten Text Recognition | |
| dataset: | |
| type: UkrainianCatholicUniversity/rukopys | |
| name: Rukopys, page-level val | |
| split: validation | |
| metrics: | |
| - type: cer | |
| value: 0.0935 | |
| name: CER | |
| - type: wer | |
| value: 0.2713 | |
| name: WER | |
| # parseq-s-rukopys | |
| A PARSeq-S (permuted autoregressive sequence) line recognizer for single-line | |
| handwritten Ukrainian document text, fine-tuned on the | |
| [Rukopys](https://huggingface.co/datasets/UkrainianCatholicUniversity/rukopys) dataset. | |
| The training data included `handwritten`, `printed`, `annotation` and `table` region crops from Rukopys. | |
| ## TL;DR | |
| | | value | | |
| |---|---| | |
| | Architecture | PARSeq โ ViT-S encoder + 1-layer permutation-LM decoder (ECCV 2022) | | |
| | Parameters | ~24M | | |
| | Init from | [`Hukyl/parseq-s-cyrillic-handwritten`](https://huggingface.co/Hukyl/parseq-s-cyrillic-handwritten) (mixed pretrain) | | |
| | Handles | `handwritten`, `printed`, `annotation` | | |
| | Trained on | **nearly all** the gold data | | |
| | Decode | AR greedy + 1 refinement iteration | | |
| | Representative gold-val CER / WER | **0.0935 / 0.2713** | | |
| | Input | a single cropped text line, BGR (numpy) โ resized to 48ร512 | | |
| | Output | the transcribed string (232-char frozen Cyrillic charset) | | |
| ## Intended use | |
| Recognizing single cropped lines/regions of handwritten Ukrainian documents, downstream | |
| of a line/region detector or on already-segmented lines. | |
| A companion ViT-B variant fine-tuned with the same recipe is available at | |
| [`Hukyl/parseq-b-rukopys`](https://huggingface.co/Hukyl/parseq-b-rukopys). | |
| ## How to use | |
| Note: the checkpoint is a plain `torch.save` archive, not transformers-compatible. The | |
| payload is a dict: | |
| ```python | |
| import torch | |
| from huggingface_hub import hf_hub_download | |
| path = hf_hub_download("Hukyl/parseq-s-rukopys", "best.pt") | |
| payload = torch.load(path, map_location="cpu", weights_only=True) | |
| # keys: "model_state" (state_dict), "charset" (str), "config" (dict), | |
| # "metrics" (dict), "epoch" (int) | |
| print(payload["config"]) | |
| # {'img_height': 48, 'img_width': 512, 'patch_size': (4, 8), 'embed_dim': 384, | |
| # 'enc_num_heads': 6, 'enc_mlp_ratio': 4, 'enc_depth': 12, 'dec_num_heads': 12, | |
| # 'dec_mlp_ratio': 4, 'dec_depth': 1, 'max_label_length': 100, 'dropout': 0.1, | |
| # 'decode_ar': True, 'refine_iters': 1, 'drop_path_rate': 0.05} | |
| ``` | |
| To run it: construct a PARSeq model (ViT encoder + 1-layer two-stream permutation | |
| decoder; see [`baudm/parseq`](https://github.com/baudm/parseq), Apache-2.0) with the | |
| embedded `config`, build a tokenizer over the embedded `charset` (token order is | |
| `[E]` + the charset string + `[B]` + `[P]`; `[E]` is id 0), then | |
| `model.load_state_dict(payload["model_state"])`. | |
| The input geometry ships in the checkpoint's `config`: each crop is resized | |
| unconditionally to 48ร512 RGB (no aspect preservation; `INTER_AREA` on downscale โ an | |
| `INTER_LINEAR` mismatch aliases on the heavy downscale and degrades accuracy) and | |
| normalized `(x โ 0.5) / 0.5`. Output confidence is the mean per-step max-softmax over | |
| the decoded sequence. | |
| ## Architecture | |
| PARSeq, after Bautista & Atienza, "Scene Text Recognition with Permuted Autoregressive | |
| Sequence Models" (ECCV 2022, [arXiv:2207.06966](https://arxiv.org/abs/2207.06966)). Model | |
| code is derived from [`baudm/parseq`](https://github.com/baudm/parseq) (Apache-2.0). | |
| Geometry and charset are identical to the pretrain stage it is fine-tuned from. | |
| | component | value | | |
| |---|---| | |
| | encoder | ViT-S โ 12 layers, dim 384, 6 heads, MLP ratio 4, patch 4ร8 | | |
| | decoder | 1-layer two-stream permutation-LM, 12 heads, MLP ratio 4 | | |
| | training | K=6 permutations (permuted AR sequence modeling) | | |
| | decode | autoregressive greedy + 1 refinement iteration | | |
| | dropout / drop-path | 0.1 / 0.05 | | |
| | input | 48ร512 RGB, unconditional resize, `(xโ0.5)/0.5` | | |
| | max label length | 100 | | |
| | parameters | ~24M | | |
| ### Charset | |
| A frozen 232-character Cyrillic charset (`sha256 d1b9161eโฆ3ff976`). Token IDs follow | |
| string order; specials are `[E]` (id 0) first and `[B]`, `[P]` last. Text is not | |
| NFKD-normalized. The set covers the full Ukrainian Cyrillic block (incl. ั ั ั า and | |
| the apostrophe), the Russian-only capitals ะญ ะ and letters ั ั ั, digits, | |
| punctuation, and a small set of Greek/math symbols. Latin lookalikes that are | |
| visually identical to Cyrillic letters (`a c e i o y`; `A B C E H K M O P T X`) are | |
| deliberately excluded โ the target output space is Cyrillic-normalized, so those are | |
| folded to their Cyrillic counterparts. | |
| ## Training curriculum | |
| Two stages, seed 42. The trainer filters samples to the frozen charset and drops labels | |
| longer than 100 characters before training. | |
| 1. **Mixed pretrain** โ | |
| [`Hukyl/parseq-s-cyrillic-handwritten`](https://huggingface.co/Hukyl/parseq-s-cyrillic-handwritten): | |
| general Cyrillic handwriting reader trained from scratch on a real + synthetic mix | |
| (~0.75M crops/epoch, synthetic:real โ 1.5:1). See the pretrain card for the full mix | |
| and recipe. | |
| 2. **Rukopys gold fine-tune** (15 ep, LR 5e-5 peak): warm-started from the stage-1 | |
| weights, fine-tuned on the human-labeled Rukopys gold data, with page-level | |
| validation split. Note: to incorporate as much of the Rukopys gold split as possible, | |
| this checkpoint had a near-zero validation set (~5 pages): 22,293 train / 105 val | |
| crops (`handwritten` 21,381 / `annotation` 519 / `printed` 288 / `table` 142, before | |
| charset/length filtering). The published weights are the best-val-CER epoch (11 of 15). | |
| ### Stage-2 hyperparameters (as launched) | |
| | hyperparameter | value | | |
| |---|---| | |
| | epochs | 15 | | |
| | effective batch size | 64 (micro-batch 64 ร accum 1) | | |
| | learning rate | 5e-5 peak | | |
| | schedule | OneCycleLR, warmup 5% | | |
| | optimizer | AdamW, weight_decay 0.05 | | |
| | label smoothing | 0.1 | | |
| | drop-path rate | 0.05 | | |
| | permutations (K) | 6 | | |
| | gradient clipping | 20 | | |
| | precision | bf16 AMP | | |
| | split | by-page, near-full-gold (~5 held-out pages) | | |
| | seed | 42 | | |
| ### Online data augmentation | |
| Online augmentation was applied during training (`default` profile). Only training | |
| crops were augmented, and every train crop was augmented (`identity_prob = 0.0`), at | |
| the 48ร512 working resolution (downscaled before augmenting). | |
| Each crop was transformed once per epoch by _one geometric_ + _one or two photometric_ operations at random. Each class received a separate augmentation profile that was selected to minimize the distribution shift. | |
| | pool | handwritten / annotation | printed | | |
| |---|---|---| | |
| | geometric (pick 1) | margin pad 0.02โ0.15 / trim 0.01โ0.05, rotation ยฑ1โ5ยฐ, elastic distortion (ฮฑ25 ฯ5), baseline warp (amp 2โ8 px, freq 0.5โ2) | margin pad 0.02โ0.15 / trim 0.01โ0.05, rotation ยฑ1โ3ยฐ | | |
| | photometric (pick 1โ2) | paper-colour shift, Gaussian noise ฯ5โ15, JPEG q30โ65, contrast 0.7โ1.3 / gamma 0.6โ1.5, morphological op (k=2) | paper-colour shift, Gaussian noise ฯ3โ10, JPEG q40โ70, contrast 0.8โ1.2 / gamma 0.8โ1.3, morphological op (k=2) | | |
| ## Results | |
| Protocol: AR greedy + 1 refinement iteration, page-level gold validation split | |
| (val crops come from pages the model never saw in training), seed 42. | |
| ### Why the numbers come from a sibling | |
| Since this checkpoint included almost all of the gold split, measuring CER on the tiny | |
| val split would result in meaningless metrics. Instead, we report CER on the sibling | |
| model trained with the identical recipe (same warm-start, hyperparameters, augmentation, | |
| and seed) on a standard ~185-page held-out split. We suspect that this model is strictly | |
| equal or better than the reported numbers. | |
| | metric | training-time best | standalone predictor eval (n=3,202) | | |
| |---|---|---| | |
| | CER | 0.0935 | 0.0954 | | |
| | WER | 0.2713 | 0.2745 | | |
| | exact-match accuracy | โ | 0.4453 | | |
| ### Per class | |
| | class | n | CER | WER | accuracy | | |
| |---|---:|---:|---:|---:| | |
| | handwritten | 3,067 | 0.0792 | 0.2566 | 0.4522 | | |
| | annotation | 73 | 0.5457 | 0.6555 | 0.4110 | | |
| | printed | 48 | 0.4097 | 0.7783 | 0.1875 | | |
| | table | 14 | 1.9864 | 1.4118 | 0.0000 | | |
| We also acknowledge that `printed`/`annotation`/`table` n is quite small, so measuring | |
| CER against them is quite noisy. `table` (multi-line pipe-separated serialization) is | |
| not a target of this model; its CER above 1 is expected. | |
| ## Limitations & biases | |
| - Handwritten Ukrainian archival document material only; expect degradation on other | |
| scripts, languages, or modern born-digital text. | |
| - The character set and normalization are tuned to the document-OCR metric โ outputs are | |
| normalized text, not faithful transcription. Latin lookalikes are folded to their | |
| Cyrillic counterparts, so the model never emits the Latin forms. | |
| - `printed` and `annotation` have small training support and high CER; `table` and | |
| `formula` are unsupported. | |
| - Headline metrics are from an identical-recipe sibling, not this exact checkpoint (its | |
| own holdout is too small to score). | |
| - Single seed โ no across-run variance estimate. | |
| ## Training data & attribution | |
| | dataset | source | license | role | | |
| |---|---|---|---| | |
| | Rukopys | [`UkrainianCatholicUniversity/rukopys`](https://huggingface.co/datasets/UkrainianCatholicUniversity/rukopys) | CC BY 4.0 | gold fine-tune | | |
| | UkrHandwritten | Kaggle [`annyhnatiuk/ukrainian-handwritten-text`](https://www.kaggle.com/datasets/annyhnatiuk/ukrainian-handwritten-text) | CC BY-SA 4.0 | pretrain (lineage) | | |
| | Cyrillic Handwriting Dataset | Kaggle [`constantinwerner/cyrillic-handwriting-dataset`](https://www.kaggle.com/datasets/constantinwerner/cyrillic-handwriting-dataset) | CC0 | pretrain (lineage) | | |
| | school_notebooks_RU | HF [`ai-forever/school_notebooks_RU`](https://huggingface.co/datasets/ai-forever/school_notebooks_RU) | MIT | pretrain (lineage) | | |
| | pumb-ai/synthetic-cyrillic-large | HF [`pumb-ai/synthetic-cyrillic-large`](https://huggingface.co/datasets/pumb-ai/synthetic-cyrillic-large) | Apache-2.0 | pretrain (lineage) | | |
| | nastyboget/synthetic_cyrillic_large | HF [`nastyboget/synthetic_cyrillic_large`](https://huggingface.co/datasets/nastyboget/synthetic_cyrillic_large) | MIT | pretrain (lineage) | | |
| ## License & lineage | |
| Model weights are released under CC BY-SA 4.0 โ the most restrictive of the training | |
| inputs now that Rukopys is CC BY 4.0 (UkrHandwritten is share-alike, and shares this | |
| requirement down to the fine-tuned weights). The PARSeq model code is Apache-2.0 | |
| ([`baudm/parseq`](https://github.com/baudm/parseq)). The stage-1 | |
| [pretrain checkpoint](https://huggingface.co/Hukyl/parseq-s-cyrillic-handwritten), which | |
| did not train on Rukopys, is released under CC BY-SA 4.0. | |