--- language: - uk library_name: pytorch pipeline_tag: image-to-text license: cc-by-sa-4.0 base_model: Hukyl/parseq-s-cyrillic-handwritten tags: - ocr - htr - handwritten-text-recognition - parseq - ukrainian - cyrillic - rukopys - image-to-text datasets: - UkrainianCatholicUniversity/rukopys - annyhnatiuk/ukrainian-handwritten-text - constantinwerner/cyrillic-handwriting-dataset - ai-forever/school_notebooks_RU - pumb-ai/synthetic-cyrillic-large - nastyboget/synthetic_cyrillic_large metrics: - cer - wer model-index: - name: parseq-s-rukopys results: - task: type: image-to-text name: Handwritten Text Recognition dataset: type: UkrainianCatholicUniversity/rukopys name: Rukopys, page-level val split: validation metrics: - type: cer value: 0.0935 name: CER - type: wer value: 0.2713 name: WER --- # parseq-s-rukopys A PARSeq-S (permuted autoregressive sequence) line recognizer for single-line handwritten Ukrainian document text, fine-tuned on the [Rukopys](https://huggingface.co/datasets/UkrainianCatholicUniversity/rukopys) dataset. The training data included `handwritten`, `printed`, `annotation` and `table` region crops from Rukopys. ## TL;DR | | value | |---|---| | Architecture | PARSeq — ViT-S encoder + 1-layer permutation-LM decoder (ECCV 2022) | | Parameters | ~24M | | Init from | [`Hukyl/parseq-s-cyrillic-handwritten`](https://huggingface.co/Hukyl/parseq-s-cyrillic-handwritten) (mixed pretrain) | | Handles | `handwritten`, `printed`, `annotation` | | Trained on | **nearly all** the gold data | | Decode | AR greedy + 1 refinement iteration | | Representative gold-val CER / WER | **0.0935 / 0.2713** | | Input | a single cropped text line, BGR (numpy) — resized to 48×512 | | Output | the transcribed string (232-char frozen Cyrillic charset) | ## Intended use Recognizing single cropped lines/regions of handwritten Ukrainian documents, downstream of a line/region detector or on already-segmented lines. A companion ViT-B variant fine-tuned with the same recipe is available at [`Hukyl/parseq-b-rukopys`](https://huggingface.co/Hukyl/parseq-b-rukopys). ## How to use Note: the checkpoint is a plain `torch.save` archive, not transformers-compatible. The payload is a dict: ```python import torch from huggingface_hub import hf_hub_download path = hf_hub_download("Hukyl/parseq-s-rukopys", "best.pt") payload = torch.load(path, map_location="cpu", weights_only=True) # keys: "model_state" (state_dict), "charset" (str), "config" (dict), # "metrics" (dict), "epoch" (int) print(payload["config"]) # {'img_height': 48, 'img_width': 512, 'patch_size': (4, 8), 'embed_dim': 384, # 'enc_num_heads': 6, 'enc_mlp_ratio': 4, 'enc_depth': 12, 'dec_num_heads': 12, # 'dec_mlp_ratio': 4, 'dec_depth': 1, 'max_label_length': 100, 'dropout': 0.1, # 'decode_ar': True, 'refine_iters': 1, 'drop_path_rate': 0.05} ``` To run it: construct a PARSeq model (ViT encoder + 1-layer two-stream permutation decoder; see [`baudm/parseq`](https://github.com/baudm/parseq), Apache-2.0) with the embedded `config`, build a tokenizer over the embedded `charset` (token order is `[E]` + the charset string + `[B]` + `[P]`; `[E]` is id 0), then `model.load_state_dict(payload["model_state"])`. The input geometry ships in the checkpoint's `config`: each crop is resized unconditionally to 48×512 RGB (no aspect preservation; `INTER_AREA` on downscale — an `INTER_LINEAR` mismatch aliases on the heavy downscale and degrades accuracy) and normalized `(x − 0.5) / 0.5`. Output confidence is the mean per-step max-softmax over the decoded sequence. ## Architecture PARSeq, after Bautista & Atienza, "Scene Text Recognition with Permuted Autoregressive Sequence Models" (ECCV 2022, [arXiv:2207.06966](https://arxiv.org/abs/2207.06966)). Model code is derived from [`baudm/parseq`](https://github.com/baudm/parseq) (Apache-2.0). Geometry and charset are identical to the pretrain stage it is fine-tuned from. | component | value | |---|---| | encoder | ViT-S — 12 layers, dim 384, 6 heads, MLP ratio 4, patch 4×8 | | decoder | 1-layer two-stream permutation-LM, 12 heads, MLP ratio 4 | | training | K=6 permutations (permuted AR sequence modeling) | | decode | autoregressive greedy + 1 refinement iteration | | dropout / drop-path | 0.1 / 0.05 | | input | 48×512 RGB, unconditional resize, `(x−0.5)/0.5` | | max label length | 100 | | parameters | ~24M | ### Charset A frozen 232-character Cyrillic charset (`sha256 d1b9161e…3ff976`). Token IDs follow string order; specials are `[E]` (id 0) first and `[B]`, `[P]` last. Text is not NFKD-normalized. The set covers the full Ukrainian Cyrillic block (incl. і ї є ґ and the apostrophe), the Russian-only capitals Э Ё and letters ъ ы ё, digits, punctuation, and a small set of Greek/math symbols. Latin lookalikes that are visually identical to Cyrillic letters (`a c e i o y`; `A B C E H K M O P T X`) are deliberately excluded — the target output space is Cyrillic-normalized, so those are folded to their Cyrillic counterparts. ## Training curriculum Two stages, seed 42. The trainer filters samples to the frozen charset and drops labels longer than 100 characters before training. 1. **Mixed pretrain** — [`Hukyl/parseq-s-cyrillic-handwritten`](https://huggingface.co/Hukyl/parseq-s-cyrillic-handwritten): general Cyrillic handwriting reader trained from scratch on a real + synthetic mix (~0.75M crops/epoch, synthetic:real ≈ 1.5:1). See the pretrain card for the full mix and recipe. 2. **Rukopys gold fine-tune** (15 ep, LR 5e-5 peak): warm-started from the stage-1 weights, fine-tuned on the human-labeled Rukopys gold data, with page-level validation split. Note: to incorporate as much of the Rukopys gold split as possible, this checkpoint had a near-zero validation set (~5 pages): 22,293 train / 105 val crops (`handwritten` 21,381 / `annotation` 519 / `printed` 288 / `table` 142, before charset/length filtering). The published weights are the best-val-CER epoch (11 of 15). ### Stage-2 hyperparameters (as launched) | hyperparameter | value | |---|---| | epochs | 15 | | effective batch size | 64 (micro-batch 64 × accum 1) | | learning rate | 5e-5 peak | | schedule | OneCycleLR, warmup 5% | | optimizer | AdamW, weight_decay 0.05 | | label smoothing | 0.1 | | drop-path rate | 0.05 | | permutations (K) | 6 | | gradient clipping | 20 | | precision | bf16 AMP | | split | by-page, near-full-gold (~5 held-out pages) | | seed | 42 | ### Online data augmentation Online augmentation was applied during training (`default` profile). Only training crops were augmented, and every train crop was augmented (`identity_prob = 0.0`), at the 48×512 working resolution (downscaled before augmenting). Each crop was transformed once per epoch by _one geometric_ + _one or two photometric_ operations at random. Each class received a separate augmentation profile that was selected to minimize the distribution shift. | pool | handwritten / annotation | printed | |---|---|---| | geometric (pick 1) | margin pad 0.02–0.15 / trim 0.01–0.05, rotation ±1–5°, elastic distortion (α25 σ5), baseline warp (amp 2–8 px, freq 0.5–2) | margin pad 0.02–0.15 / trim 0.01–0.05, rotation ±1–3° | | photometric (pick 1–2) | paper-colour shift, Gaussian noise σ5–15, JPEG q30–65, contrast 0.7–1.3 / gamma 0.6–1.5, morphological op (k=2) | paper-colour shift, Gaussian noise σ3–10, JPEG q40–70, contrast 0.8–1.2 / gamma 0.8–1.3, morphological op (k=2) | ## Results Protocol: AR greedy + 1 refinement iteration, page-level gold validation split (val crops come from pages the model never saw in training), seed 42. ### Why the numbers come from a sibling Since this checkpoint included almost all of the gold split, measuring CER on the tiny val split would result in meaningless metrics. Instead, we report CER on the sibling model trained with the identical recipe (same warm-start, hyperparameters, augmentation, and seed) on a standard ~185-page held-out split. We suspect that this model is strictly equal or better than the reported numbers. | metric | training-time best | standalone predictor eval (n=3,202) | |---|---|---| | CER | 0.0935 | 0.0954 | | WER | 0.2713 | 0.2745 | | exact-match accuracy | — | 0.4453 | ### Per class | class | n | CER | WER | accuracy | |---|---:|---:|---:|---:| | handwritten | 3,067 | 0.0792 | 0.2566 | 0.4522 | | annotation | 73 | 0.5457 | 0.6555 | 0.4110 | | printed | 48 | 0.4097 | 0.7783 | 0.1875 | | table | 14 | 1.9864 | 1.4118 | 0.0000 | We also acknowledge that `printed`/`annotation`/`table` n is quite small, so measuring CER against them is quite noisy. `table` (multi-line pipe-separated serialization) is not a target of this model; its CER above 1 is expected. ## Limitations & biases - Handwritten Ukrainian archival document material only; expect degradation on other scripts, languages, or modern born-digital text. - The character set and normalization are tuned to the document-OCR metric — outputs are normalized text, not faithful transcription. Latin lookalikes are folded to their Cyrillic counterparts, so the model never emits the Latin forms. - `printed` and `annotation` have small training support and high CER; `table` and `formula` are unsupported. - Headline metrics are from an identical-recipe sibling, not this exact checkpoint (its own holdout is too small to score). - Single seed — no across-run variance estimate. ## Training data & attribution | dataset | source | license | role | |---|---|---|---| | Rukopys | [`UkrainianCatholicUniversity/rukopys`](https://huggingface.co/datasets/UkrainianCatholicUniversity/rukopys) | CC BY 4.0 | gold fine-tune | | UkrHandwritten | Kaggle [`annyhnatiuk/ukrainian-handwritten-text`](https://www.kaggle.com/datasets/annyhnatiuk/ukrainian-handwritten-text) | CC BY-SA 4.0 | pretrain (lineage) | | Cyrillic Handwriting Dataset | Kaggle [`constantinwerner/cyrillic-handwriting-dataset`](https://www.kaggle.com/datasets/constantinwerner/cyrillic-handwriting-dataset) | CC0 | pretrain (lineage) | | school_notebooks_RU | HF [`ai-forever/school_notebooks_RU`](https://huggingface.co/datasets/ai-forever/school_notebooks_RU) | MIT | pretrain (lineage) | | pumb-ai/synthetic-cyrillic-large | HF [`pumb-ai/synthetic-cyrillic-large`](https://huggingface.co/datasets/pumb-ai/synthetic-cyrillic-large) | Apache-2.0 | pretrain (lineage) | | nastyboget/synthetic_cyrillic_large | HF [`nastyboget/synthetic_cyrillic_large`](https://huggingface.co/datasets/nastyboget/synthetic_cyrillic_large) | MIT | pretrain (lineage) | ## License & lineage Model weights are released under CC BY-SA 4.0 — the most restrictive of the training inputs now that Rukopys is CC BY 4.0 (UkrHandwritten is share-alike, and shares this requirement down to the fine-tuned weights). The PARSeq model code is Apache-2.0 ([`baudm/parseq`](https://github.com/baudm/parseq)). The stage-1 [pretrain checkpoint](https://huggingface.co/Hukyl/parseq-s-cyrillic-handwritten), which did not train on Rukopys, is released under CC BY-SA 4.0.