Align Opsis g2 model card with Pyrrho house style

Standardize layout, contracts, quick start, evaluation, artifacts, limitations, and license presentation.

README.md

@@ -7,62 +7,160 @@ pipeline_tag: image-text-to-text
 language:
   - en
 tags:
-  - pyrrho
-  - opsis
   - rag
+  - visual-ingestion
   - image-to-text
   - document-ai
   - table-recognition
+  - pyrrho
+  - opsis
   - onnx
   - cpu
 ---
 
-# Pyrrho Opsis v1 nano g2
+# pyrrho-opsis-v1-nano-g2
+
+`pyrrho-opsis-v1-nano-g2` is a small local RAG visual-ingestion co-processor. It
+runs on one image or document crop and returns either concise discovery text or
+a GitHub-flavored Markdown table for downstream indexing and retrieval.
+
+It is not a general-purpose assistant, not a replacement for native PDF text
+extraction, and not an authoritative OCR system. It sits before indexing, or
+beside a PDF routing pipeline, so visual content can become searchable without
+a hosted VLM API or inference GPU.
+
+Compared with `pyrrho-opsis-v1-nano-g1`, g2 keeps the same v1 capability and
+output contract while moving to a balanced 50,000-file corpus and adapting the
+vision encoder in addition to text attention and the vision-to-text connector.
+G2 is the recommended Opsis v1 nano generation.
+
+## Native V1 Output Modes
 
-Pyrrho Opsis is a small local visual-ingestion co-processor for RAG systems. It converts one image
-into either concise discovery text or a GitHub-flavored Markdown table. Charts and diagrams are
-described with visible labels, values, and relationships. It is designed to let retrieval systems
-index visual PDF regions without a hosted VLM API or inference GPU.
+| Mode | Native output | Intended use |
+|---|---|---|
+| `description` | `<description>concise factual text</description>` | Ordinary images, charts, and diagrams that need searchable RAG metadata. |
+| `table` | `<table>GitHub-flavored Markdown table</table>` | Table images that need recoverable rows and cells rather than prose. |
 
-This is a research preview, not an authoritative OCR or financial/scientific extraction system.
+Charts stay in `description` mode and should include visible labels, values,
+and trends. Diagrams stay in `description` mode and should include visible
+nodes and relationships.
 
-## Intended use
+## Output Contract
 
-- enrich image, chart, diagram, and table crops with searchable RAG text;
-- run locally through ONNX Runtime on CPU;
-- route native PDF text around the model and send only visual regions through Opsis;
-- preserve the source image/page alongside generated text for verification.
+The raw Hugging Face model output is one tagged string. It is not unrestricted
+assistant prose. Decode it by wrapper:
 
-Do not treat generated numbers or table cells as ground truth. Do not use it for safety-critical
-decisions without independent source verification.
+| Wrapper | Parsed kind | Decoding |
+|---|---|---|
+| `<description>...</description>` | `description` | Strip the wrapper and retain concise factual text. |
+| `<table>...</table>` | `table` | Strip the wrapper and retain the complete Markdown table. |
 
-## Output contract
+The companion parser normalizes either result into one object:
 
-Input: one raster image.
+```json
+{
+  "kind": "description",
+  "text": "A line chart shows quarterly revenue rising from $12M in Q1 to $18M in Q4.",
+  "latency_seconds": 3.72,
+  "model_id": "yafitzdev/pyrrho-opsis-v1-nano-g2",
+  "prompt_version": "rag-image-v3"
+}
+```
+
+The model does not return bounding boxes, OCR confidence, source coordinates,
+PDF text blocks, or verified numeric facts. Preserve the source image or page
+crop beside generated text when exact visual evidence matters.
+
+## Intended Use
+
+Use this model when a RAG or retrieval system needs local visual signals for:
 
-Output:
+- indexing ordinary images with concise discovery descriptions,
+- converting compact table images into Markdown,
+- recording visible chart labels, values, and trends,
+- recording labeled diagram nodes and relationships,
+- enriching visual regions routed out of otherwise native-text PDFs,
+- keeping document ingestion local on CPU-only systems.
 
-- `<description>concise factual text</description>` for ordinary images, charts, and diagrams;
-- `<table>...Markdown table...</table>` for table images.
+This model is not intended to replace reliable native table extraction, parse
+full scanned pages as a layout engine, verify high-stakes measurements, or
+replace source review when exact cells and labels matter.
+
+## Input Format
+
+The model accepts one raster image per inference. For direct Transformers use,
+pair the image with the v1 parser prompt:
+
+```text
+Parse this image for RAG search. If it is a table, output only the complete
+table as Markdown. Otherwise output only a concise factual description. Include
+meaningful visible text. For a diagram, name its labels and relationships. For
+a chart, mention its labels, values, and trend. Do not speculate or add a heading.
+```
 
-The packaged parser strips the wrapper tags from its public result.
+For PDFs, extract reliable native text normally and send only table, image,
+chart, or diagram regions to Opsis. Crop quality and readable resolution have a
+direct effect on output quality.
 
-## Model
+## Quick Start
 
-- base architecture: SmolVLM-256M-Instruct;
-- immediate parent checkpoint: Pyrrho Opsis v1 nano g1;
-- generation: Pyrrho Opsis v1 nano g2;
-- total parameters: 258,099,264;
-- adapted parameters: 1,614,336 (0.625%);
-- LoRA targets: text attention, vision attention, and the vision-to-text connector;
-- training resolution: 1024-pixel longest edge.
+```python
+import torch
+from PIL import Image
+from transformers import AutoModelForImageTextToText, AutoProcessor
 
-G2 retains the v1 capability and output contract. It is generation 2 because it changes data
-balance and vision adaptation, not the fundamental task or heads.
+MODEL_ID = "yafitzdev/pyrrho-opsis-v1-nano-g2"
+PROMPT = """Parse this image for RAG search. If it is a table, output only the complete
+table as Markdown. Otherwise output only a concise factual description. Include meaningful
+visible text. For a diagram, name its labels and relationships. For a chart, mention its labels,
+values, and trend. Do not speculate or add a heading."""
 
-## CPU usage
+processor = AutoProcessor.from_pretrained(
+    MODEL_ID,
+    size={"longest_edge": 1024},
+)
+model = AutoModelForImageTextToText.from_pretrained(
+    MODEL_ID,
+    dtype=torch.float32,
+).eval()
+image = Image.open("image.png").convert("RGB")
 
-With the `rag-image-parser` project installed:
+messages = [{
+    "role": "user",
+    "content": [
+        {"type": "image"},
+        {"type": "text", "text": PROMPT},
+    ],
+}]
+prompt_text = processor.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    tokenize=False,
+)
+inputs = processor(text=prompt_text, images=[image], return_tensors="pt")
+
+with torch.no_grad():
+    output_ids = model.generate(
+        **inputs,
+        do_sample=False,
+        max_new_tokens=768,
+        repetition_penalty=1.05,
+    )
+
+generated = output_ids[:, inputs["input_ids"].shape[-1]:]
+text = processor.batch_decode(generated, skip_special_tokens=True)[0]
+print(text)
+```
+
+## CPU ONNX
+
+The repository includes the quality-preserving split FP32 ONNX runtime:
+
+- `onnx/vision_encoder.onnx`
+- `onnx/embed_tokens.onnx`
+- `onnx/decoder_model_merged.onnx`
+
+With the `rag-image-parser` project installed, run:
 
 ```powershell
 uv run rag-image-parse parse .\image.png `
@@ -73,47 +171,94 @@ uv run rag-image-parse parse .\image.png `
   --image-longest-edge 1024
 ```
 
-FP32 ONNX is the recommended and packaged artifact. Mixed INT8 reduced routing and Markdown
-validity in evaluation and is not included in this repository.
+Mixed INT8 reduced routing and Markdown validity in evaluation and is not
+included in this repository.
 
 ## Evaluation
 
-On a 50-case, source-held-out automatic comparison containing ten examples each from tables,
-charts, diagrams, ordinary images, and PDF-domain crops:
+Source-held-out automatic evaluation on 50 cases containing ten examples each
+from tables, charts, diagrams, ordinary images, and PDF-domain crops:
 
 | Metric | G1 | G2 |
 |---|---:|---:|
-| Kind accuracy | 100% | 100% |
-| Valid Markdown tables | 100% | 100% |
-| Exact tables | 0% | 7.1% |
-| Correct table shape | 21.4% | 42.9% |
-| Mean table cell F1 | 31.0% | 50.5% |
-| Required-term recall | 53.8% | 57.7% |
-| Directed-relation recall | 34.0% | 48.7% |
-| Median CPU latency | 3.71 s | 3.72 s |
+| kind accuracy | 1.0000 | 1.0000 |
+| valid Markdown tables | 1.0000 | 1.0000 |
+| exact tables | 0.0000 | 0.0714 |
+| correct table shape | 0.2143 | 0.4286 |
+| mean table cell F1 | 0.3103 | 0.5047 |
+| required-term recall | 0.5382 | 0.5773 |
+| directed-relation recall | 0.3397 | 0.4872 |
+| median CPU latency | 3.71 s | 3.72 s |
 | p95 CPU latency | 16.74 s | 10.95 s |
-| Peak process RSS | 2,377 MiB | 2,382 MiB |
+| peak process RSS | 2,377 MiB | 2,382 MiB |
+
+Bucket-level checkpoint comparison:
+
+| Benchmark slice | G1 | G2 |
+|---|---:|---:|
+| clean-table cell F1 | 0.3928 | 0.5459 |
+| PDF-domain table cell F1 | 0.1040 | 0.4017 |
+| chart required-term recall | 0.7989 | 0.8178 |
+| diagram required-term recall | 0.5207 | 0.6443 |
+| diagram directed-relation recall | 0.3583 | 0.4917 |
+| ordinary-image required-term recall | 0.3375 | 0.3250 |
+
+The benchmark is automatically labeled and too small for a product-quality
+claim. G2 materially improved tables, diagrams, and PDF-domain crops; ordinary-
+image term recall was approximately flat.
+
+## Training Data
+
+| Training bucket | Files | Validation files | Role |
+|---|---:|---:|---|
+| clean tables | 12,500 | 250 | Compact table image to Markdown. |
+| clean charts | 6,250 | 125 | Chart labels, values, and trends. |
+| clean diagrams | 6,250 | 125 | Diagram nodes and relationships. |
+| clean ordinary images | 12,500 | 250 | Ordinary-image discovery descriptions. |
+| PDF-style visual crops | 12,500 | 250 | Document-domain versions balanced across all four content types. |
+| **Total** | **50,000** | **1,000** | Source-held-out balanced visual parsing. |
+
+Sources include PubTabNet, ChartQA, AI2D, and DOCCI. After distributing the PDF
+bucket, the effective training mix is 15,625 tables, 9,375 charts, 9,375
+diagrams, and 15,625 ordinary images. Deterministic label-preserving variants
+expand source pools that contain fewer strict unique examples than the rendered
+file targets.
+
+Training used LoRA rank 8 with alpha 16 for one epoch at a 1024-pixel longest
+edge. It updated 1,614,336 of 258,099,264 parameters (0.625%), including text
+attention, vision attention, and the vision-to-text connector.
 
-This benchmark is automatically labeled and too small for a product-quality claim. Exact table
-accuracy remains low.
+## Artifacts
 
-## Training data
+This repository contains:
 
-G2 trained on 50,000 rendered files:
+- `model.safetensors`: standalone merged Transformers checkpoint,
+- `onnx/vision_encoder.onnx`: FP32 vision encoder and connector,
+- `onnx/embed_tokens.onnx`: FP32 token embedding graph,
+- `onnx/decoder_model_merged.onnx`: FP32 autoregressive decoder,
+- tokenizer, processor, generation, and model configuration files,
+- `rag_image_parser_config.json`: Opsis release and training metadata,
+- `SHA256SUMS`: checksums for the packaged ONNX graphs.
 
-- 12,500 clean tables;
-- 6,250 clean charts;
-- 6,250 clean diagrams;
-- 12,500 clean ordinary images;
-- 12,500 PDF-style crops balanced across the four content types.
+## Limitations
 
-Sources include PubTabNet, ChartQA, AI2D, and DOCCI. Charts, diagrams, and ordinary images use
-deterministic label-preserving variants because their strict unique-source pools are smaller than
-the rendered file counts.
+1. **Exact table recovery remains low.** G2 improved table structure and cell
+   F1, but exact table accuracy was 7.1% on the small held-out benchmark.
+2. **Evaluation is small and automatically labeled.** The 50-case benchmark is
+   useful for checkpoint comparison, not broad product certification.
+3. **Generated labels and values can be wrong.** Descriptions and cells are RAG
+   discovery metadata, not verified evidence.
+4. **English-centric mixed-source training.** Multilingual behavior is not
+   established, and some source pools use deterministic augmentation.
+5. **Crop quality matters.** Tiny text, dense pages, poor scans, and incorrect
+   visual-region routing reduce performance.
+6. **CPU latency varies by image complexity.** Dense tables and long outputs can
+   be substantially slower than the reported median.
 
 ## License
 
-Mixed-source research preview. The training mixture includes research-restricted AI2D data and
-sources with separate attribution, redistribution, or underlying-image terms. This package must
-not be represented as commercially cleared. Review and satisfy every source obligation before
-making the repository public or using the weights commercially.
+Mixed-source research preview. The training mixture includes research-restricted
+AI2D data and sources with separate attribution, redistribution, or underlying-
+image terms. This package must not be represented as commercially cleared.
+Review and satisfy every source obligation before making the repository public
+or using the weights commercially.