HOWC hierarchical perception: card + runnable code

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/hero.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/hero_poster.jpg filter=lfs diff=lfs merge=lfs -text

README.md

@@ -7,51 +7,131 @@ tags:
   - functional-safety
   - neuro-symbolic
 pipeline_tag: object-detection
+thumbnail: https://raw.githubusercontent.com/freshNfunky/IE2025-Research-Paper/main/figures/title_scene_annotated.png
 ---
 
-# HOWC : hierarchical taxonomic perception for novel road objects
+# HOWC : Hierarchical Open-World Classification for road perception
 
-A training-free layer that turns a flat object detector into a hierarchical,
-open-set one. Each detection is classified by **taxonomic abstraction**: the most
-specific level the evidence safely supports, or an explicit **UNKNOWN OBSTACLE**,
-never a confident wrong leaf.
+<p>
+  <a href="https://arxiv.org/abs/2608.07577"><img src="https://img.shields.io/badge/arXiv-2608.07577-b31b1b.svg" alt="arXiv"></a>
+  <a href="https://youtu.be/Awy3qmCCe78"><img src="https://img.shields.io/badge/YouTube-explainer-red.svg?logo=youtube" alt="Video"></a>
+  <a href="https://github.com/freshNfunky/IE2025-Research-Paper"><img src="https://img.shields.io/badge/GitHub-source-181717.svg?logo=github" alt="GitHub"></a>
+  <img src="https://img.shields.io/badge/license-CC%20BY--NC%204.0-lightgrey.svg" alt="License">
+  <img src="https://img.shields.io/badge/training-free-2fd6a6.svg" alt="Training-free">
+</p>
 
-Paper (open access): *Hierarchical Taxonomic Abstraction for the Safe Handling of
-Novel Objects in Autonomous Driving Perception*, F. Schaller,
-[doi:10.5281/zenodo.21593472](https://doi.org/10.5281/zenodo.21593472).
-Source & full history: <https://github.com/freshNfunky/IE2025-Research-Paper>.
+> A **training-free** layer that turns a flat object detector into a hierarchical, open-set one. Every detection is resolved by **taxonomic abstraction**: the most specific level the evidence safely supports, or an explicit **UNKNOWN OBSTACLE**, never a confident wrong leaf.
 
-## Why it is different
+<video autoplay loop muted playsinline width="100%" poster="https://huggingface.co/freshNfunky/howc/resolve/main/assets/hero_poster.jpg">
+  <source src="https://huggingface.co/freshNfunky/howc/resolve/main/assets/hero.mp4" type="video/mp4"/>
+  <img src="https://huggingface.co/freshNfunky/howc/resolve/main/assets/hero_poster.jpg" alt="HOWC live tool: a deer crossing a road is classified as Living Being, with the taxonomy descent path and a segmentation cross-check" width="100%"/>
+</video>
 
-A flat detector returns one fixed class or nothing. On an untrained object (a
-horse-drawn carriage, an overloaded truck) it must mislabel it or drop it, both
-unsafe. HOWC abstracts up a taxonomy to a still-useful category
-(… → Truck → Transport Vehicle → Vehicle), bounded by a per-branch **safety
-floor** so it never collapses into a useless "Object"; below the floor it flags an
-explicit **UNKNOWN OBSTACLE** with an inspectable decision path.
+<sub>The live tool. A deer, a class no closed detector is asked to output on the road, is not dropped and not mislabeled: it resolves to <code>Living Being</code>, with the full taxonomy descent path and an independent segmentation cross-check shown alongside.</sub>
 
-## Honest scope
+## Watch the 3-minute explainer
 
-- Not new weights, and not a closed-set-accuracy win: on COCO mAP a trained YOLO
-  is more accurate. The contribution is the **taxonomic abstraction layer** over
-  open-vocabulary (CLIP) features.
-- Where it wins: on known objects, **0% categorical (off-branch) errors** with
-  ~24% calibrated abstention, vs a flat head's ~53% off-branch errors; on novel
-  objects, a safe coarse label or a flagged UNKNOWN instead of a confident wrong
-  leaf.
-- Training-free (pretrained YOLO + CLIP zero-shot). First run downloads weights
-  (~360 MB).
+<p align="center">
+  <a href="https://youtu.be/Awy3qmCCe78">
+    <img src="https://img.youtube.com/vi/Awy3qmCCe78/maxresdefault.jpg" alt="HOWC explainer video" width="70%">
+  </a>
+</p>
+
+---
+
+## The problem
+
+A closed-set detector must assign every object one of a fixed set of labels. On an object outside that set (a horse-drawn carriage, road debris, livestock on a rural road) it can only do one of two unsafe things: force a **confident but wrong** specific label, or **drop** the object entirely. Real incidents trace back to exactly this failure.
+
+## How it works
+
+HOWC puts a **taxonomy** over the detector and, per object, aggregates leaf probability mass up the tree. It commits only as deep as one branch holds enough mass; otherwise it stops at a coarser node. A per-branch **safety floor** (`Vehicle`, `Living Being`, `Static Object`) means it never collapses into a useless bare `Object`: below the floor, it emits an explicit **UNKNOWN OBSTACLE** with a fully inspectable decision path.
+
+<img src="https://raw.githubusercontent.com/freshNfunky/IE2025-Research-Paper/main/figures/taxonomy_architecture.png" alt="The object taxonomy, with safety floors marked" width="100%">
+
+## How it decides (worked examples)
+
+Each detection produces a decision path: probability mass per node, the safety floor, and the resulting action.
+
+<table>
+<tr>
+<td width="50%" valign="top">
+<img src="https://raw.githubusercontent.com/freshNfunky/IE2025-Research-Paper/main/figures/ex_road_anomaly_04_2_abstracted.png" alt="ABSTRACTED: a truck YOLO calls 'car' is abstracted up to the safe Vehicle floor" width="100%">
+<br><sub><b>ABSTRACTED.</b> YOLO calls a distant truck "car" (0.33). The mass is ambiguous below the <code>Vehicle</code> floor, so HOWC does not guess a leaf: it reports <b>Vehicle</b>, still useful, never wrong.</sub>
+</td>
+<td width="50%" valign="top">
+<img src="https://raw.githubusercontent.com/freshNfunky/IE2025-Research-Paper/main/figures/ex_road_anomaly_02_1_unknown.png" alt="UNKNOWN: mass splits above the safety floor, so the object is flagged UNKNOWN OBSTACLE" width="100%">
+<br><sub><b>UNKNOWN.</b> A rural-road anomaly YOLO calls "person" (0.72). The mass splits <i>above</i> the floor across Living Being / Vehicle / Static, so HOWC refuses a category and flags <b>UNKNOWN OBSTACLE</b>.</sub>
+</td>
+</tr>
+</table>
+
+## Beyond 2D: a depth cue for flat detections
+
+A 2D box cannot tell a real vehicle from a flat picture of one (a billboard, a truck livery). The open-world study adds a monocular-depth relief cue: detections that are geometrically **flat** get flagged, and the depth field feeds the open-world proposal front-end.
+
+<img src="https://raw.githubusercontent.com/freshNfunky/IE2025-Research-Paper/main/figures/depth3d_04.png" alt="Left: 3D mode flags flat detections on a horse trailer scene. Right: monocular depth (nearer = bright)" width="100%">
+
+<sub>Left: 3D mode flags flat detections. Right: the monocular depth field (nearer = bright). This is a feasibility signal, not a solved module, and the paper is explicit that no single 2D cue suffices.</sub>
+
+## Results (honest)
+
+**Out-of-vocabulary objects** (v3 ground-truth leave-classes-out on COCO, n=235). Hold seven classes out of the taxonomy, then classify their real annotated crops:
+
+<img src="https://raw.githubusercontent.com/freshNfunky/IE2025-Research-Paper/main/figures/v3_openworld_benchmark.png" alt="Flat head is 100% confidently wrong; HOWC is 0% confidently wrong and 94% safe" width="90%">
+
+| On out-of-vocabulary objects | Flat / closed head | **HOWC** |
+|---|---|---|
+| Confident **wrong** specific label | **100%** | **0%** |
+| ... of which in the **wrong super-category** | 37% | 0% |
+| Safely handled (right super-cat, or honest UNKNOWN) | 0% | **94%** |
+| Correct super-category recovered | — | 26% |
+| Conservatively flagged UNKNOWN | — | 69% |
+
+On **known, in-taxonomy** objects: **0% off-branch (categorical) errors** with ~24% calibrated abstention, versus a flat arg-max head's ~53% off-branch errors.
+
+The safety/specificity trade-off is a dial, not a fixed point. Every hierarchical configuration stays **100% safe on novel objects** while still usefully classifying up to 76% of known ones; the flat baseline sits alone at 0% novel-safe.
+
+<img src="https://raw.githubusercontent.com/freshNfunky/IE2025-Research-Paper/main/figures/calibration_tradeoff.png" alt="Safety vs specificity: every hierarchical config is 100% novel-safe; the flat baseline is 0% novel-safe" width="80%">
+
+> **What this is, and is not.** This is a **safety** result, not a specificity one. On closed-set COCO mAP a trained YOLO is more accurate, and we say so. The contribution is the abstraction layer over open-vocabulary (CLIP) features: it never makes a confident categorical mistake on an unknown object, at the honest cost of 69% abstention. The point is the tail, not the leaderboard.
+
+## An independent second path
+
+The box classification is cross-checked against a class-agnostic **segmentation** path: does the pixel evidence back up the box verdict? Across sampled detections it **confirms or stays neutral 63%** of the time and **flags 37%** for review, with **0% hard conflicts**, and it agrees most where it should (on abstracted and unknown objects).
+
+<img src="https://raw.githubusercontent.com/freshNfunky/IE2025-Research-Paper/main/figures/segmentation_agreement.png" alt="Segmentation cross-check verdict: 37% confirm, 26% neutral, 37% flag, 0% conflict" width="70%">
 
 ## Run it locally
 
-This repository is self-contained (code + taxonomy + a Gradio app):
+Self-contained: code + taxonomy + a Gradio app. Training-free (pretrained YOLO + CLIP zero-shot); first run downloads weights (~360 MB).
 
 ```bash
 pip install -r requirements.txt
-python app.py            # Gradio UI: upload an image, see the taxonomy decision
+python app.py            # Gradio UI: upload an image, see the taxonomy decision path
 python app.py --share    # same, but also prints a temporary public URL (~72h)
 ```
 
+## Papers
+
+- **Open-world (v3):** *Open-World Hierarchical Perception: Taxonomic Abstraction over Class-Agnostic Proposals for the Safe Handling of Out-of-Vocabulary Road Objects*, F. Schaller, [arXiv:2608.07577](https://arxiv.org/abs/2608.07577).
+- **Foundational (v1):** *Hierarchical Taxonomic Abstraction for the Safe Handling of Novel Objects in Autonomous Driving Perception*, [doi:10.5281/zenodo.21593472](https://doi.org/10.5281/zenodo.21593472).
+- **Source & full history:** <https://github.com/freshNfunky/IE2025-Research-Paper>
+
+```bibtex
+@misc{schaller2026howc,
+  title         = {Open-World Hierarchical Perception: Taxonomic Abstraction over
+                   Class-Agnostic Proposals for the Safe Handling of
+                   Out-of-Vocabulary Road Objects},
+  author        = {Schaller, Felix},
+  year          = {2026},
+  eprint        = {2608.07577},
+  archivePrefix = {arXiv},
+  primaryClass  = {cs.CV},
+  url           = {https://arxiv.org/abs/2608.07577}
+}
+```
+
 ## License
 
 CC BY-NC 4.0, matching the paper.

assets/hero.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d947155a39a506b5a9793c0ca04811978592117515dfd7ec6c91e1395a350d76
+size 451523