--- license: cc-by-nc-4.0 tags: - autonomous-driving - object-detection - open-set-recognition - functional-safety - neuro-symbolic pipeline_tag: object-detection thumbnail: https://huggingface.co/freshNfunky/howc/resolve/main/assets/hero_poster.jpg --- # HOWC : Hierarchical Open-World Classification for road perception

arXiv Video GitHub License Training-free

> A **training-free** layer that turns a flat object detector into a hierarchical, open-set one. Every detection is resolved by **taxonomic abstraction**: the most specific level the evidence safely supports, or an explicit **UNKNOWN OBSTACLE**, never a confident wrong leaf. The live tool. A deer, a class no closed detector is asked to output on the road, is not dropped and not mislabeled: it resolves to Living Being, with the full taxonomy descent path and an independent segmentation cross-check shown alongside. ## Watch the 3-minute explainer

HOWC explainer video

--- ## The problem A closed-set detector must assign every object one of a fixed set of labels. On an object outside that set (a horse-drawn carriage, road debris, livestock on a rural road) it can only do one of two unsafe things: force a **confident but wrong** specific label, or **drop** the object entirely. Real incidents trace back to exactly this failure. ## How it works HOWC puts a **taxonomy** over the detector and, per object, aggregates leaf probability mass up the tree. It commits only as deep as one branch holds enough mass; otherwise it stops at a coarser node. A per-branch **safety floor** (`Vehicle`, `Living Being`, `Static Object`) means it never collapses into a useless bare `Object`: below the floor, it emits an explicit **UNKNOWN OBSTACLE** with a fully inspectable decision path. The object taxonomy, with safety floors marked ## How it decides (worked examples) Each detection produces a decision path: probability mass per node, the safety floor, and the resulting action.
ABSTRACTED: a truck YOLO calls 'car' is abstracted up to the safe Vehicle floor
ABSTRACTED. YOLO calls a distant truck "car" (0.33). The mass is ambiguous below the Vehicle floor, so HOWC does not guess a leaf: it reports Vehicle, still useful, never wrong. 		UNKNOWN: mass splits above the safety floor, so the object is flagged UNKNOWN OBSTACLE
UNKNOWN. A rural-road anomaly YOLO calls "person" (0.72). The mass splits above the floor across Living Being / Vehicle / Static, so HOWC refuses a category and flags UNKNOWN OBSTACLE.
## Beyond 2D: a depth cue for flat detections A 2D box cannot tell a real vehicle from a flat picture of one (a billboard, a truck livery). The open-world study adds a monocular-depth relief cue: detections that are geometrically **flat** get flagged, and the depth field feeds the open-world proposal front-end. Left: 3D mode flags flat detections on a horse trailer scene. Right: monocular depth (nearer = bright) Left: 3D mode flags flat detections. Right: the monocular depth field (nearer = bright). This is a feasibility signal, not a solved module, and the paper is explicit that no single 2D cue suffices. ## Results (honest) **Out-of-vocabulary objects** (v3 ground-truth leave-classes-out on COCO, n=235). Hold seven classes out of the taxonomy, then classify their real annotated crops: Flat head is 100% confidently wrong; HOWC is 0% confidently wrong and 94% safe | On out-of-vocabulary objects | Flat / closed head | **HOWC** | |---|---|---| | Confident **wrong** specific label | **100%** | **0%** | | ... of which in the **wrong super-category** | 37% | 0% | | Safely handled (right super-cat, or honest UNKNOWN) | 0% | **94%** | | Correct super-category recovered | — | 26% | | Conservatively flagged UNKNOWN | — | 69% | On **known, in-taxonomy** objects: **0% off-branch (categorical) errors** with ~24% calibrated abstention, versus a flat arg-max head's ~53% off-branch errors. The safety/specificity trade-off is a dial, not a fixed point. Every hierarchical configuration stays **100% safe on novel objects** while still usefully classifying up to 76% of known ones; the flat baseline sits alone at 0% novel-safe. Safety vs specificity: every hierarchical config is 100% novel-safe; the flat baseline is 0% novel-safe > **What this is, and is not.** This is a **safety** result, not a specificity one. On closed-set COCO mAP a trained YOLO is more accurate, and we say so. The contribution is the abstraction layer over open-vocabulary (CLIP) features: it never makes a confident categorical mistake on an unknown object, at the honest cost of 69% abstention. The point is the tail, not the leaderboard. ## An independent second path The box classification is cross-checked against a class-agnostic **segmentation** path: does the pixel evidence back up the box verdict? Across sampled detections it **confirms or stays neutral 63%** of the time and **flags 37%** for review, with **0% hard conflicts**, and it agrees most where it should (on abstracted and unknown objects). Segmentation cross-check verdict: 37% confirm, 26% neutral, 37% flag, 0% conflict ## Run it locally Self-contained: code + taxonomy + a Gradio app. Training-free (pretrained YOLO + CLIP zero-shot); first run downloads weights (~360 MB). ```bash pip install -r requirements.txt python app.py # Gradio UI: upload an image, see the taxonomy decision path python app.py --share # same, but also prints a temporary public URL (~72h) ``` ## Papers - **Open-world (v3):** *Open-World Hierarchical Perception: Taxonomic Abstraction over Class-Agnostic Proposals for the Safe Handling of Out-of-Vocabulary Road Objects*, F. Schaller, [arXiv:2608.07577](https://arxiv.org/abs/2608.07577). - **Foundational (v1):** *Hierarchical Taxonomic Abstraction for the Safe Handling of Novel Objects in Autonomous Driving Perception*, [doi:10.5281/zenodo.21593472](https://doi.org/10.5281/zenodo.21593472). - **Source & full history:** ```bibtex @misc{schaller2026howc, title = {Open-World Hierarchical Perception: Taxonomic Abstraction over Class-Agnostic Proposals for the Safe Handling of Out-of-Vocabulary Road Objects}, author = {Schaller, Felix}, year = {2026}, eprint = {2608.07577}, archivePrefix = {arXiv}, primaryClass = {cs.CV}, url = {https://arxiv.org/abs/2608.07577} } ``` ## License CC BY-NC 4.0, matching the paper.