docling-project/docling-layout-heron-101
76.7M
โข
Updated
โข
1.23k
โข
5
AI & ML interests
None defined yet.
Recent Activity
View all activity
Add technical report link and text improvements
#1 opened 2 days ago
by
nlivathinos
Add technical report link and text improvements
#3 opened 2 days ago
by
nlivathinos
Add technical report link and text improvements
#3 opened 2 days ago
by
nlivathinos
nlivathinosย
updated
a
model
2 days ago
Add technical report link and text improvements
#2 opened 2 days ago
by
nlivathinos
Improve text styling in Readme
#9 opened 2 days ago
by
nlivathinos
Add technical report link and text improvements
#8 opened 2 days ago
by
nlivathinos
nlivathinosย
updated
a
model
2 days ago
Dual licensing
1
#22 opened about 1 month ago
by
julien-c
lucas-morinย
authored
6
papers
2 months ago
MolGrapher: Graph-based Visual Recognition of Chemical Structures
Paper
โข
2308.12234
โข
Published
Docling: An Efficient Open-Source Toolkit for AI-driven Document Conversion
Paper
โข
2501.17887
โข
Published
โข
1
SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion
Paper
โข
2503.11576
โข
Published
โข
128
Advanced Layout Analysis Models for Docling
Paper
โข
2509.11720
โข
Published
MarkushGrapher: Joint Visual and Textual Recognition of Markush Structures
Paper
โข
2503.16096
โข
Published
โข
1
SubGrapher: Visual Fingerprinting of Chemical Structures
Paper
โข
2504.19695
โข
Published
anditoย
authored
a
paper
3 months ago
Post
1930
Finally, our new paper is out! "๐๐ถ๐ป๐ฒ๐ฉ๐ถ๐๐ถ๐ผ๐ป: ๐ข๐ฝ๐ฒ๐ป ๐๐ฎ๐๐ฎ ๐๐ ๐๐น๐น ๐ฌ๐ผ๐ ๐ก๐ฒ๐ฒ๐ฑ"! ๐ฅณ
FineVision: Open Data Is All You Need (2510.17269)
If you've ever trained a VLM, you know this problem: nobody shares their data mixtures. It's a black box, making replicating SOTA work impossible.
We wanted to change that.
FineVision unifies 200 sources into 24 million samples. With 17.3 million images and 9.5 billion answer tokens, it's the largest open resource of its kind.
In the paper, we share how we built it:
๐ finding and cleaning data at scale
๐งน removing excessive duplicates across sources
๐ค decontaminating against 66 public benchmarks
My favorite part is Figure 6 (in the video!). It's our visual diversity analysis. It shows that FineVision isn't just bigger; it's more balanced and conceptually richer than other open datasets.
NVIDIA's Eagle 2 paper highlighted just how critical this visual diversity is, and our results confirm it: models trained on FineVision consistently outperform those trained on any other open dataset on 11 benchmarks!
๐ To celebrate the paper, Iโm also releasing a concatenated and shuffled version of the full dataset! ๐
Itโs ready to stream, so you can start training your own models right away:
from datasets import load_dataset
d = load_dataset("HuggingFaceM4/FineVision_full_shuffled", split="train", streaming=True)
print(next(iter(d)))
A big shoutout to the first authors: Luis Wiedmann and Orr Zohar. They are rockstars!
FineVision: Open Data Is All You Need (2510.17269)
If you've ever trained a VLM, you know this problem: nobody shares their data mixtures. It's a black box, making replicating SOTA work impossible.
We wanted to change that.
FineVision unifies 200 sources into 24 million samples. With 17.3 million images and 9.5 billion answer tokens, it's the largest open resource of its kind.
In the paper, we share how we built it:
๐ finding and cleaning data at scale
๐งน removing excessive duplicates across sources
๐ค decontaminating against 66 public benchmarks
My favorite part is Figure 6 (in the video!). It's our visual diversity analysis. It shows that FineVision isn't just bigger; it's more balanced and conceptually richer than other open datasets.
NVIDIA's Eagle 2 paper highlighted just how critical this visual diversity is, and our results confirm it: models trained on FineVision consistently outperform those trained on any other open dataset on 11 benchmarks!
๐ To celebrate the paper, Iโm also releasing a concatenated and shuffled version of the full dataset! ๐
HuggingFaceM4/FineVision_full_shuffled Itโs ready to stream, so you can start training your own models right away:
from datasets import load_dataset
d = load_dataset("HuggingFaceM4/FineVision_full_shuffled", split="train", streaming=True)
print(next(iter(d)))
A big shoutout to the first authors: Luis Wiedmann and Orr Zohar. They are rockstars!
Post
3052
Many VLMs claim to process hours of video. But can they follow the story?๐ค
Today, we introduce TimeScope: The benchmark that separates true temporal understanding from marketing hype. Let's see how much VLMs really understand!โณ
We test three skills that matter for real-world use:
๐ Localized Retrieval: Find a specific action.
๐งฉ Information Synthesis: Piece together scattered clues.
๐ Fine-Grained Perception: Analyze detailed motion (e.g., count how many times a person swings an axe).
The results are in, and they're revealing. Only Gemini 2.5 pro handles 1-hour-long videos.
Performance drops sharply with duration, proving that long video understanding is still challenging. We've found the breaking pointsโnow the community can start fixing them.๐
Want to learn more? TimeScope is 100% open-source. Benchmark your model and help us build the next generation of video AI.
๐ Blog:
https://huggingface.co/blog/timescope-video-lmm-benchmark
๐ฉโ๐ป Leaderboard & Demo: Apollo-LMMs/TimeScope
๐ Dataset: Apollo-LMMs/TimeScope
โ๏ธ Eval Code: https://github.com/EvolvingLMMs-Lab/lmms-eval
Today, we introduce TimeScope: The benchmark that separates true temporal understanding from marketing hype. Let's see how much VLMs really understand!โณ
We test three skills that matter for real-world use:
๐ Localized Retrieval: Find a specific action.
๐งฉ Information Synthesis: Piece together scattered clues.
๐ Fine-Grained Perception: Analyze detailed motion (e.g., count how many times a person swings an axe).
The results are in, and they're revealing. Only Gemini 2.5 pro handles 1-hour-long videos.
Performance drops sharply with duration, proving that long video understanding is still challenging. We've found the breaking pointsโnow the community can start fixing them.๐
Want to learn more? TimeScope is 100% open-source. Benchmark your model and help us build the next generation of video AI.
๐ Blog:
https://huggingface.co/blog/timescope-video-lmm-benchmark
๐ฉโ๐ป Leaderboard & Demo: Apollo-LMMs/TimeScope
๐ Dataset: Apollo-LMMs/TimeScope
โ๏ธ Eval Code: https://github.com/EvolvingLMMs-Lab/lmms-eval
Post
4066
๐ง ๐๏ธ Can AI visualize solutions?
Humans often solve visual problems by sketching ideas in our minds. What if Vision-Language Models (VLMs) could do something similar, not by generating full images, but by using internal โmental sketchesโ?
Thatโs the idea behind Mirage, a new framework that empowers VLMs to reason using latent visual tokens. Instead of just thinking in words, Mirage mixes in abstract visual representations that help the model solve complex tasks.
These aren't photorealistic images. They're compact, internal representations optimized purely to support reasoning.
๐ง Mirage is trained in two phases:
1) Grounding: It learns to produce latent tokens anchored in real images.
2) Refinement: The model drops the images and learns to generate visual tokens on its own.
๐ And yes, it works!
On challenging benchmarks like Visual Spatial Planning, Jigsaw puzzles, and Spatial Attention Tasks, Mirage clearly outperforms GPT-4o and other strong baselines.
Smart sketches > empty words.
By mimicking the way humans visualize solutions, Mirage gives AI a new kind of imagination, one thatโs faster, more efficient, and more human-like.
Kudos to the teams at UMass Amherst and MIT behind this exciting work.
Check the paper: Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens (2506.17218)
Humans often solve visual problems by sketching ideas in our minds. What if Vision-Language Models (VLMs) could do something similar, not by generating full images, but by using internal โmental sketchesโ?
Thatโs the idea behind Mirage, a new framework that empowers VLMs to reason using latent visual tokens. Instead of just thinking in words, Mirage mixes in abstract visual representations that help the model solve complex tasks.
These aren't photorealistic images. They're compact, internal representations optimized purely to support reasoning.
๐ง Mirage is trained in two phases:
1) Grounding: It learns to produce latent tokens anchored in real images.
2) Refinement: The model drops the images and learns to generate visual tokens on its own.
๐ And yes, it works!
On challenging benchmarks like Visual Spatial Planning, Jigsaw puzzles, and Spatial Attention Tasks, Mirage clearly outperforms GPT-4o and other strong baselines.
Smart sketches > empty words.
By mimicking the way humans visualize solutions, Mirage gives AI a new kind of imagination, one thatโs faster, more efficient, and more human-like.
Kudos to the teams at UMass Amherst and MIT behind this exciting work.
Check the paper: Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens (2506.17218)
-
- 4 replies
anditoย
authored
a
paper
7 months ago