HuggingFaceH4 (Hugging Face H4)

posted an update about 1 hour ago

Post

28

New REPL environment in OpenEnv available! ✨
Used in the Recursive Language Models (RLM) paper by Alex Zhang.

Ready for inference & post-training using trajectories. Handles long contexts:

> Run Python code in a sandbox
> Make recursive calls to LMs
> Explore data programmatically
> Return final result

Docs: https://meta-pytorch.org/OpenEnv/environments/repl/
Inference script: https://github.com/meta-pytorch/OpenEnv/blob/main/examples/repl_oolong_simple.py

sergiopaniego

posted an update about 24 hours ago

Post

167

Recursive Language Models (RLM) is a new interface for LLMs with cool ideas by Alex Zhang!

⚠️ LLMs struggle with long prompts → attention overload & lost info
🔄 RLMs inspect, split & call themselves on chunks, then aggregate results
✅ Handles millions of tokens, reduces noise, improves reasoning
💡 System prompt guides recursion
🎯 RLM trajectories can be used for RL training or distillation (OpenEnv+TRL!!)

We're adding it to OpenEnv (with Kashif Rasul): https://github.com/meta-pytorch/OpenEnv/pull/282

More resources:

> Paper: Recursive Language Models (2512.24601)
> Paper blog: https://alexzhang13.github.io/blog/2025/rlm/
> RLM repo: https://github.com/alexzhang13/rlm

1 reply

·

sergiopaniego

posted an update 5 days ago

Post

2106

New GRPO + TRL free Colab notebook out! 🔥

Fine-tune 7B+ models on T4 GPUs thanks to a ton of memory optimizations for GRPO

7B model uses only 9.2 GB VRAM (~7× reduction) 🤯

Try the notebook here 👉 https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/grpo_trl_lora_qlora.ipynb

sergiopaniego

posted an update 11 days ago

Post

2512

The list of hands-on notebooks (some beginner-friendly!) to get started with fine-tuning using TRL keeps growing!!

• SFT
• GRPO
• Tool calling & agents
• RL environments with OpenEnv
• LLMs and VLMs
✨ Many run on FREE Colab, making it super easy to get started fast!

https://github.com/huggingface/trl/tree/main/examples/notebooks

sergiopaniego

posted an update 14 days ago

Post

421

As the year comes to an end, it’s a good moment to catch up on some of the best long-form pieces published by the @huggingface team.

I’ve gathered them all here if you want to read or save them for later:
https://huggingface.co/collections/sergiopaniego/research-and-long-form-blog-posts

sergiopaniego

posted an update 15 days ago

Post

2251

This super detailed tutorial by @Paulescu is pure gold 🪙 "Fine-tuning a Small Language Model for browser control with GRPO and OpenEnv"

LFM2-350M ( @LiquidAI ) + BrowserGym (OpenEnv) + GRPO (TRL) for learning browser control 🤝

https://paulabartabajo.substack.com/p/fine-tuning-lfm2-350m-for-browser

qgallouedec

submitted a paper to Daily Papers 20 days ago

INTELLECT-3: Technical Report

Paper • 2512.16144 • Published 27 days ago • 16

sergiopaniego

posted an update 21 days ago

Post

453

if you’re on holidays 🎄 and want some reading, here are blogs I contributed to this year:

🦄 VLMs in TRL: https://huggingface.co/blog/trl-vlm-alignment
🦖 VLMs in 2025: https://huggingface.co/blog/vlms-2025
👾 tokenization in transformers v5: https://huggingface.co/blog/tokenizers
🛸 faster transformers: https://huggingface.co/blog/faster-transformers

sergiopaniego

posted an update 22 days ago

Post

1971

The Christmas holidays are here! 🎄
Thinking about learning something new in AI?

@huggingface offers 12 FREE courses covering all the relevant topics, for every level of experience. A great challenge for the holidays (and worth saving for later 🙄)

Let’s explore them!

🧠 𝗟𝗟𝗠 𝗖𝗼𝘂𝗿𝘀𝗲: large language models with HF tools
https://huggingface.co/learn/llm-course

🤖 𝗔𝗴𝗲𝗻𝘁𝘀 𝗖𝗼𝘂𝗿𝘀𝗲: build and deploy AI agents
https://huggingface.co/learn/agents-course

🎨 𝗗𝗶𝗳𝗳𝘂𝘀𝗶𝗼𝗻 𝗖𝗼𝘂𝗿𝘀𝗲: diffusion models with 🤗 Diffusers
https://huggingface.co/learn/diffusion-course

🔊 𝗔𝘂𝗱𝗶𝗼 𝗖𝗼𝘂𝗿𝘀𝗲: transformers for audio tasks
https://huggingface.co/learn/audio-course

🎮 𝗗𝗲𝗲𝗽 𝗥𝗟 𝗖𝗼𝘂𝗿𝘀𝗲: deep reinforcement learning
https://huggingface.co/learn/deep-rl-course

👁️ 𝗖𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝘆 𝗖𝗼𝗺𝗽𝘂𝘁𝗲𝗿 𝗩𝗶𝘀𝗶𝗼𝗻 𝗖𝗼𝘂𝗿𝘀𝗲: modern computer vision with HF
https://huggingface.co/learn/computer-vision-course

🦾 𝗥𝗼𝗯𝗼𝘁𝗶𝗰𝘀 𝗖𝗼𝘂𝗿𝘀𝗲 (𝗟𝗲𝗥𝗼𝗯𝗼𝘁): learning-based robotics
https://huggingface.co/learn/robotics-course

🧩 𝗠𝗖𝗣 𝗖𝗼𝘂𝗿𝘀𝗲: Model Context Protocol explained
https://huggingface.co/learn/mcp-course

🧪 𝗔 𝗦𝗺𝗼𝗹 𝗖𝗼𝘂𝗿𝘀𝗲: post-training AI models
https://huggingface.co/learn/a-smol-course

🕹️ 𝗠𝗟 𝗳𝗼𝗿 𝗚𝗮𝗺𝗲𝘀: AI in game development
https://huggingface.co/learn/ml-for-games-course

🧊 𝗠𝗟 𝗳𝗼𝗿 𝟯𝗗: machine learning for 3D data
https://huggingface.co/learn/ml-for-3d-course

📘 𝗢𝗽𝗲𝗻-𝗦𝗼𝘂𝗿𝗰𝗲 𝗔𝗜 𝗖𝗼𝗼𝗸𝗯𝗼𝗼𝗸: practical AI notebooks
https://huggingface.co/learn/cookbook

All of them can be found here: https://huggingface.co/learn

eliebak

submitted a paper to Daily Papers 26 days ago

SonicMoE: Accelerating MoE with IO and Tile-aware Optimizations

Paper • 2512.14080 • Published 29 days ago • 6

sergiopaniego

posted an update 26 days ago

Post

1863

Google DeepMind releases FunctionGemma, a 240M model specialized in 🔧 tool calling, built for fine-tuning

TRL has day-0 support. To celebrate, we’re sharing 2 new resources:

> Colab guide to fine-tune it for 🌐 browser control with BrowserGym OpenEnv
> Standalone training script

> Colab notebook: https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/grpo_functiongemma_browsergym_openenv.ipynb
> Training script: https://github.com/huggingface/trl/blob/main/examples/scripts/openenv/browsergym_llm.py (command to run it inside the script)
> More notebooks in TRL: https://huggingface.co/docs/trl/example_overview#notebooks

lewtun

in HuggingFaceH4/MATH-500 29 days ago

adds_eval_yaml

1

#5 opened about 1 month ago by

SaylorTwift

in HuggingFaceH4/MATH-500 29 days ago

adds_eval_yaml

1

#5 opened about 1 month ago by

SaylorTwift

sergiopaniego

posted an update 29 days ago

Post

497

best wrapped has arrived go get yours >
huggingface/2025-wrapped

sergiopaniego

posted an update about 1 month ago

Post

2120

🎄 last talk of the year about open AI and HF today at Universidad Rey Juan Carlos for undergrad students

always a pleasure to be back at my alma mater

🎅 slides: https://github.com/sergiopaniego/talks

1 reply

·

sergiopaniego

posted an update about 1 month ago

Post

1694

TRL now includes agent training support for GRPO‼️

Train 🕵️ agents with 🔧 tools, enabling interaction with external functions and APIs.

And of course, a new notebook and scripts to get you up to speed

📘 notebook tutorial: https://github.com/huggingface/trl/blob/main/examples/notebooks/grpo_agent.ipynb

📂 script examples: https://github.com/huggingface/trl/blob/main/examples/scripts/grpo_agent.py

📦 TRL v0.26.0 release: https://github.com/huggingface/trl/releases/tag/v0.26.0

2 replies

·

sergiopaniego

posted an update about 1 month ago

Post

2856

ICYMI, you can fine-tune open LLMs using Claude Code

just tell it:
“Fine-tune Qwen3-0.6B on open-r1/codeforces-cots”

and Claude submits a real training job on HF GPUs using TRL.

it handles everything:
> dataset validation
> GPU selection
> training + Trackio monitoring
> job submission + cost estimation
when it’s done, your model is on the Hub, ready to use

read more about the process: https://huggingface.co/blog/hf-skills-training

1 reply

·

sergiopaniego

posted an update about 1 month ago

Post

2281

We just released TRL v0.26.0!

It comes packed with updates:
> Agent training with tools in GRPO
> New CISPO & SAPO losses + reasoning rewards
> vLLM quantization in colocate mode
> Dataset shuffling in SFT
> Lots of NEW examples
> Tons of fixes and documentation improvements

3 replies

·

sergiopaniego

posted an update about 1 month ago

Post

2999

NEW: @EssentialAI just released Rnj-1, their first 8B model.

You can easily fine-tune it with GRPO using TRL to add reasoning capabilities to a compact mode

Free Colab link: https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/grpo_rnj_1_instruct.ipynb

More free TRL notebooks: https://huggingface.co/docs/trl/main/en/example_overview#notebooks

alozowski

authored a paper about 1 month ago

YourBench: Easy Custom Evaluation Sets for Everyone

Paper • 2504.01833 • Published Apr 2, 2025 • 22

Hugging Face H4

AI & ML interests

Recent Activity

INTELLECT-3: Technical Report

SonicMoE: Accelerating MoE with IO and Tile-aware Optimizations

adds_eval_yaml

adds_eval_yaml

YourBench: Easy Custom Evaluation Sets for Everyone

AI & ML interests

Recent Activity

Team members 42

HuggingFaceH4's activity

adds_eval_yaml

adds_eval_yaml