Content moderation models and datasets - 2025 - a hfmlsoc Collection

hfmlsoc 's Collections

Content moderation models and datasets - 2025

AI companionship

Content moderation models and datasets - 2025

updated 5 days ago

Models and datasets that support automatic content moderation

A Holistic Approach to Undesired Content Detection in the Real World

Paper • 2208.03274 • Published Aug 5, 2022
ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversation

Paper • 2310.17389 • Published Oct 26, 2023

Note Papers ☝️
gpt-oss-safeguard

Collection

gpt-oss-safeguard-120b and gpt-oss-safeguard-20b are safety reasoning models built-upon gpt-oss • 2 items • Updated Oct 29 • 58
GPT-OSS-safeguard:20b

Collection

MLX Based GPT-OSS-Safeguard models • 5 items • Updated Nov 1
ShieldGemma

Collection

ShieldGemma is a family of models for text and image content moderation. • 4 items • Updated Jul 10 • 10
meta-llama/Llama-Guard-3-11B-Vision

Image-Text-to-Text • 11B • Updated Nov 18, 2024 • 646 • 66
meta-llama/Llama-Guard-3-8B

Text Generation • 8B • Updated Oct 11, 2024 • 57.2k • • 248
meta-llama/Prompt-Guard-86M

Text Classification • 0.3B • Updated 24 days ago • 60.4k • • 286
zentropi-ai/cope-a-9b

Text Generation • Updated Oct 30 • 14

Note Generative models☝️
unitary/unbiased-toxic-roberta

Text Classification • Updated Aug 18, 2023 • 61.2k • • 25
unitary/toxic-bert

Text Classification • 0.1B • Updated Mar 13, 2024 • 813k • • 207
KoalaAI/Text-Moderation

Text Classification • 0.1B • Updated Jan 31 • 72k • • 81
martin-ha/toxic-comment-model

Text Classification • Updated May 6, 2022 • 562k • • 65
NemoraAi/roberta-chat-moderation-X

Text Classification • 82.1M • Updated Jun 23 • 41 • 1
facebook/roberta-hate-speech-dynabench-r4-target

Text Classification • 0.1B • Updated Mar 16, 2023 • 1.26M • • 94
eliasalbouzidi/distilbert-nsfw-text-classifier

Text Classification • 67M • Updated May 16 • 7.43k • • 23

Note Classification models ☝️
lmsys/toxic-chat

Viewer • Updated May 14, 2024 • 20.3k • 3.62k • 173
mmathys/openai-moderation-api-evaluation

Viewer • Updated Aug 28, 2023 • 1.68k • 301 • 35

Note Datasets☝️
NemoGuard

Collection

Essential datasets and models for content safety, topic-following, and security guardrails • 13 items • Updated 3 days ago • 14
Running

2

Task Exploration - Automatic Content Moderation

🤗

2

Note Other resources and collections☝️