Ai2

non-profit

Verified

https://allenai.org/

AI & ML interests

Building breatkthrough AI to solve the world's biggest problems.

Recent Activity

Ai2Comms published an article about 23 hours ago

StateBench: Testing how language models track state changes over long documents

Ai2Comms published a dataset about 23 hours ago

allenai/state-bench

chrisg-ai2 published a dataset 1 day ago

allenai/Dolci-Think-SFT-Clean-v2

View all activity

Papers

Old Ideas, Novel Problems: The Instability of LLM-Based Novelty Evaluation

Learning to Solve Hard Problems in RL for LLMs by Never Giving Up

View all Papers

Articles

StateBench: Testing how language models track state changes over long documents

about 23 hours ago

Can we trust LLMs to judge whether a research idea is novel?

LitPivot: Developing research ideas through dynamic contextualization within the literature landscape

OlmoEarth v1.3: State-of-the-art embeddings for the whole planet

FairStress: A deeper evaluation of fairness in LLM decision-making

HiRO-ACEv2: Simulating extreme weather at 3 km resolution for a fraction of the cost

Impactful scheduling for GPU clusters

Open-sourcing AstaBrief, the fast report-generation model in Asta

Introducing Olmo-core 3: Open, scalable training infrastructure for large MoEs

BenchMIRT: What are LLM benchmarks actually measuring?

Introducing OlmoEarth embeddings: Custom embedding exports from OlmoEarth Studio for downstream analysis

TutorMoments: Do AI tutors know when to help and when to hold back?

The OlmoEarth Platform: Geospatial inference at planetary scale

What building Shippy taught us about building agents

DiScoFormer: One transformer for density and score, across distributions

Which tokens does a hybrid model predict better?

MolmoMotion: Language-guided 3D motion forecasting

olmo-eval: An evaluation workbench for the model development loop

OlmoEarth v1.1: A more efficient family of Earth observation models

EMO: Pretraining mixture of experts for emergent modularity

View all articles

published an article about 23 hours ago

Article

StateBench: Testing how language models track state changes over long documents

allenai

•

about 23 hours ago

• 14

published a dataset about 23 hours ago

allenai/state-bench

Viewer • Updated 16 days ago • 68k • 381 • 3

published a dataset 1 day ago

allenai/Dolci-Think-SFT-Clean-v2

Viewer • Updated 1 day ago • 2.05M • 295 • 2

updated a dataset 1 day ago

allenai/Dolci-Think-SFT-Clean-v2

Viewer • Updated 1 day ago • 2.05M • 295 • 2

in allenai/IFBench_test 2 days ago

Add eval.yaml to register IFBench as a native Hub benchmark

#4 opened 7 days ago by

updated a dataset 3 days ago

allenai/asta-bench-submissions

Updated 3 days ago • 120 • 3

updated a dataset 3 days ago

allenai/asta-summary-citation-counts

Viewer • Updated 3 days ago • 129M • 1.72k • 12

updated a dataset 5 days ago

allenai/aimip-phase1-submissions

Viewer • Updated 5 days ago • 27 • 366

updated a dataset 6 days ago

allenai/fairstress

Viewer • Updated 6 days ago • 13M • 1.01k • 7

gabrieltsengai2

updated a dataset 7 days ago

allenai/olmoearth_pretrain_dataset

Preview • Updated 7 days ago • 4.52k • 21

published a model 7 days ago

allenai/OlmoEarth-v1_2-FT-LFMC-Base

Updated 9 days ago

updated 2 datasets 8 days ago

allenai/asta-bench-results

Viewer • Updated 8 days ago • 102 • 905 • 1

allenai/asta-bench-submissions

Updated 3 days ago • 120 • 3

published a dataset 8 days ago

allenai/XYBench-dataset

Updated 8 days ago • 33

published an article 8 days ago

Article

Can we trust LLMs to judge whether a research idea is novel?

allenai

•

8 days ago

• 18

updated a dataset 9 days ago

allenai/Dolci-Think-SFT-Clean-v1

Viewer • Updated 9 days ago • 3.93M • 899 • 2

published a dataset 9 days ago

allenai/Dolci-Think-SFT-Clean-v1

Viewer • Updated 9 days ago • 3.93M • 899 • 2

updated a model 9 days ago

allenai/OlmoEarth-v1_2-FT-LFMC-Base

Updated 9 days ago

published a dataset 9 days ago

allenai/novelty-judge-bench

Viewer • Updated 12 days ago • 3.68k • 512 • 4

in allenai/Dolci-Think-SFT-32B 11 days ago

Benchmark contamination: MATH test (205 items), ifeval_mt, IFBench, MMLU and GPQA overlap

#2 opened 11 days ago by