daVinci-origin-3B

📝 A note on naming: Across different versions of the paper, this model has appeared under slightly different names. The model released here as daVinci-origin-3B is referred to as S3B in some versions of the paper — they refer to the same artifact.

daVinci-origin-3B is a fully transparent, 3-billion parameter foundation model trained from scratch. It serves as a "clean-room" baseline for the research paper Data Darwinism -- Part1: Unlocking the Value of Scientific Data for Pre-training.

Unlike most open-source models, daVinci-origin-3B was explicitly trained on a dataset strictly excluding scientific content (books and research papers). This unique design allows researchers to unambiguously attribute performance gains to specific domain data injection strategies during continued pre-training.