Common Pile v0.1

Common Pile v0.1 is an 8TB pretraining corpus assembled by EleutherAI with around a dozen partner institutions across 30 source datasets, released June 2025. The distinguishing feature is license-cleanness: every text source in the corpus is either explicitly under a permissive license (Creative Commons, public domain) or has documented permission for redistribution and retraining. EleutherAI also trained two reference models from it (Comma 7B variants) to demonstrate the corpus is usable. Common Pile matters because the open-source AI conversation hinges on whether you can train a competitive model without relying on web-scraped content of ambiguous license status. Compared to FineWeb (15T tokens, but not license-clean) and The Pile (the original 800GB EleutherAI corpus, 2021), Common Pile is smaller but comes with a credible license claim. This directly addresses the OSAID v1.0 contested clause: under the strictest reading of "open source AI," only models trained on license-clean data can qualify. Production-readiness: shipping and used by the reference Comma 7B models, but no frontier-class model has trained on Common Pile at scale yet (as of this entry's update). The honest ceiling: 8TB is a reasonable substrate for 7-30B-scale models, but pushing to 70B+ would require either a much larger v1.0 or mixing with non-license-clean sources. Common Pile is the existence proof; whether it scales to frontier remains the open question.