Hugging Face—Hugging Face hosted LAION-5B until child-safety researchers found CSAM material, prompting takedown
In December 2023 Stanford Internet Observatory researchers disclosed that the widely-used LAION-5B image-text training dataset hosted on Hugging Face contained at least 1,008 identified child sexual abuse material (CSAM) images, with researchers estimating likely thousands more. Hugging Face and LAION removed the dataset within days of disclosure. The episode highlighted gaps in pre-hosting safety screening for very large training datasets and prompted Hugging Face to expand its dataset-content policy enforcement.
Scoring Impact
| Topic | Direction | Relevance | Contribution |
|---|---|---|---|
| AI Safety | -against | primary | -1.00 |
| Child Safety | -against | primary | -1.00 |
| Content Moderation | -against | secondary | -0.50 |
| Overall incident score = | -0.553 | ||
Score = avg(topic contributions) × significance (high ×1.5) × confidence (0.59)× agency (reactive ×0.75)
Evidence (1 signal)
Stanford Internet Observatory found CSAM in LAION-5B dataset hosted on Hugging Face
The Stanford Internet Observatory's December 2023 report identified at least 1,008 confirmed CSAM images in the widely-used LAION-5B dataset, prompting takedown by LAION and Hugging Face within days. The report highlighted broader gaps in pre-hosting safety screening for large training datasets.