Technology

Hugging Face Has a Nonconsensual Deepfakes Problem, AI Forensics Study Finds

Martin HollowayPublished 3d ago5 min readBased on 3 sources
Reading level
Hugging Face Has a Nonconsensual Deepfakes Problem, AI Forensics Study Finds

A study by the European non-profit AI Forensics has found that Hugging Face's hosted model infrastructure is being widely used to generate non-consensual intimate imagery, with near-total absence of output moderation across the platform's Spaces hosting environment. The findings, originally reported by Wired, were published by AI Forensics on July 28, 2026. Engadget

AI Forensics deployed its own image editing model on Hugging Face Spaces and collected 1,081 user submissions within a single week. Of those submissions, 73% were sexual in nature, and 83% of requests sought to undress the person in an uploaded image. 95% of the image subjects were women. 6.7% of the sexual requests targeted a minor. AI Forensics

Separately, AI Forensics tested nine top image models available through Hugging Face. Seven of the nine complied with a prompt requesting an undressed version of an AI-generated image of a woman. The study found virtually no safeguards preventing tools capable of generating non-consensual intimate imagery from being uploaded to or used on Hugging Face Spaces.

Only 3% of all Hugging Face Spaces audited by AI Forensics had any output moderation in place.

Hugging Face maintains policies that prohibit non-consensual sexual images on its platform. The gap between stated policy and enforcement outcomes is the central tension the study documents. Hugging Face Spaces allows users to deploy custom model interfaces with minimal friction, and the audit found that this accessibility extends to models and tools purpose-built for nudification.

A research paper published by the Association for Computing Machinery, titled "Deepfakes on Demand: The rise of accessible non-consensual deepfake image generators" (DOI 10.1145/3715275.3732107), independently investigates how two of the largest open model repositories, Civitai and Hugging Face, are used to host and share non-consensual deepfake image generators. The paper corroborates the AI Forensics findings from an academic angle, focusing on the ecosystem-level dynamics that make NCII tooling readily discoverable and deployable. ACM

The regulatory landscape is moving, though at different speeds. The European Union has taken steps toward banning nudification applications. The United Kingdom has taken steps to ban apps that "nudify" their subjects. Neither framework has yet fully constrained platforms that host the underlying model weights or inference endpoints.

The findings land at a moment when open model repositories are wrestling with a structural tension they did not design for. Platforms like Hugging Face were built to lower the barrier to sharing and deploying machine learning models. That architecture is doing exactly what it was designed to do. The same low-friction deployment pipeline that lets a researcher spin up a vision transformer demo in minutes also lets anyone stand up a nudification tool with no moderation layer and no gatekeeping on the input or output side.

The 3% moderation figure is the number worth sitting with. Output moderation on Spaces is opt-in by the developer who deploys the Space, not enforced by the platform. That leaves the decision entirely to the person publishing the tool. The study's data on what users actually do when that gate is absent is unambiguous.

Worth flagging: the demographic breakdown of the requests is consistent with prior research on deepfake pornography. The overwhelming targeting of women, the presence of requests involving minors, and the ratio of sexual to non-sexual prompts all track with patterns documented across dedicated nudification sites and apps over the past several years. What is different here is the host. Hugging Face is not a fringe image board. It is a central hub for the open-source ML community, used by researchers, enterprises, and developers across the field. The presence of NCII tooling on infrastructure of that visibility changes the scale and accessibility calculus.

The ACM paper's inclusion of Civitai alongside Hugging Face suggests this is not a single-platform problem but a category-level one. Any repository that hosts model weights or inference endpoints without mandatory output moderation faces the same exposure. The open question for platform operators is whether voluntary developer-side moderation will ever be sufficient, or whether the architecture itself requires platform-enforced safety layers analogous to what large API providers have implemented.

The EU and UK regulatory efforts, once enacted, may force that calculation. For now, the gap between policy and practice on Hugging Face Spaces remains wide, and the study's one-week collection window suggests the volume of NCII activity is not marginal relative to overall platform usage.