Technology

Apple Says No to Apps That Make Fake Nude Photos

Martin HollowayPublished 3w ago2 min readBased on 3 sources
Reading level
Apple Says No to Apps That Make Fake Nude Photos
source:apple.com

Apple has updated its App Store rules to clearly ban apps that use AI to make sexual images of people without their permission, according to The Verge.

The new wording says app makers "must proactively prevent and remove" these images. They cannot wait for complaints. They must build protection into the app, with checks on user requests and on the images the AI creates, whether the AI runs on the phone or on company servers.

The punishment is removal from the store. Apple says a developer can be removed for not having good enough safeguards, or for not fixing a problem after a warning. Either reason alone is enough.

Apple also clarified its rule against apps that change after review. Such apps have long been banned. The rule now clearly includes apps that add or change features, including "nudification," after approval. This covers cases where a harmless-looking app passes review and the harmful feature is switched on later by an internet update, a remote setting, or a changed AI model.

The rules also cover wider harm. Apple says apps that help with human trafficking or harm to children will be removed from the App Store and blocked from other Apple-controlled ways to install apps.

The change follows months of pressure and news reports. In April, Apple and Google still offered apps that let users make sexual images without consent, despite rules against them, according to Bloomberg. In June, protesters at Apple's WWDC event called on the company to remove Grok, X, and all nudify tools from the App Store. In July, San Francisco's city attorney ordered Apple to remove 13 nudify apps, and Apple said it removed them.

That July order was part of a wider demand. The San Francisco City Attorney's Office sent cease-and-desist letters to Apple and Google telling them to stop profiting from 13 face-swap apps, according to Wired. Apple did not remove X or Grok after X users used Grok to make sexual deepfakes without consent.

The broader context here is enforcement, not just wording. Reviewing an app is a one-time check. AI features keep running and can be changed from afar. A safety filter or AI model can be altered without a new app, which makes it hard for any app store to catch every problem later. Clearer rules give Apple a stronger basis for removal, but finding violations at scale remains hard.

In my view, this turns basic safety work into a store requirement. Keeping logs, offering a way to report abuse, removing harmful images fast, and tracking changes to models and prompts become standard duties. That helps honest makers of photo-editing, dating, and social apps, and raises the cost for those who try to dodge review.

Looking outside the store, App Store rules alone cannot cover everything. Alternative distribution, web apps, and downloaded AI models sit outside the same review. Apple will block those other paths for trafficking or harm to children, which helps in the worst cases. For other nonconsensual images, AI companies, app makers, and store operators must still work together. The rule is now clearer. How it is enforced is the test.