SAN FRANCISCO — San Francisco City Attorney David Chiu has sent letters to Apple and Google demanding they purge "nudify" apps that generate non-consensual deepfake pornography from their app stores, warning both companies could face civil penalties. Chiu says Apple and Google have made millions of dollars in fees from these apps and have been on notice for nearly a year.

Both companies have removed some apps in response. Apple terminated three developer accounts and Google suspended five cited Play Store apps. But the city argues much more action is needed under California law, and the letters represent the first time a major US city has directly targeted the app store operators themselves for profiting from AI-generated exploitation content.

Why it matters: This isn't just another tech regulation story. It is a direct assault on the app store business model. If San Francisco succeeds in holding Apple and Google legally liable for third-party AI abuse apps running on their platforms, it could set a sweeping precedent that reshapes Section 230 protections and forces every platform to proactively police AI-generated harmful content. For Apple and Google, the stakes are existential: if they're on the hook for every app that uses AI to break state laws, their 30% commission model becomes a massive liability, not a profit center.

The Letters: What Chiu Is Demanding

Chiu's letters, sent July 17, 2026, detail specific apps that he alleges violate California's laws against non-consensual pornography and deepfake exploitation. The apps, which use AI to generate realistic nude images from ordinary photos without the subject's consent, have proliferated across both the Apple App Store and Google Play Store throughout 2025 and 2026. Chiu's office has been investigating the ecosystem for months, identifying at least a dozen apps that remain available for download despite clear violations of both app store policies and state law.

The city attorney is demanding that Apple and Google do more than remove the specific apps identified in the letters. Chiu wants the companies to implement proactive screening systems that prevent similar apps from being published on their stores in the future. This includes requiring developers to certify that their apps do not generate non-consensual intimate imagery, implementing automated scanning for apps that use keywords or APIs associated with nudify functionality, and conducting periodic audits of apps in the AI image generation category.

"These companies have the technology and resources to stop this," Chiu said in a statement. "They choose not to because the revenue from these apps is too lucrative. Every time a nudify app is downloaded through their stores, Apple and Google take a cut. That makes them profiteers, not platforms."

The Revenue Numbers: Millions in App Store Fees

Chiu's office estimates that Apple and Google have collectively earned tens of millions of dollars in commission fees from nudify and deepfake apps since 2024. The apps typically charge $10-30 per month for subscriptions, with many converting free users to paid subscribers at rates exceeding 15%. Given the viral nature of these apps — some have been downloaded millions of times through promo campaigns on social media — the revenue flowing through app store billing systems is substantial.

Apple's 30% commission on all in-app purchases and subscriptions means that for every $20 monthly subscription sold through a nudify app, Apple keeps $6. With some of the largest apps reporting over 100,000 paid subscribers, the economics are significant. Google's 15-30% tiered commission adds another revenue stream. Combined, Chiu alleges the two companies have made more than $50 million from deepfake exploitation apps since the beginning of 2025.

The city attorney's office is also investigating whether these apps violate California's unfair competition law, which allows for civil penalties of up to $2,500 per violation. If applied to each individual download or subscription transaction, the potential liability could run into the hundreds of millions of dollars.

What Apple and Google Have Done — And What They Haven't

Both companies have taken some action. Apple confirmed it terminated three developer accounts associated with nudify apps and removed their apps from the App Store. Google suspended five apps identified in Chiu's letter and says it has "dedicated teams" reviewing app submissions for policy violations. But neither company has acknowledged systemic responsibility for the problem, and both have defended their review processes as industry-leading.

The reality is more complicated. App store review processes, despite significant investments, remain largely automated and reactive. Apple's App Store review team reviews approximately 100,000 submissions per week. Google's Play Store processes even more. Neither company has a dedicated screening process for apps that use generative AI to create intimate content, despite the category's rapid growth.

"The app stores are playing whack-a-mole," said Dr. Rebecca Huang, a technology policy researcher at Stanford. "They remove an app, the developer publishes it under a different name, and the cycle repeats. What Chiu is asking for is a fundamental redesign of the review process that makes it impossible for these apps to get through in the first place. That's a much harder engineering problem."

The Section 230 Question

Perhaps the most significant legal question raised by Chiu's action is whether Section 230 of the Communications Decency Act protects app store operators from liability for third-party apps that violate state law. Courts have generally interpreted Section 230 broadly to protect platforms from liability for user-generated content. But the question of whether an app store is a "platform" or a "publisher" has not been definitively settled in the context of AI-generated harmful content.

California law explicitly prohibits the distribution of non-consensual deepfake pornography, and state legislators have been pushing for stronger enforcement tools. Chiu's office believes that by collecting commissions from these apps and actively processing payments, Apple and Google are engaging in conduct that goes beyond passive platform hosting and enters the realm of active participation in illegal commerce.

If Chiu's interpretation prevails, it would represent a significant narrowing of Section 230 protections in the context of AI-generated harmful content. Every platform that processes payments, takes commissions, or exercises editorial control over app listings could face similar liability. The implications extend far beyond nudify apps to include AI voice cloning services, deepfake video generators, and other AI tools that can be used for harassment and exploitation.

What Happens Next

Chiu has given Apple and Google 30 days to respond to his letters and outline specific steps they will take to address the problem. If the companies' responses are deemed insufficient, Chiu's office has indicated it will pursue legal action, including filing a civil lawsuit seeking injunctive relief and financial penalties.

The case could take years to litigate, but the political and regulatory pressure is immediate. Other city attorneys and state attorneys general are watching closely, and similar letters could be filed in jurisdictions across the country. Federal legislators, including members of the Senate Judiciary Committee's Subcommittee on Privacy, Technology, and the Law, have already reached out to Chiu's office for information about the case.

For AI founders, the message is clear: the legal framework for AI-generated content is being rewritten in real time, and platform liability is the front line. The era of "we're just a platform" as a liability shield for AI abuse is ending. Whether through state enforcement actions like Chiu's or federal legislation that Congress is actively considering, the rules of the road are being set right now.

The question isn't whether regulation will come. It's whether your startup is prepared for it.