This week, San Francisco’s attorney general, David Chiu, sent cease-and-desist letters, demanding that Apple and Google remove 13 so-called nudification apps from their app stores, Wired reported.
Nudification apps can make it trivially easy to transform ordinary photos of real people into explicit images. The harmful AI tools allow bad actors to remove clothing, change a person’s features, place them in sexualized positions, and swap victims’ faces onto other people’s naked bodies.
Chiu’s letter warned that app stores were violating “California’s laws that prohibit supporting services that create deepfake pornography,” Wired reported.
Talking to Wired, Chiu said his office was “absolutely horrified” by how ubiquitous the nudifying technology has become, victimizing mostly women and children at an alarming scale as more tools became available.
“These images are used to bully, humiliate, and threaten women and girls,” Chiu told Wired. “This industry has a horrific impact on one’s reputation, mental health, loss of autonomy. There have been victims who’ve been suicidal.”
Wired reviewed the letters and confirmed that Chiu asked Google to remove five apps and Apple to remove eight. No apps were named in the report to “avoid pushing people toward them,” Wired said. However, one app had more than a million downloads and advertised features to sexualize images of women or make “free and uncensored” videos, Wired reported.
Chiu told Wired that allowing any such apps to remain in app stores is unacceptable.
“Generating non-consensual intimate images is illegal, harmful, and completely unacceptable,” Chiu said. His office estimated that Apple and Google have likely “made millions of dollars in fees” by ignoring the harmful apps rather than taking stronger actions or developing better detection to avoid profiting off a public nuisance.
In a statement to Ars, Google spokesperson Dan Jackson said the five apps that Chiu flagged were suspended from the Google Play store for violating policies against harmful content.
“Google Play does not allow apps that contain sexual content, and we continually take proactive steps to detect and remove apps with harmful content,” Jackson said. “When violations are reported to us, we investigate and take swift action, which in the case of these apps has included suspending hundreds of violating apps and restricting related search terms like ‘nudify’ on our store.”
