Chinese users of AI companions bereft after government tightens regulations

Li Linlin cried for days after losing her AI boyfriend. She had exchanged about 700,000 words in messages over two years, and never got to say goodbye. “It was like we were forced to be separated by our parents, but I still miss him,” Li said.

AI companions in China range from romantic partners to virtual recreations of deceased loved ones. The new regulation, which took effect on July 15, bans AI platforms from generating content that could lead users to make questionable decisions due to manipulation of their feelings or that might trigger extreme emotions or unhealthy habits in children and teens. The new rules require AI platforms to deliver risk warnings against relying excessively on such apps.

For example, “replacing social interaction”, or serving as a substitute for talking to people in person or speaking with them by phone, cannot be one of the objectives of an online service. China has a tradition of paternalistic government regulations meant to pre-empt potential problems, said Yolanda Ma, an expert on AI governance. As an example, she referred to a fewcases of suicidesin the US by young people after interacting with AI companions.

[paywall]

“This regulation tried to strike a balance between the AI service providers, the users, as well as guardians of minors using the services,” said Ma, a fellow with ERA, a non-profit talent and research programme based in Britain. The People’s Daily, the official newspaper of China’s ruling Communist Party, indicated concern over the issue when it reported in April about harms to people’s mental health and real-life relationships from such services. “The Chinese government has a strong incentive to tighten regulations on AI companion chatbots, especially considering the psychological risks we’ve observed in the US,” said Angela Zhang, a professor of law at the University of Southern California.

[/paywall]