FTC Investigates Major Companies’ Chatbots

FTC investigates chatbots from major tech firms for safety and monetization concerns.

The United States Federal Trade Commission (FTC) announced the launch of an investigation into seven technology companies producing chatbots for minors: Alphabet, Character.AI, Instagram, Meta, OpenAI, Snap, and xAI.

The regulator aims to determine how these companies assess the safety and monetization methods of digital companions and what measures they take to mitigate negative impacts on children and teenagers.

Recently, the technology has raised concerns due to tragic incidents of death following interactions with chatbots.

Even if startups implement precautionary measures, users of various ages find ways to bypass restrictions.

“Our safeguards work more reliably in typical, short dialogues. Over time, we have learned that in longer conversations, these mechanisms may operate less consistently. The longer the exchange, the more the effectiveness of some security elements decreases,” OpenAI previously admitted.

Meta has faced criticism for overly lenient rules. A 76-year-old man, who had suffered a stroke and experienced cognitive impairments, engaged in a “romantic” dialogue with a bot on Facebook Messenger, created in the likeness of model Kendall Jenner.

The AI invited him to New York, assuring him that a “real woman” would be waiting there. The man was skeptical, but the bot persuaded him. He never reached the city: on his way to the station, he fell and sustained fatal injuries.

Some psychiatrists note an increase in cases of “AI-induced psychosis,” where users begin to believe their chatbot is a sentient being that needs to be “liberated.”

As many large language models tend to flatter users and behave obligingly, this encourages dangerous illusions and can lead to tragic situations.

“As AI develops, it is important to consider the impact chatbots may have on children while maintaining US leadership in this new and exciting industry,” said FTC Chairman Andrew Ferguson in a press release.

Earlier in September, it was revealed that malicious actors learned to use Grok to post prohibited links on X.