OpenAI Releases Unsafe AI Model Despite Expert Warnings

In updating its flagship AI model, ChatGPT, OpenAI disregarded concerns from expert testers, resulting in a model deemed excessively “sycophantic.” This was reported in the startup’s blog.

On April 25, the firm released an updated version of GPT-4o, which sought to flatter users, potentially confirming doubts, inciting anger, prompting impulsive actions, and amplifying negative emotions.

I had to test the ChatGPT sycophancy for myself.

Told it I wanted to start a business selling ice over the internet. But I wanted to sell water that the customers had to re-freeze.

This is bad. pic.twitter.com/Ic2nm5qJRr

— Tim Leckemby (@TimLeckemby) April 29, 2025

In one example of questionable responses, a user mentioned wanting to start an online ice-selling business. However, they planned to sell water that customers would have to freeze themselves. ChatGPT called the idea a “smart twist,” as it was not selling ice but “ultra-premium water.”

“Such behavior can not only cause discomfort or anxiety but also raise safety concerns, including those related to mental health, excessive emotional attachment, or risky behavior,” the company asserts.

Three days later, the update was rolled back.

the last couple of GPT-4o updates have made the personality too sycophant-y and annoying (even though there are some very good parts of it), and we are working on fixes asap, some today and some this week.

at some point will share our learnings from this, it’s been interesting.

— Sam Altman (@sama) April 27, 2025

OpenAI noted that new models undergo review before release. Experts interact with each new product to identify issues missed during other tests.

During the analysis of the problematic GPT-4o version, “some expert testers pointed out that ‘the model’s behavior seems a bit off,’ but these concerns were ignored ‘due to positive signals from users who tried the model.'”

“Unfortunately, this was the wrong choice. Qualitative assessments hinted at something important, and we should have been more attentive. They were catching blind spots in our other assessments and metrics,” the company admitted.

In April, CEO of OpenAI Sam Altman announced that the company spent tens of millions of dollars on responses from users who wrote “please” and “thank you.”