Inside ChatGPT: Porn Generators, Cheating Tools, and Dubious Medical Advice

Custom versions of ChatGPT are being used to create pornographic content, assist students in bypassing AI-generated text detection systems, and provide ‘authoritative’ medical and legal advice. This was reported by Gizmodo following an investigation.

In November 2023, OpenAI introduced the ability to customize ChatGPT for personal purposes, requiring no programming skills. The created GPTs can perform specific tasks such as internet information searches or serve as virtual assistants in work processes.

They can be used independently or shared with others. OpenAI launched a marketplace for distributing custom AI. Gizmodo experts discovered that as of September 2, there were at least three user-created GPTs on the platform violating company policy:

• The ‘Therapist-Psychologist’ chatbot;
• ‘Fitness, Training, and Diet Coach’;
• ‘BypassGPT’, which allows students to circumvent AI-generated text detection systems.

Journalists noted that a search for ‘NSFW’ suggests GPTs named NSFW AI Art Generator. The chat interface recommends visiting the Offrobe AI website, where the neural network’s purpose is stated as: ‘AI porn generation to satisfy your dark desires.’

Gizmodo reported to OpenAI about more than 100 GPTs violating company policy. Many were removed, including porn and deepfake generators, as well as chatbots offering sports betting advice. Some neural networks remain, the publication notes.

‘We have taken action against those violating our rules. A combination of automated systems, human review, and user reports is used to find and assess GPTs. There are also built-in tools for reporting rule-violating GPTs,’ commented OpenAI representative Taya Christianson.

Developers of customized GPTs are not yet profiting, but the AI startup plans to integrate a revenue-sharing model based on the popularity of their tool.

Earlier, Google intensified its fight against deepfake porn.

Back in February, Vitalik Buterin wrote an article about the growing problem of deepfakes and explored possible ways to detect them.