OpenAI unveils ChatGPT, a chatbot designed for dialogue

OpenAI released the ChatGPT chatbot, which can answer questions, acknowledge mistakes, argue and reject inappropriate requests.

Try talking with ChatGPT, our new AI system which is optimized for dialogue. Your feedback will help us improve it. https://t.co/sHDm57g3Kr

— OpenAI (@OpenAI) November 30, 2022

The developers built the model using reinforcement learning from human feedback (RLHF). They used the same methods as InstructGPT, but with additional data from human dialogues.

To assemble the conversation dataset, the developers enlisted so-called trainers. They conducted conversations on behalf of a human and an AI assistant. Trainers also had access to model-generated proposals that helped them write answers.

For the reinforcement learning reward model, the team recorded conversations between the trainer and the chatbot. Then it randomly selected AI-generated responses and asked the instructors to rank them.

To improve the model’s accuracy, the developers employed proximal policy optimization. They performed several iterations of this process.

According to the developers, the model has a number of limitations. Despite plausible responses, ChatGPT often makes mistakes, is sensitive to wording, overuses certain phrases, demonstrates bias and infers context.

However, the company said that the experience deploying GPT-3 and Codex helped increase the model’s safety and reduce the number of unreliable answers thanks to RLHF.

ChatGPT available for free. The developers asked users to provide feedback on problematic outputs from the model, false triggers of the external content filter, which is part of the interface.

“We are particularly interested in feedback […], which helps us identify and understand new risks and possible measures to address them,” the press release says.

According to the developers, the information gathered will assist in developing more modern and advanced language models.

ChatGPT was trained on the Azure AI supercomputer. The chatbot is a refinement of the GPT-3.5 model, whose development was completed in early 2022.

In September, OpenAI unveiled speech-recognition system Whisper, enabling transcription in multiple languages.

In January the company released a new version of GPT-3, which produces fewer offensive expressions, disinformation and errors overall.

Subscribe to ForkLog AI on Telegram: ForkLog AI — all the news from the world of AI!