Almost human: a major update to ChatGPT goes live

OpenAI released a global update for the ChatGPT chatbot, which has learned to “see, hear and speak.” The update marks an important step in the development of artificial intelligence that can perceive and process information in multiple formats, not just text.

«We are starting to implement voice and graphical capabilities in ChatGPT. They offer a new, more intuitive type of interface, allowing you to carry on a conversation with the neural network or show it the object of the discussion», — OpenAI explained.

Conversations with AI

The updated chatbot can hear and recognise users’ speech. Any request to the AI can be made by voice, which now resembles virtual assistants like Apple’s Siri.

To enable voice features, you need to turn them on in the app settings. ChatGPT offers a choice of five voices — “Juniper”, “Bay”, “Sky”, “Breeze” and “Charcoal”. They were voiced by professional actors.

For speech recognition, the neural network uses the open-source Whisper system.

«We are starting to implement voice and graphical capabilities in ChatGPT. They offer a new, more intuitive type of interface, allowing you to carry on a conversation with the neural network or show it the object of the discussion», — OpenAI explained.

Show and Tell

Users can also send ChatGPT various images in addition to ordinary prompts. The Vision or GPT-V feature helps the neural network provide more accurate answers.

As an example, developers cited a scenario where something needs fixing. The faulty area can be outlined with drawing tools to ease the chatbot’s task.

Image analysis is provided by multimodal GPT-3.5 and GPT-4. These models apply their language-thinking skills to a broad range of attachments: from screenshots and diagrams to ordinary photographs.

«Vision is intended to assist you in everyday life. The network performs best when it sees the same things as you. The approach is based directly on our work with Be My Eyes, a free mobile app for blind and visually impaired people, to understand the boundaries of use and limitations», — OpenAI representatives explained.

New capabilities — new risks

OpenAI’s overarching aim is to create a safe and beneficial artificial general intelligence (AGI). However, concerns about user protection have grown more pressing with the advent of these new features.

They warn that voice synthesis opens new avenues for fraud; criminals could create deepfakes impersonating famous people.

Visual models also pose problems: from misinterpreting images to making offensive comments about people in photos. Before launch, OpenAI tested the tool on a “red team” to assess extremism and inaccurate scientific statements.

«We have also taken technical measures to significantly limit the neural network’s ability to analyze and make direct statements about people, since ChatGPT is not always accurate, and these systems must respect privacy», — OpenAI stressed.

In July, developers released a new plugin for the chatbot, which can analyze data, generate Python code, build charts and solve mathematical problems. The neural networks managed to debunk the ‘Flat Earth’ theory.

In August, OpenAI launched ChatGPT Enterprise — a faster, more secure and powerful version of the chatbot for enterprise clients.