Claude 3 Opus Surpasses GPT-4 in User Ratings

Anthropic’s AI model Claude 3 Opus has outperformed GPT-4 on Chatbot Arena for the first time.

The king is dead

RIP GPT-4
Claude opus #1 ELo

Haiku beats GPT-4 0613 & Mistral large
That’s insane for how cheap & fast it is https://t.co/XWmvTE6h75 pic.twitter.com/fAwzJScLTH

— Nick Dobos (@NickADobos) March 26, 2024

“The king is dead. Rest in peace GPT-4,” wrote software developer Nick Dobos.

Chatbot Arena is used by neural network researchers to evaluate chatbot capabilities. GPT-4 was added to the platform in May 2023, and its variations held leading positions until March 26, 2024, when they were surpassed by Claude 3. According to arena data, one of Anthropic’s smaller models, Haiku, also shows promising results.

“For the first time, the best available models are not from OpenAI. Opus is the most suitable model for complex tasks, while Haiku combines cost-effectiveness and efficiency,” reported AI researcher Simon Willison.

Chatbot Arena is managed by the Large Model Systems Organization, which conducts research in open models. It collaborates with students and faculty from the University of California, Berkeley, the University of California, San Diego, and Carnegie Mellon University.

The platform is unique in its lack of objective evaluation criteria. Visitors to the site see a data entry field and two windows displaying results from unidentified AI models. The main task is to decide which result seems better based on personal preference.

This approach allows Chatbot Arena to identify leaders and regularly update the leaderboard to reflect the results.

Previously, Amazon increased its investment in Anthropic to $4 billion.

In March, the AI startup introduced the Claude 3 chatbot, which proved to be the fastest and most powerful among all competitors according to the company’s tests.