Meta Unveils New Series of AI Models: Llama 4

Meta Corporation has released a new lineup of open AI models, Llama 4. According to internal tests, these models outperform competitors across various benchmarks.

The series is anchored by Llama 4 Behemoth, a large language model (LLM) with 2 trillion parameters. It is currently in the training phase and has not yet been released. Two of its multimodal distillations—Maverick and Scout—are available to developers and users.

Meta AI assistant, available in various company products like WhatsApp, Messenger, and Instagram, has already been updated to use Llama 4 in 40 countries. However, multimodal features are currently available only in the United States.

It is claimed that Behemoth, the LLM teacher of the other two models, surpasses GPT-4.5, Claude Sonnet 3.7, and Gemini 2.0 Pro in STEM-oriented benchmarks like MATH-500 and GPQA Diamond.

“This is just the beginning for the Llama 4 collection. We believe the most intelligent systems should be capable of performing generalized actions, naturally communicating with people, and solving complex tasks they have not encountered before. Empowering Llama with super capabilities in these areas will lead to the creation of better products for people on our platforms and expand developers’ ability to innovate in the next major consumer and business sectors,” the company announcement states.

New Architecture

Llama 4 is the first series of models to use the Mixture of Experts (MoE) architecture. Maverick has 128 “experts” and 400 billion total parameters, but only 17 billion are active. Scout’s figures are 16, 109 billion, and 17 billion, respectively.

According to the company’s internal tests, Maverick outperforms models like GPT-4o and Gemini 2.0 in some programming, reasoning, language support, long contexts, and image tests. However, the neural network falls short of the more powerful and modern Gemini 2.5 Pro from Google, Claude 3.7 Sonnet from Anthropic, and GPT-4.5 from OpenAI.

Maverick is better suited for use as a general assistant and chat. Scout’s strengths lie in document summarization and reasoning over large databases. The latter can operate on a single Nvidia H100 graphics processor, while Maverick requires a Nvidia H100 DGX system or its equivalent.

Controversy Surrounding Llama 4

Maverick secured second place in the LLM Arena—a test where people compare the performance of various models and form a “user” ranking.

Several researchers noted that a specially optimized version of Maverick, unavailable to developers, participated in the tests. The version for LLM Arena uses more emojis and provides unusually long responses.

Okay Llama 4 is def a littled cooked lol, what is this yap city pic.twitter.com/y3GvhbVz65

— Nathan Lambert (@natolambert) April 6, 2025

This makes it difficult for users to predict the real performance of the neural network in “everyday” conditions.

Denial

Meta’s Vice President for Generative Artificial Intelligence, Ahmad Al-Dahle, denied the information about model tuning for specific tests.

We’re glad to start getting Llama 4 in all your hands. We’re already hearing lots of great results people are getting with these models.

That said, we’re also hearing some reports of mixed quality across different services. Since we dropped the models as soon as they were…

— Ahmad Al-Dahle (@Ahmad_Al_Dahle) April 7, 2025

“This is simply not true, and we would never do such a thing,” he emphasized.

According to the executive, “the variable quality people are observing is due to the need to stabilize the implementation.”

“Since we released the models as soon as they were ready, we expect it will take a few days for all public deployments to be configured,” he added.

Back in November 2024, Meta opened its AI technologies to U.S. government agencies and defense contractors, as well as allies.

Earlier, it introduced Movie Gen—an AI generator for creating new videos, editing existing ones, and adding sound to them.