Meta has introduced a new collection of AI systems, Llama 3.1, including the much-anticipated 405B, described as the “first open-source state-of-the-art model.” The Meta AI assistant has also been updated.
Starting today, open source is leading the way. Introducing Llama 3.1: Our most capable models yet.
Today we’re releasing a collection of new Llama 3.1 models including our long awaited 405B. These models deliver improved reasoning capabilities, a larger 128K token context… pic.twitter.com/1iKpBJuReD
— AI at Meta (@AIatMeta) July 23, 2024
Meta’s flagship AI model was trained on data from 15 trillion tokens using 16,000 Nvidia H100 graphics processors. It boasts 405 billion parameters, a term describing problem-solving skills. The larger the number, the better the performance. The system can perform tasks such as coding, answering basic math questions, or summarizing documents.
“Llama 3.1 405B is in a class of its own, offering unmatched flexibility, control, and cutting-edge capabilities that rival the best closed-source models,” the announcement states.
Llama 3.1 405B is text-only. Developers are experimenting with multimodality, but there are no public models for images, video, or speech yet.
The new Meta AI system has a larger context window compared to previous systems—128,000 tokens, equivalent to a 50-page book. A larger context allows for longer conversations without forgetting previously discussed topics.
Running Llama 3.1 405B requires powerful hardware. Alongside the flagship neural network, Meta introduced two smaller models, Llama 3.1 8B and Llama 3.1 70B. They also have a context window of 128,000 tokens.
Mark Zuckerberg Advocates for Open Source
In an open letter, Mark Zuckerberg endorsed the development of open-source artificial intelligence.
“Today, several tech companies are developing leading closed models. But open source is quickly closing the gap. Last year, Llama 2 could only compete with the older generation of models. This year, Llama 3 is competitive with the most advanced models and leads in some areas. We expect that from next year, future Llama models will be the most advanced in the industry. But even now, Llama already leads in openness, modifiability, and cost-effectiveness,” he noted.
According to the billionaire, open source will provide more people worldwide with access to the benefits and opportunities of AI, prevent power concentration in the hands of a few companies, and enable a more equitable and safe integration of technology into society.
New Features Added to Meta AI
Alongside the release of the new AI model, Meta AI, the AI-based assistant on Facebook, WhatsApp, Instagram, and Messenger, has been updated. It can now speak more languages and create stylized selfies.
The Imagine Yourself feature allows users to modify a photo of themselves using prompts. For example, one can take a selfie and request, “Show me surfing.” The service is currently in beta in the U.S.
New editing features will soon be added to Meta AI. Next month, an “Edit with AI” button will be available for additional settings.
The new Llama 3.1 405B model is integrated into the AI service, allowing users to direct questions to it.
Mark Zuckerberg’s Vision for AI Clones
In a new interview with Rowan Chung, Zuckerberg shared his vision of a future where content creators use AI-based bots for personal and corporate purposes. They will be able to delegate some community interaction tasks to these bots, freeing up time for other activities.
“I think there will be a huge breakthrough when every creator can train these systems to reflect their values and goals, and then people will interact with that,” noted the Meta co-founder.
Back in June, Zuckerberg’s company announced the addition of AI characters to Instagram.
In April, the corporation launched the Meta AI assistant across its platforms.