Israeli startup develops an affordable alternative to GPT-3

Israeli startup AI21 Labs has developed the language model Jurassic-1 Jumbo, which has surpassed the competitive GPT-3 in the number of parameters and vocabulary size.

Today we’re launching AI21 Studio, a platform where you can instantly access our state-of-the-art language models to build your own applications — including a 178B parameters model, Jurassic-1 Jumbo. We can’t wait to see what you create! https://t.co/NhCnF1Emcy

— AI21 Labs (@AI21Labs) August 11, 2021

The company said that the largest version of the model contains 178 billion parameters. That is 3 billion more than GPT-3, though it significantly lags behind PanGu-Alpha from Huawei or Wu Dao 2.0.

Jurassic-1 can recognize 250,000 lexical elements, including expressions, words and phrases. This is five times more than other similar systems, according to the developers. 

The Jurassic-1 Jumbo vocabulary has also been among the first to cover ‘multiword’ elements, for example, ‘Empire State Building’. This means the model can have a richer semantic representation of concepts that make sense to people, the developers said.

“Expanding the boundaries of language-based artificial intelligence requires more than simple pattern recognition offered by current language models”, —said Yoav Shoham, CEO of AI21 Labs.

The company also said that their goal is to democratize the use of language models for everyone who wants them.

Currently the algorithm is in open beta testing and available to developers via API or an interactive web environment.

As reported in August OpenAI introduced the Codex platform for automatically writing code from textual prompts based on GPT-3.

In early August Microsoft developed the language model MEB with 135 billion parameters and integrated it into the Bing search engine.

In June the Beijing Academy of Artificial Intelligence presented the WuDao 2.0 model, ten times larger than GPT-3.

In January Google introduced the Switch Transformer language model, which contains 1.6 trillion parameters.

Subscribe to ForkLog news on Telegram: ForkLog AI — all the news from the AI world!