China develops a language model ten times larger than GPT-3

The Beijing Academy of Artificial Intelligence introduced WuDao 2.0, a new language model. According to the developers, it surpasses Google’s and OpenAI’s similar technologies.

WuDao 2.0 uses 1.75 trillion parameters to model conversational speech, write texts and understand images. It operates in Chinese and English thanks to training on 4.9 terabytes of diverse data.

“These complex models, trained on vast datasets, require only a small amount of new information to launch a given function. … They can transfer knowledge to new tasks, like humans,” said AI researcher Blake Yang.

The developers said they beat Google’s record, which in January 2021 introduced Switch Transformer with 1.6 trillion parameters. Previously, the largest language model was OpenAI GPT-3 with 175 billion parameters.

Researchers also said they have already found 22 partners, including smartphone maker Xiaomi, delivery service Meituan and short-video platform Kuaishou.

Earlier in March, OpenAI said that the GPT-3 language model generates 4.5 billion words daily.

In January, OpenAI unveiled the GPT-3-based algorithm DALL-E, which generates realistic images of non-existent things from textual descriptions.

Subscribe to ForkLog news on Telegram: ForkLog AI — all the news from the world of AI!