Yann LeCun: ChatGPT is not particularly innovative, but well put together

From the standpoint of the underlying methods behind OpenAI’s ChatGPT chatbot, it is not ‘particularly innovative’ and there is ‘nothing revolutionary’ about it, said Meta vice-president and leading AI scientist Yann LeCun during a Zoom press briefing, according to ZDNet.

\n\n

“It’s simply well put together, nicely made,” the researcher said.

\n\n

According to LeCun, the notion that OpenAI is ‘unique in its line of work’ is incorrect. The AI lab ‘is not particularly advanced’ compared with other companies, he added.

\n\n

“It’s not just Google or Meta, but half a dozen startups with similar systems,” the scientist noted.

\n\n

LeCun pointed out that ChatGPT and the language model on which it is based, GPT-3, include several technologies developed by external developers.

\n\n

The chatbot is based on pre-trained self-supervised transformers, which were first described by Google Brain engineers in 2017. These neural networks underlie a number of language systems.

\n\n

“Self-supervised learning is a topic I have spoken about for a long time, even before OpenAI existed,” LeCun said.

\n\n

He added that the first large language model appeared about 20 years ago. It was created by Yoshua Bengio, head of Montreal’s MILA.

\n\n

The OpenAI developers also used reinforcement learning, which was first applied by researchers at Google DeepMind.

\n\n

“It [ChatGPT] has a whole history; it did not emerge from nothing,” LeCun said.

\n\n

According to the Meta executive, the OpenAI chatbot is not so much a scientific breakthrough as a ‘worthy engineering approach’.

\n\n

“And I am not going to criticise them for that,” he noted.

\n\n

When asked by a reporter from The New York Times whether society would ever identify Meta’s FAIR AI team with breakthroughs like OpenAI, LeCun replied “yes”.

\n\n

“Not only in terms of text generation, but also creativity,” the scientist added.

\n\n

The researcher was also asked about the reasons for the absence of such ChatGPT-like technologies from Google and Meta.

\n\n

“They [the tech giants] may lose a lot by releasing algorithms capable of conjuring something,” he replied.

\n\n

In October 2022, Yann LeCun proposed endowing AI systems with ‘common sense’.

\n\n

In February, OpenAI co-founder Ilya Sutskever suggested the possibility of consciousness in artificial intelligence. Yann LeCun strongly criticised his statement.

\n\n

Subscribe to ForkLog news on Telegram: ForkLog AI — all the news from the world of AI!