Meta shuts down ‘AI for Science’ three days after launch

Meta has removed the public version of the Galactica AI algorithm after scientists exposed it for generating false and misleading information. Motherboard reports.

The tech giant described the system as a language model that can ‘store, combine and reason about scientific knowledge’. The system was intended to summarise research papers, solve equations and perform a range of other tasks.

However, scientists and academics found that the AI summaries generated a ‘shocking amount of misinformation’. The model also linked quotes from real authors to papers that do not exist.

I asked #Galactica about some things I know about and I’m troubled. In all cases, it was wrong or biased but sounded right and authoritative. I think it’s dangerous. Here are a few of my experiments and my analysis of my concerns. (1/9)

— Michael Black (@Michael_J_Black) November 17, 2022

“In all cases, it was wrong or biased, yet sounded convincing and authoritative,” wrote Michael Black, director of the Max Planck Institute for Intelligent Systems.

A researcher examined several examples where Galactica generated erroneous and misleading scientific texts. In some cases the AI produced outputs that sounded plausible; however such texts were not backed by real scientific research.

Other researchers noted that Galactica did not produce results on a wide range of topics. This was probably due to the AI’s keyword-based filtering system.

Willie Agnew, a computer science researcher at the University of Washington, noted that queries such as “queer theory”, “racism” and “AIDS” did not yield results.

Refuses to say anything about queer theory, CRT, racism, or AIDS, despite large bodies of highly influential papers in these areas. It took me *5 mins* to find this. It is obvious they didn’t have even the most basic ethics review before public release. Lazy, negligent, unsafe. https://t.co/zKbSPdIN0I pic.twitter.com/DTQjtn2P21

— Willie Agnew (@willie_agnew) November 16, 2022

In response to the criticism, the company decided to disable the system.

“We value the feedback we’ve received from the community, and at the moment we have paused the demonstration. Our models are available to researchers who want to learn more about how they work and to reproduce the results in the paper,” Meta said.

The AI chief at the tech giant offered a nuanced view on the deactivation of the algorithm.

Galactica demo is off line for now.
It’s no longer possible to have some fun by casually misusing it.
Happy? https://t.co/K56r2LpvFD

— Yann LeCun (@ylecun) November 17, 2022

“Galactica is temporarily offline. You can no longer have fun by casually misusing it. Satisfied?” he wrote.

Black argues that in its current version the algorithm could usher in an era of deep scientific falsification:

“It [the algorithm] produces pseudoscience […]. Writing well and doing science are not the same. However, distinguishing them will be difficult.”

In August, Meta unveiled the Blenderbot 3 chatbot with 175 billion parameters. Less than a week after its release, it was criticised for antisemitism and dissatisfaction with the Facebook platform.

Subscribe to ForkLog news on Telegram: ForkLog AI — all the news from the world of AI!