DeepMind’s AI programmer AlphaCode reaches human-level performance

The company DeepMind presented the results of testing the AlphaCode code-writing algorithm. According to the developers, the model reached the level of a “novice programmer”.

In @ScienceMagazine, we present #AlphaCode — the first AI system to write computer programs at a human level in competitions.

It placed in the top 54% of participants in coding contests by solving new and complex problems.

How does it work? 🧵 https://t.co/j427Nh16wO pic.twitter.com/Uqeyob6gy4

— DeepMind (@DeepMind) December 8, 2022

According to the statement, AlphaCode in a Codeforces competition was able to solve problems described in natural language that were previously unknown to it. The model predicted segments of code and generated millions of potential variants, from which it selected the ten most suitable.

According to the researchers, all solutions were generated “without any embedded knowledge of the structure of computer code”.

As a result, AlphaCode ranked among the top 54.3% of contest participants.

Under the contest rules, each developer was limited to ten attempts to find a solution. The researchers said their model succeeded on the first attempt in 66% of cases.

The developers noted that achieving success in coding competitions is incredibly difficult. First, the algorithm must understand complex programming problems described in natural language. Then it must learn to “reason” about unforeseen circumstances, rather than merely memorize code fragments.

According to the researchers, they did not find evidence that AlphaCode simply copied basic logic from training data. Taken together, these factors make the model’s performance a “big leap forward”, they added.

“AlphaCode handles coding tasks that are previously unknown to it very well, regardless of the extent to which it truly understands the task,” — wrote J. Zico Kolter, professor at the Bosch Center for AI at Carnegie Mellon University.

DeepMind is confident that in the future the algorithm will prove useful for developers. In addition to boosting overall productivity, AlphaCode could “make programming more accessible”.

Researchers also believe that at the highest level, the model could lead to a cultural shift in how code is written, with humans formulating problems and artificial intelligence solving them.

Earlier in February, DeepMind introduced an AI programmer AlphaCode.

In November the lab developed agents that interact with people naturally.

In December, DeepMind taught the AI to play Stratego at a professional level.

Subscribe to ForkLog news on Telegram: ForkLog AI — all news from the AI world!