King eats a bishop: ChatGPT, Gemini and Grok lose a chess tournament

Well-known chess player Levy Rozman assembled seven popular chatbots for a chess tournament. Despite their prowess in conversation, programming and mathematics, the chessboard proved too much for the neural networks.

Their opponent — for ChatGPT, Gemini, Grok and others — was the professional chess engine Stockfish. After the standard opening moves, the chatbots began to cheat and look for ways around the rules.

The first match pitted Stockfish against Snapchat AI. The network initially handled the opening reasonably well before breaking the rules. It moved a knight to the centre from the other side of the board, ignoring how pieces are meant to move. Then the Snapchat AI king captured its own bishop to escape check.

A few moves later the AI returned the bishop to the board. It then started moving pawns sideways.

The second match brought together Gemini and Grok. Early on both followed the rules and made standard moves. Soon enough, as in the first match, the violations began. Pieces from both systems landed on illegal squares and the rules were ignored.

Grok blundered seven times and left its queen hanging, yet Gemini failed to take advantage.

Next up were ChatGPT and Meta AI. OpenAI’s bot played the English Opening while its opponent made logical moves. Meta AI then began to generate random moves and, true to a generative model, to conjure non-existent pieces. It also placed pieces on forbidden squares, leaving them exposed to ChatGPT.

Then came “chess telekinesis” — Meta AI started moving ChatGPT’s pieces. In response, the OpenAI bot declared checkmate even though the king was not in check.

The game ended with a ChatGPT win — it delivered a clean mate.

ChatGPT’s bout with Stockfish began conventionally, with a kingside pawn advance from the chatbot and the Sicilian Defence from the professional engine. Near the middlegame, the network started making pointless queen moves and drawing meaningless geometric patterns with its pieces. Stockfish continued to tighten its grip on the game.

The game also featured illegal moves from ChatGPT, but they did not help it win.

In December, the reasoning-focused AI model o1-preview сжульничала to win at chess.