OpenAI’s ChatGPT has been pitted against other major AI chatbots in the market such as Google’s Gemini or Anthropic’s Claude. But what happens when ChatGPT goes head-to-head with a 46-year-old video game console in a game of chess.
That is exactly what an engineer recently set out to do. Robert Caruso, who works at cloud computing company Citrix, said that he designed an experiment in which ChatGPT squared off against Atari 2600, which was first released back in 1977.
Caruso said that he used a software emulator to set up run the 1979 Atari video game called Video Chess. The match between ChatGPT and the gaming system surprisingly did not go well for the AI chatbot, according to Caruso.
“ChatGPT got absolutely wrecked at the beginner level. It made enough blunders to get laughed out of a 3rd-grade chess club,” Caruso wrote in a LinkedIn post.
“Despite being given a baseline board layout to identify pieces, ChatGPT confused rooks for bishops, missed pawn forks and repeatedly lost track of where pieces were — first blaming the Atari icons as too abstract, then faring no better even after switching to standard chess notations,” he added.
He also said that the AI chatbot repeatedly requested to start the match over during the 90-minute contest.
While the experiment does not definitively indicate that ChatGPT lacks the intelligence to play chess, it seems to suggest that the AI chatbot cannot be used for that specific purpose as it is a chatbot that may be better suited to analyse or discuss chess moves.
Story continues below this ad
It is also unclear whether Caruso conducted the experiment with GPT-4o as the default large language model (LLM) or chose one of OpenAI’s reasoning models such as o1, o3, or o4-mini which are supposedly trained to solve complex problems by breaking them down into steps.
History of AI and chess
Experiments assessing AI systems based on their ability to defeat humans at chess is not entirely new. In 1997, IBM’s Deep Blue technology grabbed headlines by defeating chess grandmaster Garry Kasparov in several matches.
In 2016, Google DeepMind’s AlphaGo made history by becoming the first computer programme to defeat a world champion at the ancient Chinese game of Go. Following the launch of ChatGPT in 2022, a developer created a plugin called ChessGPT so that users could play chess with the chatbot.
However, a recent study by Palisade Research found that AI reasoning models such as OpenAI’s o1-preview and DeepSeek R1 don’t always concede when sensing defeat in a match against a skilled chess bot. Instead, the models that were tested opted to cheat by hacking its opponent so that the bot automatically forfeits the game.
© IE Online Media Services Pvt Ltd
