As misinformation proliferated during India’s recent four-day conflict with Pakistan, social media users sought verification from an AI chatbot, only to encounter further falsehoods. This incident underscored the unreliability of AI chatbots as a fact-checking tool, as reported by AFP. With tech platforms reducing their investment in human fact-checkers, users are increasingly turning to AI-powered chatbots—including xAI’s Grok, OpenAI’s ChatGPT, and Google’s Gemini—in their search for credible information.
“Hey @Grok, is this true?” has become a common inquiry on Elon Musk’s platform X, where the AI assistant is integrated, reflecting a growing trend of seeking immediate debunkings on social media. However, the responses provided by these AI models are often themselves riddled with misinformation. Grok, currently under renewed scrutiny for incorporating “white genocide”—a far-right conspiracy theory—into unrelated queries, incorrectly identified old video footage from Sudan’s Khartoum airport as a missile strike on Pakistan’s Nur Khan airbase during the recent conflict between India and Pakistan. Similarly, unrelated footage of a building on fire in Nepal was erroneously presented as “likely” depicting Pakistan’s military response to Indian strikes.
“The growing reliance on Grok as a fact-checker comes as X and other major tech companies have scaled back investments in human fact-checkers,” McKenzie Sadeghi, a researcher with the disinformation watchdog NewsGuard, told AFP. She warned, “Our research has repeatedly found that AI chatbots are not reliable sources for news and information, particularly when it comes to breaking news.”
‘Fabricated’
NewsGuard’s research revealed that 10 leading chatbots were prone to reiterating falsehoods, including Russian disinformation narratives and false or misleading claims related to the recent Australian election. In a recent study of eight AI search tools, the Tow Center for Digital Journalism at Columbia University found that chatbots were “generally bad at declining to answer questions they couldn’t answer accurately, offering incorrect or speculative answers instead.”
When AFP fact-checkers in Uruguay inquired with Gemini about an AI-generated image of a woman, the chatbot not only confirmed its authenticity but also fabricated details about her identity and the probable location where the image was taken. Grok recently labeled a purported video of a giant anaconda swimming in the Amazon River as “genuine,” even citing credible-sounding scientific expeditions to support its false claim. In reality, the video was AI-generated, as AFP fact-checkers in Latin America reported, noting that many users cited Grok’s assessment as evidence that the clip was real.
Such findings have raised significant concerns, especially as surveys indicate that online users are increasingly shifting from traditional search engines to AI chatbots for information gathering and verification. This shift also coincides with Meta’s announcement earlier this year that it was ending its third-party fact-checking program in the United States, entrusting the task of debunking falsehoods to ordinary users under a model known as “Community Notes,” popularized by X. Researchers have repeatedly questioned the effectiveness of “Community Notes” in combating falsehoods.
‘Biased Answers’
Human fact-checking has long been a contentious issue in a hyperpolarized political climate, particularly in the United States, where conservative advocates contend it suppresses free speech and censors right-wing content—a claim vehemently rejected by professional fact-checkers. AFP currently collaborates with Facebook’s fact-checking program in 26 languages, including in Asia, Latin America, and the European Union.
The quality and accuracy of AI chatbots can vary significantly, depending on their training and programming, prompting concerns that their output may be susceptible to political influence or control. Musk’s xAI recently attributed an “unauthorized modification” as the cause for Grok generating unsolicited posts referencing “white genocide” in South Africa. When AI expert David Caswell asked Grok who might have modified its system prompt, the chatbot identified Musk as the “most likely” culprit. Musk, the South African-born billionaire backer of President Donald Trump, has previously circulated the unfounded claim that South Africa’s leaders were “openly pushing for genocide” of white people.
“We have seen the way AI assistants can either fabricate results or give biased answers after human coders specifically change their instructions,” Angie Holan, director of the International Fact-Checking Network, told AFP. “I am especially concerned about the way Grok has mishandled requests concerning very sensitive matters after receiving instructions to provide pre-authorised answers.”
