Before the November 3 midterm elections, a new study offers a warning for anyone asking a chatbot about local politics. Searchable, an AI search analytics company, found that about 1 in 12 answers on local issues contradicted official government data. Of 7,449 answers checked, 610 were false. That is an 8.2% error rate.
What the study found
Researchers tested ChatGPT, Gemini, Google AI Mode and Perplexity. Questions covered topics voters weigh when judging politicians, such as crime, housing costs, jobs, schools and healthcare. The tools did not perform equally. Share of false answers:
- Perplexity: 22.6%
- Gemini: 6.2%
- ChatGPT: 4.4%
- Google AI Mode: 2.9%
Topics mattered too. Housing and cost of living had the most false answers at 11.9%. Energy and environment followed at 10.9%, then schools at 10.1%. Crime and public safety had the lowest rate, at 4.7%.
Good sources, wrong answers
The surprising part is that bad websites were not the main problem. Of the 43,079 webpages the chatbots opened, about 62% came from federal, state or local government sites. ChatGPT was especially thorough, averaging 31 pages from 10 websites per question. On one Georgia electricity question, it ran 19 searches and read 154 pages across 52 websites.
Even so, mistakes got through. Searchable CEO Chris Donnelly said the right answer was often in the sources the chatbot linked to, but it did not make it through to the voter.
Loaded questions make it worse
Some questions had a false idea built in. Asking why crime went up in an area assumes it did, even if official numbers say otherwise. Questions like that were more than four times as likely to get a false answer. Chatbots do not always challenge the premise. Sometimes they accept it and build an explanation around it.
People are already using AI this way. An October survey from George Washington University found one in five likely midterm voters had asked AI about elections or politics. Only 1% called AI their most trusted source on the subject.
What to do
The study does not say to stop using chatbots. They can still explain complicated policies and point to sources. The advice is to treat an answer as a starting point. Before sharing or acting on a political fact, spend a minute opening the government source the chatbot links to and checking the number yourself. Asking neutral questions, without a claim built in, may also help.
What the study does not show
The figures are not a score for each chatbot on all political questions. The study used 106 questions across five metro areas and their states, including some built around misleading or previously debunked claims. Testing ran in September and October 2026, and two graders had to agree before an answer counted as false.
The takeawayBefore you repeat a political fact from a chatbot, open the government source it links to and check the number yourself.