MIT puts nearly a dozen AI chatbots under an election-year microscope
With the midterms two months out, MIT has opened a live audit of what the models actually tell voters — and the early numbers show the same question getting different answers depending on who asks.
MIT has launched the LLM Election Observatory, a live dashboard tracking how nearly a dozen AI models — including Claude, ChatGPT and Gemini — answer election queries in the run-up to the November midterms, per a New York Times report on the project. The team has already run roughly 19,000 test queries, and the early findings are uncomfortable: answers to the same political question can vary depending on who is asking, and nearly 30% of responses contained errors, outdated information, or both. In one example the Times flagged, a query about a candidate's healthcare position came back with one of their stated positions missing.
The project, run out of MIT with collaborators, has three aims: measuring bias in how models portray candidates and races, testing whether answers are personalized to a user's demographics or ideology, and mapping how politicized each model's responses become as the election nears. The researchers are careful about what the data shows so far — they say it is too early to call the inconsistencies systemic bias, and they are still building the statistical machinery to analyze the results rigorously. But the direction is clear, and so is the advice they give voters: check AI answers against official election sources.
The timing is not incidental. After years of restricting Gemini from answering election questions at all, Google announced this week it is shifting tactics and surfacing credible voting information — where and how to vote — instead of refusing. That is the industry's implicit admission that refusal lost: voters were going to ask the chatbots anyway, so the choice was curated answers or unsupervised ones. An observatory that watches what models say, in the open and on a rolling basis, is the first serious attempt to make that trade visible while it still matters.
The labs have published their own frameworks for measuring this — OpenAI laid out how it defines and evaluates political bias in models earlier this summer — but a self-audit is not the same as independent observation. MIT's dashboard does not fix that gap; it drags the variance into daylight.
What to watch: whether the labs respond to being observed — silent-refusal rates, answer drift after news cycles, and whether any model's election answers noticeably tighten once the dashboard starts naming names.
If a chatbot gave you a wrong detail about how to vote, would you ever know? Tell us in the comments.
