Large language models show accuracy failures when prompted on climate information
A study testing 33 large language models found that all systems produced errors when evaluating evidence and answering prompts about climate science.
FTMQ Science, written by our newsroom2 views

Ars Technica Science and Ars Technica reported that researchers evaluated 33 large language models to test their performance on climate information. Large language models are machine learning algorithms trained on expansive text datasets for natural language processing and text generation tasks. [1][2][4]
The evaluation went beyond checking direct factual recall. According to Ars Technica Science, researchers explicitly prompted the models to determine if specific evidence supported or refuted particular statements, such as analyzing whether renewable energy investment eliminates jobs based on provided employment statistics. [1]
The tests revealed significant performance variations across different systems. Ars Technica Science reported that the top-performing models achieved approximately 70 percent accuracy when evaluating evidence, whereas the lowest-performing models scored below 40 percent accuracy. [1]
Two primary trends emerged across the tested categories. Ars Technica Science noted that newer frontier models performed better than older generations of the same system, and larger open-weight models consistently outperformed smaller open-weight models. [1]
In short
- Researchers evaluated 33 large language models on climate information.
- The best-performing models scored around 70 percent accuracy in evaluating evidence.
- The worst-performing models scored below 40 percent accuracy.
- Newer frontier models and larger open-weight models scored better than older or smaller counterparts.
Sources
Every paragraph above points to the numbered items it rests on. Read the originals here.
- [1]"You’re right"—LLMs will get climate info wrong if you askArs Technica Science, 1d ago (the report this story comes from)
- [2]"You’re right"—LLMs will get climate info wrong if you askArs Technica, 1d ago
Background
- [3]Political positions of Donald Trump on Wikipedia
- [4]Large language model on Wikipedia
Our newsroom writes these reports with the help of software, from the 4 sources listed and nothing else, and checks them against those sources. Facts can still be wrong or move on; the originals are the record. Spotted a mistake? Write to daniel@monsterkong.com.
Related from FTMQ Science
Earlier reports of ours on the same people and subjects.
- One-off training facts boost artificial intelligence discovery capacity in scientific researchTop Stories, 15d ago
- Neuroscientists report artificial intelligence speeds up brain research and model trainingTop Stories, 6d ago
- Self learning AI isolates particle jets in simulated ion collisionsPhysics, 6d ago
- Quantum physics models indicate theoretical possibility of repulsive gravityTop Stories, 15h ago
- Zapata Quantum Launches ForShor AI Verification Challenge for Quantum AlgorithmsTop Stories, 2d ago
More in New Research
- Autonomous AI agents generate hundreds of scientific preprints in twelve weeks15h ago
- Researchers identify over two thousand indoor arthropod species across North America15h ago
- Astronomers observe giant star swallowing stellar companion in Andromeda Galaxy15h ago
- Handheld Raman spectroscopy tested for ex vivo cervical lesion assessment1d ago
- Male mice gain defensive benefits from observing aggressive encounters1d ago
- Brain processes food rewards similarly to monetary gains study finds1d ago
Get the day in one email
Reports like this one, the top news of the last 24 hours, every morning. Free, one email a day; readers can comment under every report.
By signing up you agree to our terms and privacy policy. Unsubscribe any time.

Comments
Loading
Join the conversation
Comments are open to readers of our daily email: the top news of the last 24 hours, every morning, free. Sign up and the comment box opens.
Already on the list? Enter the same address and we will send a sign-in link.
By signing up you agree to our terms and privacy policy. Unsubscribe any time.