Skip to the stories
Saturday, 21 November 2026
Saved

Next edition

7:38

Headlines

New Research

Large language models show accuracy failures when prompted on climate information

A study testing 33 large language models found that all systems produced errors when evaluating evidence and answering prompts about climate science.

FTMQ Science, written by our newsroom2 views

Share
Picture: Ars Technica Science

Ars Technica Science and Ars Technica reported that researchers evaluated 33 large language models to test their performance on climate information. Large language models are machine learning algorithms trained on expansive text datasets for natural language processing and text generation tasks. [1][2][4]

The evaluation went beyond checking direct factual recall. According to Ars Technica Science, researchers explicitly prompted the models to determine if specific evidence supported or refuted particular statements, such as analyzing whether renewable energy investment eliminates jobs based on provided employment statistics. [1]

The tests revealed significant performance variations across different systems. Ars Technica Science reported that the top-performing models achieved approximately 70 percent accuracy when evaluating evidence, whereas the lowest-performing models scored below 40 percent accuracy. [1]

Two primary trends emerged across the tested categories. Ars Technica Science noted that newer frontier models performed better than older generations of the same system, and larger open-weight models consistently outperformed smaller open-weight models. [1]

Share

In short

  • Researchers evaluated 33 large language models on climate information.
  • The best-performing models scored around 70 percent accuracy in evaluating evidence.
  • The worst-performing models scored below 40 percent accuracy.
  • Newer frontier models and larger open-weight models scored better than older or smaller counterparts.

Sources

Every paragraph above points to the numbered items it rests on. Read the originals here.

  1. [1]"You’re right"—LLMs will get climate info wrong if you askArs Technica Science, 1d ago (the report this story comes from)
  2. [2]"You’re right"—LLMs will get climate info wrong if you askArs Technica, 1d ago

Background

  1. [3]Political positions of Donald Trump on Wikipedia
  2. [4]Large language model on Wikipedia

Our newsroom writes these reports with the help of software, from the 4 sources listed and nothing else, and checks them against those sources. Facts can still be wrong or move on; the originals are the record. Spotted a mistake? Write to daniel@monsterkong.com.

Earlier reports of ours on the same people and subjects.

More in New Research

Get the day in one email

Reports like this one, the top news of the last 24 hours, every morning. Free, one email a day; readers can comment under every report.

By signing up you agree to our terms and privacy policy. Unsubscribe any time.

Comments

Loading

Join the conversation

Comments are open to readers of our daily email: the top news of the last 24 hours, every morning, free. Sign up and the comment box opens.

Already on the list? Enter the same address and we will send a sign-in link.

By signing up you agree to our terms and privacy policy. Unsubscribe any time.