INTERNATIONAL RESEARCH / PEER-REVIEWED ARTICLE

Are AI messages as persuasive as human messages?

In three preregistered experiments, 4,829 people read policy messages written by AI and ordinary people. We explain the small shifts in attitude, why the messages were perceived differently and what remains unknown for marketing.

ACADEMIC WORK REVIEWED

LLM-generated messages can persuade humans on policy issues

ResearchersHui Bai, Jan G. Voelkel, Shane Muldowney, Johannes C. Eichstaedt, and Robb Willer.

Nature Communications · 1 July 2025 · 16:6037 · Peer-reviewed article · Source language: English

Open the original publication

ENGLISH REVIEW AND COMMENTARY: DIJITALPI

We explain the paper in plain English, discuss what it may mean for organisations and identify our own examples separately.

Read the explanation Go to 3 business examples

English review: 9 September 2026 · The source date appears in the citation above.

Published review

Source and editorial checks are complete. DijitalPi did not conduct a customer experiment for this paper.

LET'S READ THE RESEARCH TOGETHER

We first explain the researchers' question, method and findings. We then discuss how to interpret the results, clearly separating DijitalPi's commentary from the source.

01 / WHAT DID THE RESEARCHERS WANT TO UNDERSTAND?

Can one-way messages written by AI change opinions as much as messages written by ordinary people?

Bai and colleagues investigated whether open-access large language models could produce unique persuasive messages on different policy issues and whether these messages led to attitude change compared to human-written messages.

Explaining the context · DijitalPi commentary

This research examines a single message of approximately 200 words rather than a live chat. This distinction is important: the message does not respond to the reader's objection or adapts itself in conversation. What is measured is the change in opinion after reading a short text.

The interesting aspect from a marketing perspective is that AI can produce large numbers of messages quickly and cost-effectively. But policy support and product choice are not the same psychological and commercial context. The following business implications are not a direct finding of the research but a careful interpretation of practice.

02 / HOW WAS THE RESEARCH CONDUCTED?

How did they test the question?

Three preregistered online experiments were conducted in November and December 2022 with 4,829 US participants. The first two studies compared a GPT-3 or GPT-3.5 message, a message written by an ordinary person, a message that a person selected from five AI options and an unrelated neutral message. The third study compared an AI message with the control condition. Participants reported policy support on a 101-point scale before and after reading the message.

Understanding the method · DijitalPi commentary

The studies covered topics as diverse as a smoking ban in public places, an assault weapons ban, a carbon tax, a child tax credit, paid parental leave and automatic voter registration. Thus, the result did not stick to a single theme; however, the samples were in the United States and the messages were in English.

The human comparison group did not consist of professional copywriters or campaign consultants. Researchers used texts written by ordinary people. Therefore, it cannot be concluded that “AI is better than the expert writer”.

03 / RESEARCH FINDINGS

AI messages produced small but measurable attitude shifts.

Across three experiments, policy support among people who read an AI message shifted in the advocated direction relative to those who read a neutral message. The effects were approximately 2 to 4 points on the 101-point scale. In the first two studies, AI and layperson messages generally had persuasive effects of a similar size. Participants perceived the AI author as more logical, knowledgeable and calm, while they saw the human author as more original and vivid in storytelling.

Total participants: 4,829The combined sample across three preregistered US studies; not a single campaign or customer base.
Attitude change: 2–4 pointsApproximate effect on a 101-point policy-support scale; not a percentage increase in sales.

Source: Nature Communications: full article, methods and results of three experiments ↗

04 / DIJITALPI'S EXPLANATION

How should we interpret these findings?

The human criterion in the expression "as convincing as a human" must be read correctly. The comparison is with ordinary people who write messages of about 200 words with specific instructions. A professional creative team, strong branding or expert writer were not put to the test in this experiment.

The perception that AI and human texts have different strengths may offer a hypothesis for hybrid production. AI blueprints can be used in the knowledge and logic weave, authentic experience and human contribution can be used in the brand story. However, it should be tested separately whether this division of labor will produce better results.

It would be misleading either to dismiss the small effect or to turn it into a large commercial promise. At broad reach, a small shift in opinion can matter. The same scale can magnify harm from false or manipulative messages, so source checks and editorial responsibility remain essential.

05 / CONCLUSION AND OPEN QUESTIONS

What did we learn, and what do we still not know?

The research shows that open-access models used in 2022 can produce similar persuasive effects in English-language policy messages as ordinary people. The effects are small, the context is political, and the measure is short-term attitude support; purchasing, brand trust, or long-term behavior were not examined.

How do the contributions of AI and experienced human writers differ in Turkish brand communication? Answering this would require an experiment comparing texts under the same brief and offer, measuring actual behavior as well as preference.

Return to the researchers' original publication

Nature Communications: full article, methods and results of three experiments DOI and publication record

DIJITALPI'S APPLICATION COMMENTARY

What could this look like in your organisation?

We created these scenarios to make the topic concrete. They are not cases from the paper or measured client results.

EXAMPLE 01

Message that seems logical

The AI text contains more data and justification. The team thinks this will lead to higher conversions.

Does appearing reasonable prove sales results?

Open the recommendation for example 01

Measure perception and behavior separately.

You can evaluate the clarity and credibility of the message with a short survey. Keep click, form completion, and sales data separate. Do not report an increase in one metric as an actual result of another.

EXAMPLE 02

Originality of human text

In the experiment, AI text is found to be organized and knowledgeable, while human text is found to be more personal and original.

Would you apply a single production method to the entire campaign?

Open the recommendation for example 02

Establish a division of labor according to the task of the message.

Enhance the knowledge-intensive section with AI drafting, brand experience and real customer context with human editor. Check the accuracy and brand voice of the final version together. Measure the outcome of this hybrid method separately.

EXAMPLE 03

Small impact, big reach

The change in one person seems small, but the campaign can receive millions of impressions.

Do you consider small impact unimportant?

Open the recommendation for example 03

Consider impact size along with reach and risk.

Model the message's positive outcome alongside misinformation, misleading emphasis and reputational risk. Estimate how many people even a low error rate could affect at scale, then set the review threshold accordingly.

TRY IT WITH YOUR TEAM

Try the same message task through two productions.

Choose a single campaign message. Prepare the AI draft and the human draft with the same brief, length and proposal condition. Determine the evaluation criteria before viewing the texts.

Check out our human-controlled AI content process

Source and ownership

Hui Bai, Jan G. Voelkel, Shane Muldowney, Johannes C. Eichstaedt, and Robb Willer.
Stanford University; Political Belief Lab; Cornell University.

Nature Communications · 1 July 2025 · 16:6037 · Peer-reviewed article. Open the original publication

The academic work belongs to the researchers named above. This page contains DijitalPi's explanatory review and original business examples; it is not a full translation of the paper.

← Return to all research articles