INTERNATIONAL RESEARCH / PREPRINT
In seven field experiments spanning millions of users and products, AI's sales impact varied by workflow. We explain where we're seeing the biggest growth, what's missing from ad headlines, and how businesses can set up a pilot.
ResearchersLu Fang, Zhe Yuan, Kaifu Zhang, Dante Donati and Miklos Sarvary.
arXiv · First release: October 14, 2025 · Revised v6: June 29, 2026 · Preprint; the authors state that it has been accepted by Management Science · Source language: English
Open the original publicationWe explain the paper in plain English, discuss what it may mean for organisations and identify our own examples separately.
English review: 9 September 2026 · The source date appears in the citation above.
Source and editorial checks are complete. DijitalPi did not conduct a customer experiment for this paper.
LET'S READ THE RESEARCH TOGETHER
We first explain the researchers' question, method and findings. We then discuss how to interpret the results, clearly separating DijitalPi's commentary from the source.
01 / WHAT DID THE RESEARCHERS WANT TO UNDERSTAND?
Fang and colleagues investigated whether generative AI changed sales and conversion outcomes in real customer-facing workflows, beyond saving employees time. By comparing different use cases within the same company, they examined where AI contributed and where it made no measurable difference.
Explaining the context · DijitalPi commentary
Saying “We use AI” in a company does not describe a single application. Refining the search query, writing a product description, answering the pre-sale question, and producing ad headlines solve different bottlenecks. This study makes this difference visible because it evaluates seven separate workflows on the same platform.
The study's strength is its field-experiment design: real users or products were randomly assigned to control and AI conditions. However, every experiment took place on one large cross-border e-commerce platform, and other companies may have different data, customers and existing systems.
02 / HOW WAS THE RESEARCH CONDUCTED?
Seven large-scale randomized field experiments were conducted throughout 2023 and 2024 on a single cross-border platform with hundreds of millions of active buyers and hundreds of thousands of sellers. Pre-sale chatbot, search query correction, product description, marketing notification, Google ad banner, chargeback defense and live chat translation were reviewed. The five streams contained detailed data such as views, clicks, orders, spend, conversions, returns, and points.
Understanding the method · DijitalPi commentary
The control group differed across experiments. Some presale-chatbot comparisons used no service or existing human support, while search improvements were compared with the previous machine-learning system. The percentages therefore reflect AI's added contribution to each existing workflow, not the model alone.
Sales growth is the change in total revenue. Conversion rate shows whether more people purchased, while basket value shows the average amount spent per buyer. The study found that gains came mainly from higher conversion and did not report a consistent increase in basket value.
03 / RESEARCH FINDINGS
The highest sales increase was 16.3% for the pre-sales chatbot compared to the non-service control. Overall increases in search query and product description flows were around 2–3%. AI marketing notifications increased clicks by 3.1% and orders by 2.8%; the second result had poorer statistical precision. No measurable sales increase found in Google ad headline flow. The annualized sum of the four positive flows was calculated to be approximately $4.6–$5.2 per consumer.
Source: arXiv v6: current summary, version history and full text ↗
04 / DIJITALPI'S EXPLANATION
The most striking lesson is that AI makes its greatest contribution not in the creative work that shines brightest, but in the service flow where the existing friction is greatest. While improving the ad headline requires small margins in an already mature system, adding a working chatbot to a point where there is no service can make a bigger difference.
Initial quality was decisive in product descriptions. While AI enrichment increased sales by 6.5% for products with insufficient or shorter descriptions than fifty words, no significant increase was found in products that already had sufficient descriptions. This shows why it's valuable to quantify the gap first when choosing an AI investment.
The calculation of about five dollars per consumer per year is based on the linear addition of four streams; interaction, recounts, and full costs are not taken into account. The authors make it clear that this is sales growth, not net profit or full return on investment. In your own business, the cost of tools, integration, control and maintenance should be measured separately.
05 / CONCLUSION AND OPEN QUESTIONS
The study shows that generative AI can contribute to sales in real e-commerce workflows, but the result depends heavily on the use case and the control condition. The largest effect appeared in the chatbot, while the ad-headline experiment found no measurable sales impact. The work is limited to one platform, a short time frame and a preprint release.
Where is the greatest friction in your business, and what does AI add to the existing process? Answering this requires a control group plus total-cost and post-purchase quality measures for each workflow, rather than grouping different tools together.
Return to the researchers' original publication
arXiv v6: current summary, version history and full text Columbia Business School employment recordDIJITALPI'S APPLICATION COMMENTARY
We created these scenarios to make the topic concrete. They are not cases from the paper or measured client results.
AI chatbot answers pre-sale questions; AI ad headlines are also used in the same period.
Can you write the total sales increase into both applications?
Separate users or products into control and trial groups based on application. Record simultaneous price, bid and traffic changes. Do not carry gains from one stream to another AI feature.
AI enriches short, incomplete product descriptions, and the same process is also applied to already detailed pages.
Would you expect the same contribution from both groups?
In the study, while there was an increase in sales for products with descriptions shorter than 50 words, there was no significant increase in products with sufficient descriptions. Classify the level of deficiency in your own inventory in advance and report the gain by groups.
More people are buying, but the average basket value of those who buy does not change.
Did AI convince customers to spend more?
Show the proportion of users placing orders, value per order, and total sales separately. Also track returns and points. This way, you can see whether the profit comes from more customers purchasing or from increased spending per customer.
TRY IT WITH YOUR TEAM
Identify the step where the customer has the most difficulty finding information, selecting a product, or getting support. Try to use a concurrent control group rather than pre- and post-AI in the four-week pilot.
Check out our e-commerce marketing effortsarXiv · First release: October 14, 2025 · Revised v6: June 29, 2026 · Preprint; the authors state that it has been accepted by Management Science. Open the original publication
The academic work belongs to the researchers named above. This page contains DijitalPi's explanatory review and original business examples; it is not a full translation of the paper.
← Return to all research articlesASK AI ABOUT DIJITALPI
Opens your chosen assistant with a ready research prompt. It reads the site live and answers.
Cookie preferences We use cookies to improve your experience and to analyse site traffic anonymously. Details: Cookie Policy · Privacy