Can last-click data mislead your advertising budget decisions?
Your advertising dashboard labels a campaign successful, and the team wants to increase its budget. The report credits actions after an ad click to that ad. A conversion is an action you measure, such as an order. But how many would have happened without the advertising? Incremental conversions are those that would not otherwise have occurred. PIE is a model that learns from earlier advertising experiments to predict additional conversions. In this preprint, which has not undergone peer review, researchers found that PIE predictions trained on experiments disagreed with experimental campaign success decisions less often than last-click measurement did. DijitalPi commentary: Before changing budgets, check whether the reported conversions are extra orders or actions that followed a click. The study evaluates measurement, not sales or revenue gains from adopting it. Its large-scale US Meta experiments do not establish equivalent accuracy for Türkiye or Turkish-language campaigns.
ACADEMIC WORK REVIEWED
Predicted Incrementality by Experimentation (PIE) for Ad Measurement
ResearchersBrett R. Gordon, Robert Moakler, Florian Zettelmeyer
arXiv · 1 April 2026 · v2 · Preprint · Source language: English
English review: 12 October 2026 · The source date appears in the citation above.
Editorial review draft
The source has been checked; the review still carries its original editorial status. DijitalPi did not conduct a customer experiment for this paper.
LET'S READ THE RESEARCH TOGETHER
We first explain the researchers' question, method and findings. We then discuss how to interpret the results, clearly separating DijitalPi's commentary from the source.
01 / WHAT DID THE RESEARCHERS WANT TO UNDERSTAND?
Can a model trained on other campaigns’ experiments predict advertising’s additional contribution better than last-click counts?
The researchers examine how to estimate incremental advertising effects when controlled experiments cannot be run for every campaign. They check how accurate the predictions are. They also compare campaign success decisions based on predictions with decisions based on experiments.
Explaining the context · DijitalPi commentary
DijitalPi commentary: Ranking campaigns in a budget meeting is different from deciding which generate sufficient additional contribution. Last-click reporting can capture the ranking while overstating the level of contribution. Its usefulness therefore depends on the measurement used to judge whether a campaign clears the success threshold, as well as which campaign appears ahead.
02 / HOW WAS THE RESEARCH CONDUCTED?
How did they test the question?
Gordon, Moakler and Zettelmeyer examined a randomly selected sample of large-scale experiments run by US advertisers between 1 November 2019 and 1 March 2020 using Meta’s Conversion Lift product. Ads ran across Facebook, Instagram and Audience Network. Inclusion required at least 1 million test-group users and 5,000 test-group conversions. The initial 839 experiments yielded 998 treatment-control pairs after separating experiments with multiple treatment groups. These pairs measured 2,226 conversion events, with each experiment-event pair counted as a randomised experiment; this does not mean 2,226 independent campaigns. Within each experiment, users in the target audience were randomly assigned to a test group eligible to see the advertising or an ineligible control group.
Understanding the method · DijitalPi commentary
A randomised controlled experiment estimates additional effects by comparing outcomes between groups eligible and ineligible to see the ads. The researchers used experimental results as reference outcomes for training and evaluating PIE. PIE combines many predictions that split the data using different conditions. Researchers call this method a random forest. Its inputs include campaign and advertiser characteristics, post-campaign test-group outcomes and last-click conversions over different windows; control-group data are not prediction inputs. The highest-performing model therefore does not rely solely on information available before a campaign starts.
The main outcome is incremental conversions per dollar: experimental estimates of additional conversions divided by campaign cost. The benchmark divides 7-day last-click conversions by spending and treats that figure directly as incremental conversions per dollar. Predictions are evaluated on data not used to train the model, with results weighted by campaign cost. R² is a measure of fit showing how closely predictions match experimental results; closer to 1 is better. It is not the percentage of campaigns classified correctly. The decision analysis compares whether each measurement places a campaign above or below the same success threshold, with thresholds ranging from 50–150% of the relevant campaign segment’s median.
03 / RESEARCH FINDINGS
PIE agreed with experimental success decisions more often than last-click measurement.
The researchers found that PIE-based success or failure decisions disagreed with experimental decisions in 8–12% of campaigns, compared with 12–20% for last-click measurement. At a success threshold equal to the relevant segment’s median, disagreement for lower-funnel campaigns, which track outcomes closer to purchase, was approximately 19.0% for last-click and approximately 12% for PIE. Last-click frequently classified experimentally unsuccessful campaigns as successful, while rarely classifying successful campaigns as unsuccessful; PIE’s two error types were lower and more balanced. For incremental conversions per dollar, out-of-sample R² was 0.88 for PIE using all features and 0.19 for raw 7-day last-click measurement. PIE using only pre-campaign features achieved R² of 0.35; adding the test group’s average outcome raised it to 0.46, and including 7-day last-click information raised it to 0.85. Last-click data were therefore not useless: they were a strong predictive input within a model trained on experiments.
PIE: out-of-sample R²
0.88R² is a measure of fit showing how closely predictions match experimental results; closer to 1 is better. Predictive performance for incremental conversions per dollar using all features; not a sales increase or percentage of correct decisions.
7-day last-click: out-of-sample R²
0.19R² is a measure of fit showing how closely predictions match experimental results; closer to 1 is better. Performance of the benchmark treating last-click conversions divided by spending directly as incremental effects, without model-based adjustment.
DijitalPi commentary: The practical lesson is to question which decisions last-click reporting supports rather than discard the data. Where a suitable experiment is available, compare the report and the incremental effect estimate against the same success threshold. Do not assume a few experiments will reproduce the paper’s predictive performance: training on 50 randomised experiments yielded R² of 0.37, compared with 0.72 using 400 randomised experiments. A small set of business tests cannot substitute for PIE trained on a broad experimental dataset.
Predictions weakened for advertisers absent from training: R² was 0.87 for existing advertisers and 0.72 for new ones. Calculating prediction inputs and outcomes from the same observations can also mechanically strengthen their association and inflate R²; pre-aggregated campaign data prevented splitting the sample to address this problem. The researchers could not compare the costs of the two decision error types. Fewer disagreements with experimental decisions do not establish higher business profits.
05 / CONCLUSION AND OPEN QUESTIONS
What did we learn, and what do we still not know?
In this dataset, predictions trained on experiments came closer than raw last-click measurement to experimental outcomes and experiment-based success decisions. The reviewed publication is the arXiv v2 preprint and has not undergone peer review; it also appears as NBER Working Paper 35044. Findings are limited to large-scale US Meta experiments and do not establish equivalent results for Türkiye, Turkish-language campaigns, other platforms or search advertising. The approach depends on representative training data and a stable relationship between features and outcomes over time; an implementing platform needs its own out-of-sample validation. Moakler works in Meta’s Ads Research team. Gordon and Zettelmeyer previously worked part-time at Facebook for 3 hours per week and now hold concurrent appointments at Northwestern and Amazon. The authors state that the work was performed at Northwestern and is not associated with Amazon. The paper also reports a production implementation at Amazon Advertising and a similar approach developed independently by Meta. No funding statement was found in the source.
Our open question at DijitalPi: would a campaign that clears your last-click success threshold still support the same budget decision when judged by its incremental conversion contribution?
We created these scenarios to make the topic concrete. They are not cases from the paper or measured client results.
EXAMPLE 01
E-commerce: deciding whether to increase spending
An illustrative e-commerce business is considering a larger campaign budget based on orders credited to the last click.
Are the reported orders sufficient grounds for increasing the budget?
Open the recommendation for example 01+
Separate orders credited to an ad from extra orders caused by that ad.
Set a threshold for incremental orders per unit of spending that makes sense for the business. For a suitable experiment, randomly assign users to groups that can or cannot see the ads. Compare the groups to estimate additional orders. Assess these estimates alongside campaign costs. The paper’s e-commerce finding does not establish equivalent accuracy for your own last-click report.
EXAMPLE 02
Retail: does the campaign clear the success threshold?
An illustrative retailer is deciding whether to continue a campaign that its dashboard describes as successful.
Does the campaign also qualify as successful when judged by its incremental contribution?
Open the recommendation for example 02+
Apply the same success threshold to the last-click report and the experimental estimate.
Keep the conversion event and evaluation period consistent throughout the comparison. Where an experiment is feasible, record whether last-click conversions and incremental conversions per unit of spending fall above or below the success threshold. Reconsider the decision if they disagree; this comparison alone does not determine the revenue effect of stopping the campaign.
EXAMPLE 03
Tourism: trusting the booking report
An illustrative tourism business interprets last-click bookings as additional demand generated by advertising.
How can we distinguish bookings that would have arrived anyway?
Open the recommendation for example 03+
Where feasible, measure additional bookings against a control group ineligible to see the ads.
Plan a test comparing booking outcomes for randomly assigned groups within the target audience. Use estimated additional bookings together with campaign spending to assess success. Do not apply the paper’s travel-sector ratio to your own bookings; predictive performance can change across periods and businesses.
TRY IT WITH YOUR TEAM
Check whether the campaign brings extra orders.
For one campaign, specify the conversion event, the incremental conversion threshold per unit of spending and whether a controlled experiment is feasible. If it is, plan to examine where last-click and experimental results lead to different budget decisions. This is a proposal that DijitalPi has not yet implemented.
Before testing whether advertising brings additional sales, check that your conversion records are accurate. We use GTMMACHINE to set up conversion tracking and audit its accuracy. An experiment measuring the additional effect of advertising must be designed separately. Explore GTMMACHINE ↗
Brett R. Gordon, Robert Moakler, Florian Zettelmeyer Brett R. Gordon — Kellogg School of Management, Northwestern University; Robert Moakler — Ads Research, Meta; Florian Zettelmeyer — Kellogg School of Management, Northwestern University, NBER.
The academic work belongs to the researchers named above. This page contains DijitalPi's explanatory review and original business examples; it is not a full translation of the paper.
Cookie preferences