Experiments

A/B test your ad set-up on live traffic

Experiments is the A/B testing tool in Fusion. You duplicate an ad set-up (a tag config), change one thing, and split live traffic between the versions. Fusion measures each one on experiment RPM, alongside viewability, fill, latency and bid rates, and switches on the winner in one click. No developer and no custom code are needed.

How it works

From one idea to the winning config

  1. 01
    Duplicate a config and change one thing
    Copy the tag config you have live and change the setting you want to test: position, sizes, SSPs, timeout, floors or refresh. Leave the copy disabled; the experiment switches it on for its share of traffic.
  2. 02
    Split the traffic
    Give each config a share, for example 20% each, while the other 60% keeps your current set-up. Narrow the test by country, region, device, browser, operating system, screen size, audience or URL.
  3. 03
    Schedule it and let it run
    Set a start and end date, or no end date. Run for at least two weeks: new units and sizes are new inventory, and buyers take time to bid on them.
  4. 04
    Compare and activate the winner
    The report ranks configs on experiment RPM. Activate the winner and Fusion switches off the others and rebuilds your tag, live within 10 minutes. Or turn on auto-activate before it starts.
What you can test

Ad experiments publishers run in Fusion

Anything in a tag config or its ad stack can be tested: where ads go, which sizes and demand partners they use, and how they load.

Experiments publishers run in Fusion: the example, what to change in the copied config and what to watch
TestExampleWhat you changeWhat to watch
Ad positionIn-image ads "in" the image vs "under" itTargeting and position in the configExperiment RPM, viewability
Page layoutTwo MPUs spaced out vs one double MPUUnit type and placementExperiment RPM, viewability, fill
Ad sizesAdding creative sizes to a placement for more competitionUnit sizes for mobile, tablet and desktopExperiment RPM, fill. New sizes ramp up, so allow two weeks
Prebid timeoutA shorter or longer bidder timeoutTimeout in a detached copy of the ad stackAd latency, bid rate, fill
SSP line-upRemoving the weakest SSPsSSPs in a detached copy of the ad stackBid rate, win rate, latency. See SSP Insights
Bid throttlingThrottling on vs off, or different settingsBid throttling in the ad stackThrottle rate, bid rate, latency
Floor pricesAI Dynamic Floors vs your current floorsFloor model in the configExperiment RPM, fill
User ID solutionsWith and without an ID moduleUser ID modules in the ad stackExperiment RPM, bid rate
Lazy loadingLoading ads closer to or further from the viewportIn-view buffer in the configViewability, unit latency, experiment RPM
Running a fair test

Six rules for an ad experiment you can trust

Change one thing

The report lists the settings that differ between configs. Keep that list to one item and you know what caused the result.

Run for two weeks or more

New units and sizes start as new inventory, and buyers ramp up their bids over time. Weekday and weekend traffic also differ.

Test on a slice of traffic

Put 20% on each config and leave the rest on your current set-up. That traffic isn't measured, so a losing test costs little.

Test where it matters

Filter to the traffic you care about, such as UK mobile visitors, rather than averaging it with everything else.

Look past RPM

Experiment RPM picks the winner, but check viewability, fill and latency too. A small gain may not be worth a slower page.

Keep testing

Leave the end date blank and move the winner to 90%, keeping 10% on the other config in case seasonality changes the answer.

The experiment report

What the experiment report shows

Each experiment is Scheduled, Live or Completed, with a progress bar while it runs. Each config has its own colour, and the report lists the settings that differ between them.

Metrics in the Fusion experiment report and what each tells you
MetricWhat it tells you
Experiment RPMRevenue per 1,000 times each config was served in the experiment, normalised for its share of traffic, so an 80/20 split still compares fairly. This picks the winner.
Gross revenue (cumulative)Revenue added up day by day, adjusted to 100% of traffic so you can forecast. Toggle it off to see raw revenue.
Gross revenue (by day)How close or far apart the configs are each day.
ViewabilityViewable impressions as a share of impressions. Worth checking if advertisers buy on viewability.
FillAds loaded as a share of ad requests. Lower fill can still mean more revenue when floors rise.
Ad and unit latencyMedian time to load the ad and the unit, in milliseconds.
Bid rate, win rate, throttle rateHow SSPs responded to each config: bids per request, wins per bid and requests held back by bid throttling.
Heavy ad interventionThe share of ads Chrome removed for using too much CPU or data.
Data tableThe raw daily numbers, with filters, grouping and export. Save a view for other people on your team.
FAQ

A/B testing ads: questions, answered

How do publishers A/B test ads?
Split live traffic between two or more versions of your ad set-up and compare the revenue each earns per 1,000 ad loads over the same days. In Fusion you duplicate a tag config, change one setting, give each config a share of traffic and schedule the experiment. The report compares them on experiment RPM, viewability, fill and latency. Our experiments guide walks through a set-up.
Why not just compare this week with last week?
Because ad revenue moves with the season, the news cycle, the day of the week and advertiser budgets. A before-and-after comparison can't separate your change from those. A split test runs both versions on the same days and the same kind of visitors, so the difference is down to the change. See demand path optimisation and split testing.
How long should an ad experiment run?
At least two weeks. New ad units and new sizes count as new inventory, and buyers take time to start bidding on them properly, so the first days understate what a new config will earn. Two weeks also covers both weekday and weekend traffic.
What is experiment RPM?
Experiment RPM is the revenue per 1,000 times a config was served as part of the experiment, normalised for the config's share of traffic. That makes uneven splits, such as 80/20, directly comparable. It is the metric Fusion uses to pick the winner.
How does Fusion pick the winner?
On experiment RPM. The report shows a large green button when there's a clear winner and a small one when it's close. Activating the winner ends the experiment, switches on the winning config and switches off the others; your tag is rebuilt and live within 10 minutes. If viewability or fill matter to you, review them before activating. You can also turn on auto-activate, so the winner goes live when the experiment ends.
What traffic split should I use?
You don't have to put all traffic into the test. A common set-up gives two configs 20% each while 60% stays on your current set-up and isn't measured. If one config pulls ahead you can move more traffic to it without fully committing, and you can end an experiment early.
Can I test on only some visitors?
Yes. An experiment can be limited by country, region, device, browser, operating system, screen size (viewport), audience and URL, for example UK mobile visitors on article pages only.
Can I change an experiment once it is live?
Live experiments are locked so the data stays clean; only the end date can be changed. Fusion also warns you if you edit a config that is part of a live experiment.
Do experiments need a developer or changes to my site?
No. Configs are managed in Fusion and delivered through one tag, so a test is set up in the dashboard and your pages stay as they are. For your first one or two, our team will check the set-up for you.
Does it work with my own Google Ad Manager?
Yes. Fusion creates a Google Ad Manager unit for each config, which keeps reporting connected when two configs compete. If you map configs to your own existing units, experiments run through hybrid mode, where Fusion creates units only for the configs you leave unmapped. See Google Ad Manager.

Not sure what to test first?

Get a free Revenue Healthcheck: your ad set-up, header bidding and Core Web Vitals scored and benchmarked against similar publishers, with the changes worth testing.