A/B testing with Google Optimize and Google Analytics experiments

A lot of things have changed, specially with Google Analytics API and how Google wants us to create experiments with their platform.

To cut to the chase, Google Experiments has been shut down in favor of Google Optimize. This has direct impact on the way we were integrating A/B testing on Magnolia, since we were using Analytics Management API to programmatically create and modify test experiments and this is not possible anymore with Optimize.

But not everything is lost since we can still take advantage of Magnolia Personalization and page variants in order to manually (not automagically) create and manage redirect experiments with Google Optimize.

Let's review what ab testing is for:

"In ab testing you could improve your conversions by comparing different versions of the same page. The idea is to change key elements that might improve your conversion rate. You usually have a original page (so called control page) and a variation of it. The users will get any of these versions randomly and after a period of time you can compare which version did better according to a goal (or conversion)."

... and how can be implemented in Magnolia with Google Optimize and the same example as before.

We are going to work with the demo website travel, specifically with the "about" page which contains a video. The goal here would be get this video played more times, so the metric we are going to use will be the play event on this element of the dom.

The original page looks like this:

And the variation will have a wider video, removing the left side links which might distract the visitor and prevent him from playing the video (our goal in this case).

To create a variation of a page, you just have to select "Add page variant" in the pages app of Magnolia.

Open the variant created and update the video component:

After publishing the variant, you need to connect these pages to Google Analytics using a Javascript snippet for tracking page views. You can embed this snippet with Marketing Tags in Magnolia (don't forget to include it in all your pages):

After publishing the snippet (or marketing tag in Magnolia) you need to send an event for every time a user plays the video included in the pages we want to test, so we need to create another marketing tag for that:


<script>
$(".video-wrapper video").on('play',function(){
  ga('send','event','video','play');
});
</script>

After publishing the snippet you need to actually record the number of play video events as conversions or "goals" in Google Analytics:

After the goal is configured, you might want to test it in the real time tab of Google Analytics, so every time you play a video in your page, an event gets registered as a conversion:

Additionally in order to use Optimize, you need to install it as a Javascript Snippet, including your container ID, in a similar way to what you did with Google Analytics:

At this point you finally have all prepared to start your ab testing on the page variants measuring video played metrics as a goal with Google Optimize:

"A redirect test is a type of A/B test that allows you to test different web pages against each other. A redirect test contains different URLs for each variant. Redirect tests are useful when you want to test multiple different landing pages, or a complete redesign of a page."

So lets create a redirect experiment with the following steps:

Step 1

Create an experience of type "redirect test". Give it a name and provide the public URL of the page you want to test, in this case: http://yourserver/travel/about.html

Step 2

Add as variant of the test, the variant of the page in Magnolia. The trick here is to use the internal URL of the page variant, in this case: http://yourserver/travel/about/variants/variant-0

Step 3

Add as an objective the goal previously configured in Google Analytics, in this case, everytime a video is played:

Step 4

Finally start the experiment! (don't worry about installation checked we are handling it with the marketing tags)

And thats all! Yu can always check how your test is going in the reporting tab of the Optimize dashboard. Remember you have to wait at least 1 day in order to see results.

After you are done testing you can always pick the winner and publish it as the final page in Magnolia :)

About the Author

Edwin Guilbert Consulting Manager at Magnolia Spain

Edwin Guilbert is a Consulting Manager at Magnolia in Madrid, Spain. He leads the Services team of Spain and he also acts as a solutions architect. He joined Magnolia in 2014, having worked almost 10 years in consultancy as an architect and functional analyst. He regularly trains developers worldwide as he creates and conducts custom on-site workshops at client locations. He is also constantly involved in pre-sales work, carrying out demos and POC developments for pre-sales.