Brian Hawkins' Blog at Web Analytics Demystified

As I talk to more and more companies that are using testing solutions, I find many of them are unaware of the test techniques that are available in their testing platform.  Testing solutions available today offer more then just A/B and multivariate testing capabilities.  There are different techniques around multivariate tests but there are also other test techniques available that offer additional strategy for your tests.  Familiarizing yourself with the different techniques available will allow you to get much more value out of your testing solution and your optimization program.

Here I will share what test techniques or types that are currently available in Adobe’s Test&Target (T&T) platform. In T&T, tests types are referred to as campaigns and campaign types.  All the campaign types use the same core components such as the mbox and the offer.  The mbox, which is short for marketing box, does many things but for this topic, it is best to think of it as the area of real estate on the website that you wish to assign content as part of a test.  That content that you assign as part of the test is your offer. Campaigns are where you assign business rules to your mboxes and offers.

1:1 Campaign

The 1:1 campaign is a campaign type that is only available to those customers that have a Test&Target1:1 license.  Test&Target1:1 is the former Touch Clarity product acquired by Omniture and has since been incorporated into the Test&Target platform as a test type allowing users to leverage a shared profile and a single platform for their optimization efforts.

The 1:1 campaign is designed to leverage models to determine the right content to present to the individual vs. a segment of visitors.  These models are focussing on a single success event that you specify in the campaign setup.  These events can be anything that can happen in a session such as:  click through, form complete, purchase, revenue per visitor, etc…

There will be two branches of this type of test, similar to an A/B test.  The first branch serves as a control and is presented to 10% of traffic.  These visitors will randomly see any one of the offers you are using in the test.  The engine learns from this 10% of traffic by understanding how visitors react to the content and then correlating that reaction to the profile attributes of those visitors.

The other 90% of traffic benefits from this by receiving targeted content based off of the real time scoring the 1:1 engine provides.

I have seen this campaign type offer a ton of value to customers in highly trafficked pages such as the home page or main landing pages.  The big benefit here is the automation.  You set it up and let it do its thing with minor tweaking here and there.

This is what the summary report looks like in the 1:1 campaign type where you can see the two branches of the test:

The other key value that this test type provides is what is called an “Insights” report.  Yep, Adobe has an Insight product for analytics and also a report in 1:1 called Insights.  This Insights report in 1:1 provides data on what profile attributes of visitors are offer a positive and negative propensity against a given offer.  In other words, this report discovers segments or profile attributes that are impactful.    Here you can learn things like people on their 3rd session and are from California respond positively to a particular offer – hence discovering this segment for you and providing a marketing insight that can be used in other tests or in offline marketing!

1:1 Campaign Display

This campaign type is the exact same at the 1:1 except that it is used in display ads versus a website.

A/B..N Campaign

This is by far the most popular of the campaign types and, as you can imagine, it is the test type that allows you to compare two different experiences.  You can have just two experiences competing against each other but you can also incorporate as many different experiences as your traffic and creative permits.  Here is what the architecture of a standard A/B test looks like with two different offers being assigned to two different mboxes:

An important thing to note regarding the A/B test is that whatever experience or branch of the test the visitor falls into, they are stuck with the experience for the life of the campaign.  That is, if they continue to visit the area that is being tested, they will continue to see that test content until they convert which is defined as the primary success event in T&T.

Flash Campaign

This campaign type is used when you wish to test content within flash files.  This is a great technique to use if you wish to apply optimization to your display ads.  Onsite profiles collected by T&T can be used for quick and easy targeting with this type of campaign.

Adobe’s CS5 of Flash has productized the integration with T&T in that within CS5 of Flash, you can leverage a Flash Extension to quickly “mbox” components of the flash asset to be used as part of a test in T&T.  In T&T then you select the Flash campaign and during the campaign setup, you either upload the “mboxed” flash file or point to where it lives in the network.  T&T then identified the “mboxed” components where you can assign alternative content to it as part of the test.

The flash campaign follows the same technique as an A/B test in that visitors are stuck with whatever experience they were originally provided.

In the follow up post, I will highlight the Monitoring Campaign, Multivariate Campaigns and the Optimizing Campaign.

Posted Tuesday, April 10th, 2012 | No responses

In the Demand Generation world it is all about the “Known” and “Unknown”.  Before visitors fill out a form they are considered to be “Unknown”.  After supplying their information on a form, they are considered “Known”.  Increasing the percentage of visitors that fill out these forms adds a significant amount of value to organizations.

If you are unfamiliar with Demand Generation tools, they are often used to capture information from prospects, score leads, and send targeted emails to prospects, among other things.

Focusing on optimization techniques will allow you to increase the progress of “Unkowns” to “Knowns”, and progress to true personalization; where you know exactly what to show to each person on your site. To start, using the traffic source, environmental variables, online behaviors and geographic variables to target content will help discover what is the most effective content to present to visitors.  These types of visitor profile attributes should serve as the foundation of your visitors’ marketing profiles.  Then, add to them with offline data and contextual data to determine content to present to visitors as part of an optimization.

Let me walk you through two examples that have allowed companies to extend the value of their Demand Generation Platform by integrating it with their Optimization Platform.

The architected solution that is shown here leverages Adobe’s Test&Target and can applied to Demand Generation tools such as Aprimo, Eloqua, and Unica.  This model can be adapted to other platforms and technologies; these are simply the ones that I have helped customers execute and see value with in the past.

This first example highlights ways to increase the percentage of visitors that complete these forms.  Here we are optimizing to the Unknowns.

Optimizing the Unknowns

1.  An Unknown contact comes to the website and we want to increase the likelihood of them filling out the form.  The areas on the website and in the email in light red represent mboxes, which is short for marketing box and is the Test&Target code that is placed on the page.  This mbox does two key things in this exercise:  it allows for injection of content to target this visitor and it sets a unique visitor ID.

In order to increase the likelihood of visitors filling out the form and becoming Known, we have to be relevant.  We can target different product promotions, messaging, or content that is relative to the referral messaging to find out what is the most relevant content that leads to increased form completes.  Optimization allows us to not only understand what is the most effective content for form completion for the general population as a whole but also across segments.  For example, we may learn that SEM traffic should be presented with promotional messaging and visitors who are on their third visit should be presented with branded messaging as that increases their likelihood to fill out the form and convert.

2.  In this step a visitor has completed completed the form.  They supplied information such as title, organizational department, company size, industry and personal information such as email, name and address.  All if this information is helpful for the Demand Generation tool to manage this particular lead.  There are two additional data points that should be communicated to the Demand Generation tool as well that only the Optimization Platform can provide.

The first is the information on what targeted content was presented to this individual.  Consider how helpful it would be for the Account Manager or the Sales Person if they knew if a visitor was presented with promotional content versus branded content.  In cases where the company is presenting differentiating products, having that data tied to this lead is even more valuable.

The second data point to be passed to the Demand Generation tool is the Test&Target unique visitor ID.  This ID, when coupled with Demand Generation unique ID, allows for augmentation of the visitor profile attributes with offline data – something I address in the second example.

This communication of optimization Attributes happens programmatically behind the scenes as part of the integration.

3.  At this point, the visitor has made the progression from an Unknown to a Known.  The Optimization Platform provided the ability to determine what content was relevant and would lead to a higher percentage of form completes.  The Demand Generation tool has an increased amount of leads to manage that also have the rich test information provided by the Optimization Platform.

Optimizing the Knowns

1.  While optimizations are running targeting content to different segments of Unknown visitors, we can also simultaneously run optimizations to different segments of Known visitors.  Clients see incredible value doing this because they are continually being relevant by personalizing the site and email communication even after they have gotten the lead.

In step two in the first example, I pointed out that the optimization platform should communicate to the demand generation the Test&Target ID.  This ID is then coupled with the ID the Demand Generation tool manages.  As activity takes place offline such as phone calls or email communication, the profile of that lead gets richer.  Test&Target allows users to augment the online ID it creates with offline information such as sales cycle stage.  This is something that is accomplished programmatically when lead data is exported from the Demand Generation tool and then feed into Test&Target’s offline profile API.

2.  With all this rich information made available to the online profile, we now have the ability to target content in the same mboxes that were being used in the first example.  Using the Web Analytics Demystified’s website as an example, if this Known visitor came back to the site after filling out our form and then attending one of our ACCELERATE conferences, we may want to use the real estate in the mbox to promote our next ACCELERATE conference.  Anything that is known about this visitor can be used.  Another great example of the personalization capabilities here would be around targeting content based off of interest expressed offline.  Lets say a visitor came to the Demystified website and submitted their information for us to contact them.  During a phone call that we had with them, we found out that they were interested in the SiteCatalyst audits that Adam Greco provides.  We notate their interest in our Demand Generation tool.  That then gets pushed into their T&T profiles and upon subsequent visits to the site we can target content associated with Adam’s offerings that clients love.  This personalization or relevance can help progress this visitor into further engaging with Adam for his services.

3.  In this step, Test&Target is augmenting the Demand Generation tool’s profile by again communicating additional optimization data points as well as any recent website behavior.  This is very similar to Step two in the first example but in this case the visitor is already Known and may have visited other pages of the site or was presented with specific targeted content that is worth noting for the Sales Person or Account Executive.

So there you have it.  If you are using a Demand Generation tool and you are also using an optimization platform that supports the augmentation of the online ID with offline data, I highly recommend integrating the two.  There is incredible value in optimizing form completes and by continuing to be relevant to visitors even after they completed a form.  Because we are using an optimization tool to accomplish this, all the effort here is easily quantifiable to show the value and ROI.



Posted Monday, March 12th, 2012 | No responses

I am extremely excited to be joining Adam, Eric, and John here at Web Analytics Demystified.  While at Offermatica, Omniture and then Adobe I witnessed first hand the value this firm brings to clients and I am incredibly proud to be working with them.

A bit about me

I’m originally from Chicago but moved to San Francisco not long after finishing Graduate School in 2004.  Soon after arriving I met the fine folks at Offermatica and started my optimization career servicing clients large and small and across every industry.  It was during these years I found out how to best help organizations scale their optimization programs and, more importantly, how to get the most value out of their optimization platform.

In the years that followed at Omniture and then at Adobe, I spent most of my time continuing to help clients build best of breed optimization teams and practices.

These practices included auditing implementation, training, campaign road mapping, and the ever important process of analysis and communicating results to other parts of the organization.

During those years I also architected solutions for clients that allowed them to get more value out of their personalization and optimization platforms by integrating those platforms with their Analytics, Demand Generation tools, tag management systems, Email service providers and with other tools that share data internally and externally.  I am a strong believer that optimization and personalization should span across all marketing efforts and not just on the website; and integrating internal and external tools enables this.

Going forward

At Web Analytics Demystified, my hope is to continue helping clients in this manner as I see the incredible value it brings them.  I want to show them how to get the most out of their optimization and personalization tools.  While my background has been heavily focused around the Adobe technologies such Test&Target, Test&Target1:1, and Recommendations, I am already branching out and applying my skill set to other vendor tools.

I really look forward to sharing my insights here on this blog and I hope to hear from all of you here as well.  If I can be of any help to your business or you would like to chat, do not hesitate to let me know.   I can be reached via this page or by email at brian.hawkins@webanalyticsdemystified.com.

Posted Thursday, February 23rd, 2012 | 2 responses