Monthly Archives: February 2016

2015 Testing Year in Review: 12 months of macro- and micro-yes experiments

Earlier this month, I brought you the first in a series of 2015 Testing Year in Review blog posts. For this next post, we’ll be looking at tests using both KPI macro-yes(s) (conversions) and micro-yes(s) (clickthrough).

Here are results of the 2015 tests we ran based on the primary test metric and level of confidence achieved. Anything under 95% level of confidence (LOC) is considered inconclusive, meaning the test did not have statistically significant findings. We want all the tests to be above 95% LOC because then we can say (with 95% confidence) that the changes we made are having the impact we want. The size of the boxes represents the range of LOCs, and the black lines are the midpoint, where half the data is above, and half below the line. The box represents the middle 50% of the data.

 

The big blue box shows the wide range of LOC’s for macro-yes tests. The base of the box is at 50%, and the black line (the median) is at 88%, indicating that less than half of these tests reach 95% level of confidence.

The small green box on the right is micro-yes tests. The range is high and tight, and the median is 97%! This is what we want. Half of these test are above 95% LOC, and the other half stay above 88%. Micro Yes tests are producing statistically significant, actionable insights about our audience.

Of course it’s much easier to increase clickthrough on a page than it is to increase the number of actual sales. The problem that these charts exposes is, less than half the macro-yes tests are reaching 95% LOC. Let’s look at it another way. In the charts below the level of confidence remains on the left, and the relative difference has been added in to show the direction and magnitude to provide insight into these parameters.

The macro-yes chart looks almost like a random scatter plot of data points with some aggregation at the top, where tests that reached 95% level of confidence start to accumulate.

The micro-yes test data points push to the top of the chart, forming a nearly solid line at the top. In addition, we see that line shoot off the right, indicating these test not only reached 95% LOC, but also had a large relative increase, a win-win! Are the macro-yes tests doing that?

Lastly we can look at the data by testing category. For macro-yes test we can see the same wide range of distributions among the various categories. Most of the boxes are big, with medians well below 95%. Some of the box plots drop all the way down to 30% LOC.

In the micro-yes tests we see straight away all the boxes are pushed high up on the chart. The median level of confidence is 97%! With these tests we are obtaining reliable, reproducible results we can act on with confidence.

 

Key takeaway

I often see tests that are optimizing a home page, landing page or some other area high up in the funnel, and tracking macro-yes as the KPI. Naturally we want to impact the bottom line with every change we make, but this approach to testing is not giving us the reproducible results we want.  We should always be tracking the macro-yes, but we don’t necessarily need to measure it as the primary metric. With every inconclusive macro-yes test, our time, effort, and resources are being wasted.

Meanwhile, micro-yes tests are getting the wins we want, without the attention they deserve. People walk toward a macro-yes one step at a time. Micro-yes tests are like the quiet gardeners who go about their work, clearing the path, pulling the weeds, consistently making small improvements that lead to big changes in the overall look and feel of your website. Let your Micro gardeners prune the tangled vines and overgrown branches. Allow them to provide a clear path for visitors to get to the places you want them to go. The data shows they are good at it, and reliable too.

 

You might also like

Conversion Rate Optimization: Building to the Ultimate Yes

Landing Page Optimization: 5 factors that lead to (and prevent) conversion

A/B Testing: Split tests are meaningless without the proper sample size

HPE Targets Exploding Mobile App Market with AppPulse Active

HPE Targets Exploding Mobile App Market with AppPulse Active

Keeping tabs on mobile app performance can feel like running a submarine. There’s a ton of dials to keep your eyes on, and all of them are monitoring mission-critical systems. If too many go haywire, you’re sunk. Hewlett Packard Enterprise (HPE) wants its new AppPulse Active  platform to be the

Continue reading...

Week in Review: Feng Shui Your Content + DX Summit 2016

Feng Shui Your Content Is it time to push content management to the C-suite — and apply principles of feng shui? Read more. Top Stories What got us talking this week: Customer Experience Google Warns: Optimize Your Mobile Content Save the Date - CMSWire’s DX Summit 2016 Google Says Goodbye to Right-Side Ads MailChimp's Mandrill Move Enrages Email Users Social Business

Continue reading...

Rainmaker Rewind: 12 Tips for Creating Calls to Action that Work on LinkedIn Pulse

Rainmaker.FM rewind

This week’s edition of Rainmaker Rewind features our own Sean Jackson and Mica Gadhia on The Missing Link.

The dynamic duo welcomed John Nemo, bestselling LinkedIn author and trainer, to discuss calls to action on LinkedIn Pulse.

missinglink-037

John Nemo is not only personable and an absolute pleasure to listen to, he also knows LinkedIn inside and out.

John is a bestselling LinkedIn author and trainer who helps small business owners, coaches, consultants, trainers, and sales and business development executives use LinkedIn to generate more sales leads, clients, and revenue for themselves.

He takes Sean and Mica through the step-by-step process of giving readers everything they need to take the action you want them to take.

Click here to listen and learn from The Missing Link.

Have time for a couple more? Check out these great episodes …

youpreneur-152

In this episode of Youpreneur, Chris and Brian talk about what it takes to be a digital entrepreneur and what that means for brand-building business owners.

Youpreneur:
The New Future of the Digital Entrepreneur, with Brian Clark

publish-050

In this episode of Hit Publish, Amy asks the question: “When it comes to marketing, what’s in a name? Does it really matter what you call your products and services?” Well … yes it does. And Hit Publish is here to help.

Hit Publish:
How to Generate Powerful Product Names That Don’t Sound Gimmicky

And one more thing …

If you want to get my Rainmaker Rewind pick of the week sent straight to your favorite podcast player, subscribe right here on Rainmaker.FM.

See you next week.

The post Rainmaker Rewind: 12 Tips for Creating Calls to Action that Work on LinkedIn Pulse appeared first on Copyblogger.

What Academia Can Teach Businesses About Collaboration

What Academia Can Teach Businesses About Collaboration

For obvious reasons, businesses are rarely inclined to work closely with their competitors. The fear that rivals will discover R&D secrets, poach customers or infringe on their intellectual property (IP) makes most companies highly suspicious of working with "the enemy."  Keep Your Friends Close and Your Enemies ...

Continue reading...

Doc Mgt Roll-Up: How Do You Like Your Living Documents?

Doc Mgt Roll-Up: How Do You Like Your Living Documents?

Document collaboration vendor Quip may not be a household name. But it's CEO and founder Bret Taylor probably is — at least indirectly. Taylor created the Facebook ‘Like’ button when he was CTO of the social network. He also co-created Google Maps and the Google Maps API.

Continue reading...

Doc Mgt Roll-Up: How Do You Like Your Living Documents?

Doc Mgt Roll-Up: How Do You Like Your Living Documents?

Document collaboration vendor Quip may not be a household name. But it's CEO and founder Bret Taylor probably is — at least indirectly. Taylor created the Facebook ‘Like’ button when he was CTO of the social network. He also co-created Google Maps and the Google Maps API.

Continue reading...