How to raise your social business batting average

by Brian Kotlyar 10 Apr 2012 Blog Post

For years the battle in social media marketing has been between the evangelists, who happened to have no budget, and the skeptics, who controlled the brand marketing purse strings. Yet in recent months a transformation has occurred – social media now has a seat at the table in serious budget conversations. Teams are staffing up and brand marketers from companies like P&G are telling the world that their social and digital efforts are paying off.  All of this is great news for the early adopters that have been able to survive the transition from lonewolf to team player, but it also presents a new problem: you’ve got a budget, but what do you do with it?

Most brands will start by walking down the ‘best practices’ route. They will do most of the same things their competitors are doing and hope they deliver value for their company. The problem of course is that a) you are not the same as your competitors and b) who is to say that what your competitors are doing is even working for them? Just because they are throwing money at a bad idea, doesn’t make it best practice.

The answer to this challenge is that social business teams within large organizations need to develop a testing capability.

Attributes of a successful testing system

In honor of the newly started baseball season, we’ve outlined some important ways to think about a social business testing program in baseball terms:

  • Build a balanced line-up. A strong brand needs a player like Ichiro Suzuki and a player like Ryan Howard. This means a portfolio of tests consisting of activities that are low cost (and risk), but enjoy a high success rate and others with a higher cost (and risk), but offer significant business value.
  • Live with failure. It’s become cliche, but never forget: Hall of fame baseball players fail MOST of the time. The reason for this is simple: baseball players enjoy a huge number of attempts per season (400+) and the incremental cost of an attempt is extremely low. As a marketer you should pursue a similar strategy:
      • Test often – If you don’t have a test or experiment in market (even something small like call to action optimization in your content calendar), then you are missing out.
      • Test cheaply - Ask yourself this: what is the cheapest and fastest thing you can do to assess the value of an idea? Brands and agencies must learn to expend the minimum amount of resources possible to gain the necessary knowledge from a test. Note, this doesn’t mean social business testing is cheap – it just means that companies should not over commit to an unproven concept.
  • Measure the right things. Start by measuring everything you can, but understand that you should only be paying attention to the right things. Baseball is a numerical playground, but you need the right measures. If you just measure surface level data you miss out on underlying causes and themes. This is most famously demonstrated by the obsession with “eyeballs” during the .com boom/bust cycle or Derek Jeter’s fielding range in baseball.

Structuring a test

Assuming you are ready to embrace a testing regime for your marketing organization, here is a simple framework to get you started:

Define your objective:

Pick a major business problem your company is facing. There is a role for social business in fixing almost any of them, but you need a clear eyed picture of what your company needs out of its social programs. These will often break down into brand outcomes (e.g., increased awareness) or financial outcomes (e.g., increased customer lifetime value, reduced customer churn)

Define your test:

Design a structure that helps to generate and properly allocate testable ideas into various buckets. Don’t limit yourself to any particular set of ideas, but do categorize your ideas. A few frameworks to consider here are stages in funnel as compared to social platforms or stages in the consumer decision journey as compared to locations of social activity (local/social/mobile).

Pick a portfolio of tests ranging from easy to run/modest impact and complex/high impact. Be realistic between what you want to test and what you can feasibly get in market and manage. In a typical Fortune 500 company it’s hard to get more than 1 or 2 large scale tests in market in any given quarter.


Super sophisticated testing framework

Build a model:

There are any number of ways to model a social test. Just know this: you are going to be wading into a world of uncertainty and assumptions. The whole reason you are testing this notion is to remove the fog of war, so don’t be surprised if you are missing key data to feel 100% comfortable with your model. Nonetheless, I recommend a ground up approach to building out your model. Map out the entirety of the user experience for your test. Look to understand the motivations and patterns within that experience. Consider possible outcomes (while accounting for attrition). When you’re done you will have a wildly inaccurate picture of what is likely to occur during your test. That’s okay.

Create a learning plan:

This should consist of every piece of data you anticipate you will capture throughout the test and what you will learn from that data. A good example might be the e-mail capture rate on an A/B tested landing page. For this page you want to pre-define critical data elements you will learn. Some samples could include: Which landing page type is more effective at capturing e-mails? Will consumers provide an e-mail for this kind of call to action? At what rate will people provide an e-mail in response to this CTA? How will these e-mails perform once they are added to the prospect database?

Launch your test:

(this one is kind of self explanatory)

Measure and optimize:

Measurement options are typically just limited by your imagination and your tools. Web analytics are almost always involved, financial and ecommerce data can be critical. If paid media is involved you should definitely be monitoring ad performance. Finally, if brand outcomes are important to you, then social performance analytics are critical.


As the test is running use your data to optimize and improve wherever possible. Remember you are not in an academic research laboratory – your goals are commercial in nature. If something isn’t working and you know it, don’t throw good money after bad to preserve the test for posterity. Fix the problem.

Assess and recommend:

Populate your learning plan with the measurement data from the test. Review the outcomes and finally make a clear-eyed determination to end/optimize/roll-out the test. The vast majority of test will finish in the “end” state, but a subset will show promise and merit optimization. An even smaller subset will be ready for roll-out. When you land on a functional new tactic allocate budget to it and swing for the fences.

Creating a testing capability in your organization is hard. It often requires a major culture shift and some significant financial/reputational risks.  However, this is a critical capability in the social business era and you need to be thinking about.