Blinded By Automated Dashboards

by Caroline Dangson 18 Jan 2011 Blog Post

In the winter, I am often forced to run on the treadmill.  I notice that while I run, I cannot pull my eyes away from the dashboard.  I’ll look away to focus on something else and before I know it I am back to watching the numbers rise on the screen.  I’ve seen other runners drape towels across the dashboard to avoid this preoccupation.  For me, the rising numbers are comforting.  Each second, the treadmill dashboard shows progress.  I’m mesmerized by the progress flashing on the treadmill. I’ve also noticed I’ll keep running until I have a nice even number – whether that’s a perfect 30 minutes or exactly 500 calories.  The numbers suddenly become more important than anything else.  But at the end of the day, these numbers do not guarantee that I’ve met my goals for improved health because they only capture the activity taking place for the half hour or so I spent on the treadmill.

Community dashboards are not unlike treadmill dashboards.  Both provide metrics as indicators of health.  The treadmill provides this with heart rate and calories burned.  The community platform provides this with number of active members, content contributors, etc. Dashboard metrics don’t lie, but they don’t tell the whole story. The treadmill does not factor calories consumed that day, hours slept, etc., just as community dashboards do not track off-site attitudes and behaviors of community members. Community managers can be blinded by the ease with which these numbers are provided by the dashboard and forget to track other important metrics. That’s why it is important that a measurement plan is developed in tandem with community strategy, not in reaction to the metrics generated by the community platform.

For companies launching marketing communities, I recommend the following measurement approach:

1. Start with the goals of the community

  • Establish a measurement plan based on focused goals for the community.  Is the goal conversion, customer retention, sentiment, etc.?
  • Avoid the trap of borrowing metrics from other marketing initiatives because they are familiar.

2. Identify the desired community behaviors to prove these goals and the associated platforms tracking these behaviors

  • Think broadly about desired community member behaviors (online and offline) that have implications for desired community outcomes.
  • Tracking these behaviors typically means pulling data from multiple sources including the community dashboard, web analytics platform, social media monitoring service, URL shortening service, CRM system as well as community member surveys.
  • Collaborating with colleagues from your analytics, customer insights, customer service, and IT departments will help you make sense of all of these data stacks.

3. Consider all initiatives that support community success

  • Keep in mind all of the paid and earned media initiatives (e.g., promotions and referrals) that will contribute to community success and consider integrating the associated metrics into a community scorecard.

4. Select the best metrics to tell the story of community progress

  • Now that you have considered all of the possible metrics for proving desired community outcomes, refocus the measurement plan with the right balance of metrics that tell the story.  That might translate to three key metrics to support each community goal, for example.

5. Establish a baseline

  • Determine if the metrics that are part of your community measurement plan were collected historically, or if you need to establish a baseline.  Do not establish specific numbered goals (e.g., 15% increase in positive sentiment) without knowing the baseline from which you are working.  Otherwise, you risk setting unrealistic goals.

6. Set specific targets to stay focused while allowing for some flexibility

  • Once you establish a baseline, set specific (realistic) monthly and quarterly targets to serve as motivators to keep you focused on your goals.
  • Maintain a flexible approach to your measurement plan as you discover what works and what doesn’t work for reporting purposes.

Capturing a holistic picture of community health and progress requires a tremendous amount of planning and integration.  Community dashboards provide only a piece of the puzzle.  This is why the marketer’s role involves both art and science.

  • http://martijnlinssen.com Martijn Linssen

    Fine post and points Caroline!

    http://www.martijnlinssen.com/2011/01/enterprise-microblogging-measuring-true.html is my own account of a classical metric-mess-up

    There are a lot of new tools on the market, and their numbers are increasing every day. Just as error handling, logging and maintenance are trailing, marketeers usually are a wee ahead of the product

    “We need metrics!” they say, and that’s what the devs give them

    It’s a question of maturity I think, and this market is in its infancy or at best growing into adolescence – thanks for being a good mom ;-)

    • http://dachisgroup.com Caroline Dangson

      Thanks Martijn. Your post that argues for relative numbers (vs. absolute numbers) in social measurement is right on. It illustrates my point that it’s up to the practitioner to add value to automated (or, as you put it, ‘absolute’) metrics. The timing of our posts is interesting and certainly signals a need for product maturity.

  • http://thebrandbuilder.wordpress.com Olivier Blanchard

    Outstanding. This post is a breath of fresh air. Bonus: I don’t think you used a single buzzword. Well played, Caroline.

    Cheers,

    Olivier

  • http://blog.sysomos.com 40deuce

    As a community manager who works for a company that makes one of the many dashboards I couldn’t agree more with you have said here. Great stuff.

    Cheers,
    Sheldon, community manager for Sysomos