3 ways we fool ourselves with social media metrics

This is part three of a series of posts about using social media metrics.

This is not the porn industry here, kids, and this is not about buying a house… SIZE DOESN’T MATTER!

– Steve Olenski, Too Many Social Media Marketers Still Believe Size Matters

An optical illusion: our brains add information that isn't there.
Would you believe the squares labeled “A” and “B” are identical shades of gray? Click the image for proof. (Developed by Edward H. Adelson of MIT.)

Don’t worry, it’s normal (statistically speaking) for people to fool themselves with statistics. So normal, in fact, that the fields of psychology and statistics can tell you exactly where things go wrong. Read on to find out how you and your organization can avoid being fooled by the Fundamental Attribution Error, Sampling Biases, and Information Cascade when you are evaluating social media metrics.

But first, what are we being fooled to believe? People would like to believe that more social media followers are better, more comments are better, more shares are better, etc. This might be, but isn’t necessarily, true. In fact the opposite may be true. Sometimes less is more. Consider the following hypotheticals, based loosely on real world examples:

  • What if an ultra portable laptop computer maker paid a large sum to teen pop star Jason Berber to promote its latest product, resulting in a huge surge in likes and inquiries from hipster teens but producing few additional sales because of its relatively high cost, while creating a negative brand image for adult business people, thus reducing purchases overall?
  • What if the H&S Black tax preparation service discovered it was able to increase its volume of social media engagement by paying to develop content for, and responding to inquiries from, people who posted and commented on complicated tax questions. However, by so doing they increased the number of followers who were hardcore do-it-yourself tax filers, not likely to become customers, while scaring away potential customers who found the technical discussions uninteresting or even aggravating?

In both cases because the organizations focused on the wrong people (at least from a profitability standpoint) the higher the engagement, the lower the return on investment, or ROI, where

      revenue generated - cost of generating revenue
ROI = ----------------------------------------------
            cost of generating revenue

Like any marketing program, it’s possible to spend more money on social media than one earns via social media. Equally importantly, one may earn money at a lower rate via social media spending than via advertising, PR, or other mechanisms. To determine whether one has a positive return on social media spending, and if so how much, one must measure both earning and spending via social media. But successfully increasing our “likes” and engagement in the abstract, without looking at revenue or cost, doesn’t tell us if we’re behaving profitably. Are we engaging with the right people? Or, more generally, is our engagement worthwhile from a business standpoint?

ROI is a classic, financially based formula used to answer that last question. Is this formula the only thing business people should ever use to make business decisions? No. See, for example, parts 1 and 2 of this series which explain why certain organizations should abandon social media metrics altogether and pursue business objectives for social media that don’t rely on quantitative measurements. However, this post is about organizations that are using metrics who risk fooling themselves into assuming they are seeing ROI when they’re not.

So why is it “normal” for us to make the fallacious assumption that more engagement is always better, no matter who we are actually engaging with? Here are three reasons:

Reason #1: The Fundamental Attribution Error

We naturally conclude that the bulk of people who “like”, or post to, our Facebook page are motivated by an affinity for our brand. In the field of psychology this is known as the fundamental attribution error: assuming that others act the way they do because of a personality trait, like loyalty, rather than acting on one-off situational motives. We assume the silent majority of those “liking” us are customers, or potential customers, or people who will speak well of us to customers. We impute that they are expressing “product loyalty” by engaging with us—even though we have no real knowledge of their motives. We conclude that more engagement = more affinity from customers and potential customers, and thus our number of customers and sales will increase as our engagement increases. We fail to recognize that there are all kinds of situational reasons for social media engagement that are unrelated to customer loyalty.

For example: from a store manager’s standpoint, when a customer walked in, found exactly what they wanted, and expressed gratitude, it was an unambiguous sign that customer loyalty had been built. From the customer’s standpoint, however, when they forgot to pack a tube of toothpaste before going on a trip, they were pleased to find their brand of toothpaste conveniently displayed in the airport shop across from their departure gate. But they are unlikely to visit the store again or even think of it.

The cold hard truth is that a lot of what happens online is like that. They just aren’t that into you. Even if they “like” you and “engage” with you they aren’t coming back for more. Ever.

It is certainly possible, even likely, that some of our social media followers are “loyal” people, and they seek us out for social media engagement because of their loyal dispositions. But like the toothpaste purchase, many social media actions are situational, one-off transactions. Maybe a friend asked them to like our page, or they reacted to a joke or a cute picture we posted that someone else shared with them, or they had a pet peeve to act on, or they wanted to enter a contest. The truth is many people aren’t demonstrating a persistent trait, like customer loyalty, when they engage with social media accounts, any more than they bond with a generic airport terminal shop where they once found toothpaste. In fact, the numbers show an abysmal commitment level from social media followers as a group: only 1% of followers “engage” via the page.

But they are still reading our posts, right? I haven’t seen any good numbers about how many followers actually read content posted by the average brand, but given Facebook’s “Edgerank” mechanism for filtering out certain posts from people’s streams, plus the fact that many people follow vastly more accounts than they have time to read (or even skim) all of the posts from, and the tendency of most of us to tune out “advertisements” without even realizing it, I’ll go out on a limb and guess that an extraordinarily small percentage of posts ever cross the perceptual threshold of the vast majority of followers.

Meanwhile, have you any idea how many social media followers out there are fake, like Twitter spambots or purchased followers on Facebook? Kind of makes competitive analysis more interesting when you think about it.

In sum, while it’s normal for us to assume that when we increase social media engagement we inexpensively produce more customers and sales, it’s a fallacious assumption. So what’s the harm in it? One possible downside is that if we take action based on this assumption in isolation, without validating its business value, it costs us something—we spend resources, if only our teams’ time and attention, on increasing engagement— but it may not be helping us achieve our business goals. Knowing this assumption is a fallacy, however, we can avoid fixating on social media metrics in a vacuum and shift our focus to the relationship between our social media activity and valid business goals like providing a brand introduction, brand recognition, improving the quality/reducing the cost of customer service, lead generation, revenue growth, or other meaningful performance indicators.

Reason #2: Sampling biases (non-representative or “accidental” sampling)

Objects may be closer than they appear
Objects may be closer…

Businesses often assume that the people who engage with their social media accounts are typical of their customer populations as a whole, or at least representative of their followers as a group. But this isn’t necessarily true. For example, in the hypothetical above the tax service assumed the people engaging with them were typical customers, and thus mistakenly tailored their messaging to appeal to those people.

In statistics this is called a sampling bias. Rather than taking direction from a representative sample of customers, or even of followers, when we take our cue from our engagement with just those few who self-select to engage with us we are relying on a non-scientific “accidental” sampling methodology. The success of our engagement with this self-selected group tells us little or nothing about our success in communicating with the larger population(s) we want to reach.

A widely recognized example of this problem is consumer opinions posted on sites like Yelp and Trip Advisor. Most ratings are high or low rather than average because they were posted by the people most motivated to post, not because most people reacted that way. Or as David Pogue says in a 2011 Scientific American piece: “You’re more likely to review something if you’re fired up about it, one way or another; the vast, quietly contented multitudes generally don’t bother.”

Contrast this with a stage performer who can tell the difference between loud applause, or boos, coming from just a few audience members, and the intensity of responses coming from an entire auditorium. Social media doesn’t deliver this sort of unfiltered feedback.

Once again, the volume of engagement our social media accounts achieve, all by itself, tells us little or nothing about the degree of success of our social media efforts from a business standpoint. It’s just telling us about our success with a self-selected group about which we may know little or nothing.

Reason #3: Information (or Availability) Cascade.

Does anybody remember that anti-drugs PSA from many years ago where the dealer says to the kid: “Come on! Everyone’s doing it.”?

We have to focus on engagement, without looking at business outcomes like revenues, if that’s the best we can do because . . .that’s what everyone else is doing. Right?

(Shades of the 2008 real estate market in this one, too. Everyone’s buying, so the market just keeps going up. Right?)

Psychologists dub this the information (or availability) cascade. The herd instinct is surprisingly strong in us humans, particularly when our bosses, our competitors, and everybody else is pointing in the same direction.

So what’s the remedy? I recommend that you figure out what works for your business, and do that, don’t mechanically follow the crowd when you know better, as you now know, having read this far. And if you aren’t in a position to calculate ROI but need a rationale for a vibrant social media strategy apart from meaningless metrics, see parts 1 and 2 in this series.

Whatever you do, build consensus in your organization around how social media serves your organization’s business goals without assuming that social media volume equates to profitability, unless or until you can measure that connection.

2 Replies to “3 ways we fool ourselves with social media metrics”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s