There’s a chapter in the 1983 book Ogilvy On Advertising that I’ve always found amusing as an agency researcher. In this chapter, called “Jobs in Advertising”, Ogilvy goes through all of standard ad agency roles (i.e., planning, account, creative, media, etc.) and he systematically praises the virtues of each and the value that they bring to an agency and to clients. When it comes to agency researchers, however, he launches into a multi-page list of grievances about how research takes too long and how researchers favor precision and unnecessary levels of accuracy over quick, directionally correct, and actionable results. Thirty years later the grievances are, sadly, as relevant as ever.

If you’ve ever worked for or with an agency, you might already know a dirty secret about the research they carry out in advance of pitches or campaigns. The secret is that traditional focus group, survey, and panel-based research takes so long to carry out, synthesize and make sense of, these are often done in parallel with planning. If the research ends up supporting campaign ideas, the campaign is presented as if research drove it all along. If the research doesn’t support the campaign, it’s simply ignored.

This is all changing today. As the publicly available, historical treasure trove of conversational and behavioral online data grows, we’re relying less and less on slower, more expensive forms of marketing research. It’s taking my colleagues at WCG and me matter of days to come to derive the same types of insights that used to take weeks or months. And when we look beyond typical social media data use-cases, which are typically focused on social media marketing and community management, we’re identifying consumer segments, audience motivations/attitudes, and insights that inform a wide-range of product development, branding, creative, and other marketing activities that occur both on- and off-line.

Social Data: It’s Representational Now

The idea that content from social media networks might be a good replacement for marketing research isn’t a new one, but the general consensus among the research community has always been that it isn’t a very good one. A couple of years ago when Joan Lewis, the head of Consumer & Market Knowledge at P&G, announced to the Advertising Research Foundations’ ReThink 2011 conference that they’d be shifting some budget from survey to social media listening, the overwhelming response from the audience (and online) was: “you can’t do that because social media isn’t representative.” To paraphrase the non-believers, the folks who use social media are too young and too likely to only share very select parts of their day-to-day lives (i.e., really great experiences and product complaints). You can’t really understand the real marketplace by assessing the few anomalous people who are into the social media fad.

The “not representational” argument worked a few years ago when social networks were dominated by the young and social media-curious. But those arguments fall flat today. There is ample evidence that you can find just about any group sharing information about just about any topic online in 2013, including these statistics:

  1. 52% of Americans are actively using social networks (eMarketer, 2013)
  2. Adoption has increased across all major demographic groups; even 43% of Americans over 65 who are online, use social networking sites today (Pew Research Center, 2013)
  3. 24% percent of people who use social media, globally, report that they share “everything” or “most things” online (KPCB, 2013)
  4. The distinction between public and private is blurring for the next generation of consumers, meaning they actually will share anything.  53% of teens posted their email address online in 2012 and 20% shared their cell phone number (up from 29% and 2%, respectively, in 2006; Pew Research Center, 2012).
  5. Social media isn’t just a place to crack jokes about the Kardashians. Even when conversational analytics fail to deliver meaningful insights for a product or brand, people’s behavioral data is often very rich. For example, American’s on Facebook like an average of 70 pages (Socialbakers, 2013).

The Social Data Advantage

When you consider that social data is often representative, behavioral (not just conversational) in nature, often candid and can be sampled at the point of experience via mobile devices instead of retrospectively through questionnaires, there’s good reason to believe that it can serve the same purpose as traditional market research. When you add in the benefits of using social data, including the relatively low data collection costs, the ability to analyze the data quickly/automatically, and real-time data flow, I expect that it will disrupt the way market research is done in the near future.

Over the past 5 years, I’ve often thought that the results of social media studies worked best in conjunction with other forms of market research. If two or three separate methods of research produced the same results, I figured that the findings were meaningful and worth acting on. As I accumulate cases where survey research and social data research produce the same types of results, I’m less convinced that it’s always worth the time and budget to use surveys and panels and simply augment those with social data. At least once a month, I’m hearing clients say, “what you just found in one week usually to take us 3 months to discover.” There’s no defensible reason to spend $500k on focus groups and surveys that identify the same groups of consumers you could have identified at a fraction of the cost using social media affinity data.

What this means is that companies that harness social data to answer big questions, not just those related to how you should grow your Twitter or Facebook follower base, will have a huge advantage over those that don’t. For one, they’ll be saving hundreds of thousands, maybe millions, of dollars each year in research budgets. More importantly, they’ll be much faster at responding to changes in the marketplace since they will have insight into changing trends months, or even years, ahead of the competition.

The abundance of relevant social media data makes for exciting times in the marketing research world. We’re now at that place where data and insights aren’t just a convenient agency add-on when they happen to fit with an a priori strategic vision. Research can move at a fast pace that puts insights ahead of big decisions in every aspect of our work.