Evaluating Market Research

What factors influence growth projections in market researchHere’s my advice for evaluating market research, as founding editor of Modern Health Talk and someone who has both purchased and written major market research reports.

Market Research can be biased and short-lived, so before you buy any expensive research reports, make sure you get a chance to interview the authors personally so you understand their assumptions, research process, and what shaped their conclusions. Make sure they aren’t just extrapolating trends but also include thoughtful discussion of market drivers, inhibitors and enablers, because you’ll need that insight to craft your strategies. 

Do the authors understand what’s driving trends in healthcare, including the ageing populations and resulting increases in chronic illness and the cost of treatment, environmental pollution, the availability of nutritious foods, and physician shortages? What do they think of obstacles posed by legal, privacy and security issues, payment models, medical school curriculum and public funding, and how might that change with politics? What impact will regulators and the political process itself have, either as drivers or inhibitors? And what will be the short- and long-term impact of broadband Internet access and the exponentially accelerating pace of tech innovation?

Consider how quickly Moore’s Law is finding its’ way into healthcare and ask the researchers what they think will likely happen as medical devices keep getting cheaper, smaller, more accurate, and easier to use. And why.

Consider the impact that IBM Watson is already having, moving from being just a physician assistance tool to actually advising and coaching consumers directly. How quickly will each of the medical functions done today by physicians in hospital & clinic settings safely move down-market to consumers at home? Won’t retail clinics and kiosks, and home doctor/nurse visits, just be stepping-stones along a path toward that eventuality? 

And finally, what phases will the market projections go through, and when? What levers might you have, to make the projections happen more quickly? What partnerships and investments should you be making now? And what should you watch out for that could cause the market projections happen more slowly?

About My Perspective

Just as you should understand what shapes the perspective of market researchers, you should also know what shapes my viewpoint before taking my advice. My perspective comes both from buying market research reports and writing them.

At IBM — Before retiring in 1999 as a market strategist, I studied emerging markets where there was no history or trends to extrapolate, and where making projections relied a lot on gut instinct, knowledge of what tech innovations were coming out of research labs, the typical time-to-market cycle, and an understanding of the market drivers and inhibitors. Often we’d buy a report from a 3rd party researcher even though we knew considerably more about the future opportunities than they did. Why? Because (1) we wanted to show executives projections with alternative outsider viewpoints, and (2) we wanted to understand how conclusions from different insights compared their our own.

After IBM — As the founder of CAZITech Consulting, an independent Digital Home consultancy, I was on the other side, writing market research. “Home Systems: Home Controls – Seventh Edition” was a 300-page report that Parks Associates sold for about $2,500 per copy in 2004. It examined new developments in lighting, HVAC, security, entertainment, and appliances and concluded that “the Home Controls industry is poised to cross a 30-year-old chasm separating high-end new homes from bigger opportunities in mainstream retrofit markets.” But that Home Controls market still has not achieved the potential I envisioned, even after 12 years.

Different Business Models — The Home Systems report I wrote was fairly expensive, but I’ve seen other research offered for free through different business models. One example was a well-researched report about Information Appliances (today we’d call that Internet of Things). It was offered for free by a large brokerage house, identifying what companies were doing interesting product development, and where they saw the opportunities. It was free in hopes that readers would invest. Brokerage fees helped fund the research in that model. Another example was a special edition of a technical magazine (I think it was CNET) that made its money from subscriptions and advertising. The magazine issue includes many well-written articles by true experts in the field that collectively provided insights that market researchers often lack.

My Top-10 List of False Predictions:

Another way to understand the challenges of predicting the future is to check this list of notable FALSE predictions.

  1. “This telephone has too many shortcomings to be seriously considered as a means of communication. The device is inherently of no value to us.” — Western Union internal memo (1876)
  2. “Everything that can be invented has been invented.” — Charles H. Duell, commissioner of the US Patent Office, recommending that his office should be abolished (1899)
  3. “The phonograph has no commercial value at all.” — Thomas Edison
  4. “Stocks have reached what looks like a permanently high plateau.” — Irving Fisher, Professor of Economics, Yale University (1929)
  5. “There is not the slightest indication that nuclear energy will ever be obtainable. It would mean that the atom would have to be shattered at will.” — Albert Einstein (1932)
  6. “A rocket will never be able to leave the earth’s atmosphere.” — New York Times (1936)
  7. “I think there is a world market for about five computers.” — Thomas J. Watson Jr., chairman of IBM (1943)
  8. “The world potential market for copying machines is 5000 at most.” — IBM to the founders of Xerox as it turned down their proposal (1959)
  9. “There is practically no chance communications space satellites will be used to provide better telephone, telegraph, television, or radio service inside the United States.” — Craven, FCC Commissioner (1961)
  10. “The concept is interesting and well-informed, but in order to earn better than a ‘C’ the idea must be feasible.” — Yale professor’s comments on a term paper submitted by Fred Smith for an overnight delivery system. Two years later, Smith founded Federal Express.