Big Data vs. Small Data
I received many interesting questions from the audience at my lecture last week, titled “Mom and Pop vs. Big Box – How Small Businesses Compete With Larger Rivals,” but one in particular stood out. One theme of my talk was that small businesses can gather and respond to richer and more detailed information about their local or niche markets and customers than can larger firms.
However, an audience member asked if this was changing because of Big Data. In retail, for example, large firms such as Walmart or Amazon can use data on the spending behavior of thousands or millions of customers to gather information at a level of detail previously unheard of. While I believe the answer to that question is no, it’s useful in highlighting the key differences in the advantages that businesses of different sizes have with respect to information.
Big Data is extremely useful in predicting the preferences, on average, of a consumer with a given set of characteristics. But for small local markets, for example, the sample size may not be large enough to meaningfully tease out idiosyncrasies in a community’s preferences that a small business on the ground might readily see. By talking to a few customers, a local grocer might easily notice what brands neighbors are recommending to each other, long before the resulting purchases would trigger anything in a Big Data algorithm. This is not to say the predictions yielded by Big Data aren’t useful—most small business owners would likely love to be privy to the information gathered by larger competitors—but such information is different in nature from that which smaller businesses are better positioned to gather and understand.
Another audience member brought up a key point in response to the discussion: Big Data will never directly capture the emotions behind consumers’ purchasing decisions or relationships to certain products and brands. As I discussed, many small businesses may be better positioned to forge deep relationships with customers, and therefore better understand this very human component of their decisions. The value of this type of information is underscored by the fact that many large companies hire ethnographic consultants to have personal conversations on the ground and find out what makes consumers tick.
Maybe it’s time to coin the term “small data” to reflect this type of understanding of one’s consumers and market. Such data can only be gathered through means such as direct conversations or roots in a local community. Small data can’t be gathered by a grocery store scanner, or aggregated across millions of observations. But it can greatly enhance the understanding of the environment in which a business operates. Both kinds of data certainly have their place, but much like we often forget the competitive strengths of small businesses, we tend to minimize the importance of small data.
Click here to sign up for the Daily Economy weekly digest!
Max Gulker
Max Gulker is a former Senior Research Fellow at the American Institute for Economic Research. He is currently a Senior Fellow with the Reason Foundation. At AIER his research focused on two main areas: policy and technology. On the policy side, Gulker looked at how issues like poverty and access to education can be addressed with voluntary, decentralized approaches that don’t interfere with free markets. On technology, Gulker was interested in emerging fields like blockchain and cryptocurrencies, competitive issues raised by tech giants such as Facebook and Google, and the sharing economy.
Gulker frequently appears at conferences, on podcasts, and on television. Gulker holds a PhD in economics from Stanford University and a BA in economics from the University of Michigan. Prior to AIER, Max spent time in the private sector, consulting with large technology and financial firms on antitrust and other litigation. Follow @maxg_econ.
