Big data is dead, long live small data

15 Apr 2016 12:00 am by Megan Burnside

Big or small?

When branding expert Martin Lindstrom identifies a trend, it is usually time for businesses to sit up and take note.  The award winning author of Buyology and Brandwashed is widely regarded as one of the leading thinkers on management and a member of Time Magazine's Influential 100 list.

In March, Lindstrom published a new book entitled: Small Data – the tiny clues that uncover huge trends.  The Foreword is written by Chip Heath, a professor at Stanford Graduate School of Business.  He says: "In today's business environment, Big Data inspires religious levels of devotion…Big Data doesn't spark insight. New ideas typically come from juxtaposition — combining two things that previously haven't been combined. But Big Data typically lives in databases that are defined too narrowly to create insight."

The reality, as Lindstrom outlines in his book, is that big data is only as useful as the insight that can be gleaned from it.  Business insight does not come from collating and processing more and more data but by knowing which data is the most critical to delivering relevant insight.

Prescriptive analytics – a model

Using a better understanding of data from the past to drive better decision making is the basis of prescriptive analytics.  Essentially this involves a process that typically works in the following way:

  • a large volume of data is collected
  • this data is aggregated to make it suitable for analysis
  • analysing this data helps to identify trends
  • these trends help to shape insight that makes for better decision making
  • the better decision making is implemented to drive improvement in efficiency or processes, which can then be analysed through collection of new large volumes of data

Finding meaning in the mass

This whole process, however, is rendered meaningless unless those responsible can identify the critical data items that hold the key to insight.  

As an example, leading brands receive more than 20 messages that need actioning on Twitter per hour – around 500 per day.  Naturally enough brands employ customer services staff to respond to these. The brand will know how many customer services representatives are needed to manage this volume of activity and act accordingly, increasing and decreasing the number of staff required to cover seasonal activities and product launches.  This is information gleaned from big data.

However what the brand might also want to know is which tweets will take off with a life of their own – being retweeted hundreds of times and creating significant brand damage.  Are there characteristics that are identifiable as triggering a social media firestorm and likely consequent interest form the mainstream media, and if so, what are they?  This is insight gleaned from small data that enables brands to identify and action priority cases before they explode.

Practical applications for small data – an example

Several companies are now using relevant data sources, manipulated through powerful algorithms to enable businesses to gain insight from small data. Many of these companies are born from academic research taking place in the world's leading Universities.

One such company is Cytora.  This company, which is funded and supported by the University of Cambridge, has created a comprehensive dataset to help companies analyse the impact of geopolitical risk on their business.  Cytora collects data from millions of different news sources.  It then cleans, structures and contextualises it to find the relevant small data that impacts supply chain risk management.  

For companies sourcing products around the globe the ability to guarantee supply of goods is a critical issue.  A number of different factors can impair their ability to ensure regular supply.  These can vary from environmental issues (such as earthquakes, hurricanes or volcanic eruptions) to political issues such as changes of Governments, political coups or outbreak of war.  

To ensure a reliable supply chain, companies need to understand the risk associated with sourcing goods, particularly in volatile, emerging or complex markets.  Cytora identifies the critical small data that enables its customers to better assess the short, medium and long term risk of sourcing goods from particular markets.

Companies that can search for and identify the critical trends that impact a business will always have a significant competitive advantage.  It is not about how much data you collect but how you use it to generate insight.

Related blogs

p.s. 3 ways you can apply this information right now to better understand news monitoring and analytics

  1. Monitor industry conversations to keep track of what the press and internet are saying. Our innovative technology and premium content – including traditional and new media – will help you transform information into actionable intelligence. Find out more.
  2. Follow this blog series; subscribe to our blog to have the updates delivered to your inbox.
  3. Leave a comment below. Let's start a conversation