Why did the pollsters and the bookmakers wrongly predict the outcome of the EU Referendum? Does a simple yes or no question mean it is easy to predict the answer?
Some organisations are driving more value out of big data than others, using it to redefine how they interact with customers. In some industries – such as public relations and marketing – big data has the potential to fundamentally change the profession.
As we discussed in a previous blog post, actionable insight can only be gained by turning big data into relevant chunks of small data. Equally important is asking the right questions of a data set.
Two high-profile failures
The claims of two predictive analytics expert organisations – pollsters and bookmakers – were wrong in two high profile political cases in the past two years: in the 2015 UK general election and the 2016 EU Referendum. In both cases pollsters did not show any accurate predictive analytics and were completely wrong with the results of opinion polling. As the polls closed for the EU Referendum vote, bookmakers were offering 7/1 odds that remain would win and 3/1 on average across all bookmakers against Brexit.
Sam Knowles explains: "The trouble is, issues are rarely as simple as we would like them to be. While politicians may force us to make binary choices – Brexit or Remain, Trump or Clinton – the truth is often very much more complex and nuanced than such rigid decision-making trees allow. Our decisions about who or what deserves our vote or our consumer choice sit along a spectrum, but the final choice is unable to reflect that."
Both the pollsters and the bookmakers will now need to question the process of their analysis. Did they use the right tools? Did they ask the right questions?
The wrong questions inevitably lead to the wrong conclusions. In the case of the EU Referendum, asking the simple yes or no question to gauge how people would actually vote when the time came did not assess the potential impact of anti-establishment feeling. If this feeling was significant enough, would voters lie about their answer to the question to express it?
Opinion is granular
One explanation for the inaccuracy of pollster, bookmaker and even media predictions has been the 'bubble effect' of London – sentiment in London did not match sentiment throughout the rest of the country. This bubble effect can happen in any demographic – whether it is an electorate or customer database being analysed – and the ability to understand the fine differences between small groups is crucial. Binary yes or no questions are not enough when there is a broad spectrum of possible answers.
Sam Knowles explained: "On one level this is a very stale yes or no answer, on the other it is incredibly nuanced. The failure of analytics here says that if you have this kind of question, do not ask people what they will answer. You need a much more granular idea of opinion across the spectrum. Use the analytics to understand the makeup – in this case the electorate, but in a marketing context the consumer base – to get a richer answer."
What can businesses learn from this?
Multiple variables interact and big data analytics offers the power to understand the different component parts of a spectrum of opinion in finer detail.
To improve forecasting around political decision making – and indeed, yes or no decisions based on a much broader subset of choices – more detailed models are needed. These models need to constantly adjust to new data if insight into what customers value – rather than how they will answer a binary question when pressed – is to be gained.
"At this stage it is a fabulous what if" explained Knowles, but businesses can take the lessons from this forward into customer analysis. By asking binary questions, you force people to make a decision, but by analysing a much broader spectrum of opinion you can get a better understanding.
Big data is made up of lots of small data – but the right tools are needed to see these subsets. Connection is key and can allow an organisation to move from assumption to knowledge. Data analytics can give you the right answers, but only if the right questions are asked.
LexisNexis recently published episode two of the Small Data Forum podcast, which focuses on the role and use of data in the campaigning for the EU referendum in the UK. In the podcast Neville Hobson, Senior Business Consultant at IBM and Sam Knowles, Founder & Managing Director of Insight Agents discuss the outcome of Brexit and ask: what can we learn from the failed predictions?
- Pokémon Go: crisis communications in the face of outstanding success?
- The role of data in Regrexit
- Weathering a social media storm
- Measuring media campaign success
3 ways you can apply this information right now to...
- Monitor industry conversations to keep track of what the press and internet are saying. Our innovative technology and premium content – including traditional and new media – will help you transform information into actionable intelligence. Find out more.
- Follow this blog series; subscribe to our blog to have the updates delivered to your inbox.
- Leave a comment below. Let's start a conversation!