How Predictive Analytics is Making Businesses more Intelligent

Post the 2008 recession JP Morgan Chase had to spend over $36 billion in legal fees and settlements. Most of the payouts were triggered because employees rigged markets, cheated clients, and violated trading rules.

However employees do not wake up one morning and decide that they will commit a crime and violate rules. Such activity is always preceded by specific behavioral patterns and in theory these white collar crimes could be prevented if monitoring was good enough.

Practically speaking, it’s tough for human managers to incorporate hundreds of signals like missed compliance meetings, data from emails and chat transcripts, breached risk limits etc to draw a coherent picture. Hence JP Morgan is using algorithms to analyze mounds of raw data and predict who is likely to trip up.

This is predictive analytics in action.

Predictive analytics as a crystal ball

Predictive analytics is one of the ways big data can be put to use like at JP Morgan. And it’s not only in financial markets where this kind of analytics is seeing widespread acceptance.

Hospitals are using predictive models to:

• Predict which patients will have a high risk of readmission
• Determine how long a particular patient might occupy a bed based on their medical history
• Ascertain the chances of survival of patients based on the time difference between a particular medical event and hospitalization.

Oil and gas companies are also emerging as heavy users of predictive analytics. Halliburton, which is a player in exploration and production industry, uses predictive analytics to:

• Detect events that can increase the cost of operations
• Improve drilling efficiency by taking into account factors like weight of the bit, rotating speed, the geological composition of the site etc
• Determine the root causes of critical metrics like Non Productive Time and Invisible Loss Time and cut down on these times to improve profitability.

However it’s not only commercial organizations that are seeing benefits of predictive analytics. At the Los Angeles County Registrar Recorder/County Clerk’s Office predictive analytics is being used to:

• Anticipate revenue changes in the coming quarters
• Help recruit election volunteers who are more likely to participate in polling.

Predictive analytics has started to gain real interest and the industry is estimated to grow to $24 billion by 2018 with growth happening in sectors as diverse as BFSI, Retail, Transportation, Energy, Travel, Telecom, Sports, and Environment.

The Challenges around Predictive Analytics

Predictive analytics looks a lot like statistical modeling but there is a crucial difference between the two.

Statistical modeling is pure math and while it also predicts outcomes based on input the results might be theoretical. For instance, a pharmaceutical company might use statistical modeling to predict the cost of testing for a particular drug based on current and past trends.

However the analysis will not take into account important factors like regulatory climate and change in government policies or market sentiments.

Predictive analytics does that, and thereby throws up an outcome that’s more closely tied to real world scenarios. It is thus both a science and an art, providing companies better business intelligence.

But implementing it in an organization comes with its own set of challenges. Unless they are overcome the project will be still born. Some of them are:

1. No specific goal

Unless you have a specific business goal or use case you cannot get the full advantages of predictive analytics. A goal will not only help you prioritize but also let you get the critical buy in from management.

2. Quality of data

You need plenty of data to make realistic predictions. While the data quality needs to be good you cannot wait around to gather perfect data. The best strategy would be to clean the data that you already have as much as possible and improve data collection processes.

3. Lack of talent

While algorithms play a huge role in predicting outcomes the process cannot be completely automated. You will need data scientists and subject matter experts to not only ensure that proper data is fed into the system but also need them to make sense of the output and draw insights that business can use in decision making.

The culture gap

Even after you have accounted for all these challenges your project can still fail because of cultural issues.

Many managers still make decisions based on their intuitions and past experience. They have no trust in math, partly because they don’t understand how the methods work. Unless managers start building trust in models and look at algorithms as something that will help them take better business decisions predictive analytics initiatives will flounder.