I Don’t Believe in Predictive Analysis
Predictive analysis is a miracle cure. At least that is how it is being pitched to and by the HR fraternity. A current talk I attended went as far as to say that if an organization didn’t invest in predictive analysis soon, they are headed to Doomsland. For the uninitiated, predictive analysis is the branch of the analytics used to make predictions about unknown future events. Sounds like magic, doesn’t it?
Except I don’t buy it. I have some very deep reservations about predictive analysis in the HR space and not one person has been able to address it. Here is why.
Two things happen once you run predictive analysis. Either the prediction comes true or it doesn’t. Let me pick a popular parameter to illustrate my discomfort with HR’s new favorite child. Attrition.
Let’s assume you run predictive analysis based on a number of carefully thought out & statistically tested variables. It tells you that X has a high probability of leaving your organization. Now two things can happen:
- X leaves the organization
- X doesn’t leave the organization
In the first case, how do you guard against self-fulfilling prophesies? Did your behavior towards X change in a way that led X to leave? On the other hand, if your interventions led to retaining X, how do you know the predictive analysis was right in the first place? What are the success metrics that convinced you that predictive analysis works?
My second apprehension is ensuring that your input variables are comprehensive and accurate. We all know the concept of garbage in & garbage out. Predictive analysis works great in other fields since the fields are slightly more consistent as against human behavior. Most HR professionals struggle with running multivariate regression accurately. How do we then know that we’ve captured the 101 variables that effect attrition correctly? Also, are we a 100% sure that what we are calling predictive analysis is not pure play regression analysis?
My final apprehension is the risk of blind belief in data. I’ve often said that it is important to be data informed vs data driven, that it is necessary to validate what data tells us with supplement information. Speaking at the 2014 Predictive Analytics World conference in Boston, John Elder, president of consulting firm Elder Research, Inc., made a good point when he noted that people “‘often look for data to justify our decisions, when it should be the other way around.”
Moreover, predictive analytics works best in a stable environment. One in which the future of the business is likely to resemble its past and present. As Harvard Business School professor, Clayton Christensen points out - in the event of a major disruption the past will do a poor job of foreshadowing future events and if we know one thing about HR, it is that the human behavior depends on just too many variables.
If you ask me, I would, any day, place my bet on multivariate regression over the predictive analysis sold to us.