This morning, Americans of all political leanings can come together an rejoice that the presidential campaign is over.
While pundits and policy wonks will discuss the campaign endlessly in the days, weeks, and months to come. They will talk about vision, strategy, leadership, and so on.
But that’s not what I want to discuss. I want to talk about how people make decisions, and why the process for making decisions in the face of uncertainty is so important.
There are 2 kinds of people making predictions about the elections: pundits and pollsters.
Pundits get paid to have opinions. The bolder and more controversial, the better. The more they support the views and desires of an audience, the more that audience wants to hear what they have to say. So, whether deliberately or not, they often start with the result they want, and then construct a narrative to support it. If they turn out to be wrong, they extend the narrative to explain away while they were wrong. (The worst examples of this are the apocalyptic preachers who raise money predicting the end of the world, then have to say that the world was spared by the power of prayer.)
Pollsters take a sample of the population and attempt to extrapolate the overall result. They have spreadsheets. They speak of margin of error. They have big spreadsheets. They are nerds. Depending on their methodologies, they may introduce biases which could throw off the prediction. Nate Silver and others aggregate polls together, which does a few things. First, it can dramatically increase sample size, which tends to improve predictive power. It can also balance out biases in the polling methodologies.
What happened last night? A lot of pundits who had been poking fun and Silver’s model ended up with egg on their face.
When executives make predictions, do they act as pundits or pollsters?
Many executives think they need to be all-knowing, certain, and “visionary”. In the face of uncertainty, they try to create certainty. This is a necessary skill. Sooner or later, someone has to make a decision. The problem is that the prediction and the decision are different. Predicting that you might miss your sales forecast is one thing. Deciding what to do about it is another. But if you can’t predict well, you are likely to make less effective decisions.
The higher up in the organizational hierarchy you go, the worse this problem gets. The sales manager talks to the sales reps and gets worried about the forecast, but everyone assures her that the deals will close. No one knows if declining sales in one sector are blips in the data, the result of a new competitive landscape, or general decline of those customers’ purchasing habits. The sales managers aggregate their forecasts for the regional managers, who then roll them up for the VP of sales, who knows this game but doesn’t know how off the forecast might be. Meanwhile, the CEO and CFO don’t know how much discounting is going to happen on the last day of the quarter to get proposals signed, so they don’t know how much profit to expect. But people like certainty more than certainty, so everyone runs around boldly saying what will happen. Then, after the books close, they try to figure out what happened.
The real lesson for executives from the election is not that data is better than guessing, it’s that aggregating various data sources produces better predictions than single sources of data.
For example, I often have to look at the potential impact of price changes on sales and profits. This might be a firm contemplating an annual price increase, or a change in discounting policy, or even debating discount negotiation limits for proposals. The biggest challenge is often anecdotal evidence that exaggerates the impact of potential lost sales. This causes a gut (“pundit”) reaction against more disciplined proposal pricing processes. One way to combat this tendency is to ask different parts of the company– sales, finance, customer support– to prepare an estimate of the impact of the changes. Don’t force them to use the same spreadsheet– let them use their own methodologies. Then look at the distribution of predictions. They will probably cluster around the correct value, or at least be much closer than using a single prediction or relying on gut feel.
If we have two different ways of making very important predictions, and one produces better results, why do we cling so stubbornly to the pundit approach?
First, it’s easier to just go with your gut than do the work to generate statistically relevant predictions. But it doesn’t have to be that much work if we get in the habit of doing it. Like many things in life, you usually end up doing less work if you take time to understand the problem correctly at the beginning.
Second, we lionize bold, visionary leaders. And as executives, we take that to be a critical part of our very identity. Admitting that we don’t know what’s happening in our market, with our business, feels like weakness, like failure. Steve Jobs famously said he doesn’t like to ask customers what they want. He’d rather tell them. Obviously, Apple has been very successful under his watch, but a lot of the credit goes not to his “vision”, but to his hiring people like Tim Cook, whose team of wonks could build accurate sourcing and forecasting models, so Apple could not just make cool products, but make money selling them. It’s not as glamorous, but it’s just as critical, and even more importantly, you don’t have to a LSD-enhanced genius to practice it. For all of Apple’s hits, we don’t know how many projects they killed because the projections didn’t work out.
As we approach the end of the year, here’s a suggestion for your team:
Discuss an annual price increase (we’ll talk about why this is a good idea in a separate post), including a list price change, as well as changes in your discount structure and negotiated pricing. Have different people prepare estimates of the impact on sales. From there, you can derive the expected changes in revenue, cost, and profits. Throw the raw predictions on a whiteboard. Then discuss the different approaches people took to generate those estimates. You may be pleasantly surprised. And by pleasantly surprised, I mean you may find yourself looking a much more profitable 2013.