Recently, we’ve been spending a lot of time talking to insurance NEDs about how to address the impact of behavioural biases on decision-making.
In this blog, Oliver Grossman discusses how the science of “superforecasting” can help improve board effectiveness by helping NEDs to:
- make better use of their independence and expertise and;
- make clearer and more robust decisions.
What is "superforecasting"?
"Superforecasting" is a term coined by Professor Philip Tetlock. Based on his research, he asserts that some individuals are capable of making more accurate predictions than the subject matter experts in a particular field. Much of their success is due to their independence from the events in question and the structured nature of their thought processes. Tetlock’s research into the approaches used by "superforecasters" has some lessons that I think are particularly relevant to NEDs.
How to make better use of independence and expertise
- Start by taking the “outside view”. View the task as someone from outside of the company would see it. For example, if you were looking to branch into the UK motor insurance market, where existing insurers operate at around a 100% net combined ratio, it’s recommended to start by assuming that your business will evolve similarly. You can then update your estimate by incorporating the “inside view”, ie specific knowledge relating to your own book of business, such as underwriting information. Thinking in this way forces your analysis to start with a meaningful and unbiased anchor, which can then be built upon. Since NEDs are a step removed from the underwriting process and the inner-workings of the business, they are in an ideal position to provide this viewpoint.
- Decompose problems into their knowns and unknowns. Tetlock’s research shows that superforecasters tend to split difficult problems into tractable sub-problems. To illustrate this, if a NED is faced with the complex question of “How will COVID-19 affect our new motor book?”, they could challenge the business to answer several meaningful sub-questions, for example looking at claims frequency vs average claim size, or new business against renewal business etc. The answers in each case can be pieced together and might provide better insight than trying to come up with an overall answer to a very difficult question. Whilst it is not necessary for the NED to review these sub-analyses in detail, the process should enable the NED to place greater confidence in the overall answer.
How to make clearer and more robust decisions
- Communicate clearly using probabilities. Superforecasters tend to think in terms of probabilities against events. Being told there’s a 70% chance of rain provides more context than being told it might rain. This framework of thinking is not solely restricted to analytical work and should also be applied to high-level decisions. NEDs can apply this to their own challenge of the business by asking for quantitative indications of uncertainty.
- Document the why, not just the what. The thought process leading to an estimated probability of a certain outcome is as important as the estimate itself. Take the following scenario: there are 3 board members, and they each independently believe there to be a 75% chance that company profits will benefit from a revised social policy. Is 75% the most accurate probability we can acquire? Given that each board member will have factored different sources of information into their estimate, it is likely that a more precise probability differs to the original 75% as beliefs are updated and estimates are revised. More generally, if the thought process behind an estimate is clearly documented, it is then much more feasible to perform a useful post-mortem review once the outcome is known. Even if predictions appear accurate, they could be the fortunate result of offsetting errors.
- Identify where predictive capabilities lie. How often do we measure the accuracy of our decisions from the perspective of information known when the decision was made? Decisions should be recorded and evaluated to form fast and robust feedback loops. One way to assess the accuracy of prediction is by using Brier scores. Where a large number of different predictions are being made by various people over time (as is common in the management of any insurer), Brier scores can help identify those with consistently stronger predictions and, importantly, which types of problems are the easiest/hardest to predict.
The takeaway
By incorporating the above into future discussions, NEDs can build a more transparent decision making framework, which will help the business to:
- answer difficult questions by breaking them into simpler sub-problems;
- communicate estimates more clearly by using probabilities; and
- improve confidence in predictions via feedback loops.