Stats errors don’t often make the news, but in January the ONS retracted and corrected the UK’s productivity growth data for the years affected by the Covid pandemic.
The correction bumped the UK’s productivity down from second best in G7 to second worst. The reason – a manual error in a spreadsheet.
Whilst stories like this grab the headlines, no doubt most of us will have spotted, or perhaps even unwittingly have caused, similar mistakes within our own firms. Given just how easy mistakes are to make, it can be surprising that relatively few insurers have a structured approach to Model Risk Management (or MRM) that extends beyond their internal capital model.
The PRA certainly sees model risk as a focus area for the insurance industry and made this clear in its January 2023 'Dear CEO' letter. This article considers what insurers can learn from the recent CP6/22 banking sector consultation, and how we can apply similar principles to reduce model risk within our firms.
What has the PRA said?
The PRA hints that model risk management will be a key area of focus in its list of 2023 supervisory priorities.
The immediate focus area for firms is to ensure that their internal capital models remain appropriate in the rapidly changing economic environment. This is already a hot topic within Lloyd’s but the same considerations apply to all internal model firms.
The PRA also expects insurers to consider the much wider ranging CP6/22 consultation which sets out future MRM requirements in the banking sector.
Broadly speaking CP6/22 would require firms to apply the following five principles to their internal MRM process:
- Model ID and classification: Define what constitutes a model, create an inventory of models used within the business, and categorise them according to the level of risk they pose to the business.
- Governance: Ensure that the MRM process sits within the formal governance structure, is reviewed and approved by the board, and is owned by a specific senior manager within the business.
- Development, implementation and use: Ensure that there is an appropriate process in place to manage and document the design of models, validate the data and model inputs, and ensure the system used to implement the model has appropriate controls.
- Independent validation: Ensure there is a robust process in place to independently challenge model developments and ongoing suitability for use.
- Risk mitigation: Implement consistent firmwide procedures to remedy or limit the use of models that are under-performing and to manage the use of out-of-model adjustments.
What should actuaries and risk teams be considering?
A good first step is to create a model inventory, to help assess the level and distribution of risk within the firm. This should capture the key tools used by the business, their purpose, and the risks they pose if they go wrong, as well as details of the current risk management process applicable to each model.
Scoping is important when building a model inventory as it can be easy to over-focus on business processes like capital, pricing, investment etc. Other easily overlooked areas are those with high customer impact, for example claim assessment or fraud detection. Such models may not pose a direct financial risk, but can pose reputational or regulatory risk, especially as a result of consumer duty rules. When setting the model inventory, it is best to initially cast the net wide and then narrow the scope later, to avoid missing risk areas.
After identifying the in-scope models, the next step should be to introduce consistent governance across the whole model inventory. At the moment, MRM is usually very siloed, with individual departments responsible for ensuring appropriate procedures are in place for the models they manage. The governance framework should include: simple reporting tools to assist the board in overseeing the MRM process; ensuring responsibility for MRM is clearly allocated to a senior manager within the business; and introducing appropriate controls to ensure that areas of higher model risk receive consistent levels of review, challenge and validation that are commensurate with the risk.
Setting the right culture is also important, as is taking culture into account when designing appropriate risk management and mitigation strategies. Errors are inevitable and can be exacerbated by a culture of finger pointing and blame. Most firms/departments have error reporting procedures, but there can be a big difference between the intended policy and employees’ actual approach, across the firm as a whole and between individual departments. Strengthening MRM procedures is important but doing so will be much more effective if people engage with those procedures sincerely.
Conclusions
Firms should take note of the PRA’s focus and assess the strength of their MRM procedures. As models become more numerous and complex, the risks associated with errors multiply.
Firms with stronger MRM culture in place, together with procedures that align with that culture, will be well placed to take advantage of new and exciting modelling technologies.
And perhaps next month ChatGPT can write my article for me…
First published in the March 2023 edition of the Actuarial Post.