Myth #1
Purchasing a third-party vendor software solution means no model validation is required
Not so fast! Buying a third-party vendor software solution for front office pricing and risk requirements rather than developing one internally does not remove the need for an independent validation that the model is fit for purpose and is functioning correctly. In our experience it is often even more important in this context, given that there is less visibility of the inner workings of the model. For institutions without the in-house capability to perform a model validation, the options have historically ranged from expensive (a bespoke validation by a generalist consultant) to risky (either doing no validation or a poor job of one in-house). A specialist model validation consulting firm (such as QuantSOS) can perform validations more efficiently (and therefore most cost effectively) than generalist competitors by leveraging their repeated exposure to similar validations to unlock scale benefits through the automation of tools and processes.
Myth #2
One software system = One model
The regulatory definition of a model is very broad – basically, anything which converts input data into something that guides decision making can be thought of as a model. Each financial institution then has to develop a practical interpretation of what that means for them when putting together their model risk management framework, but in our previous experience a large third-party software system is rarely thought of as just one model. At the very least, taking a compartmentalised approach in which key building blocks (for example yield curves and volatility surfaces) are validated as models in their own right can greatly simplify the validation of more complicated models that rely on those building blocks. In one institution we have seen upwards of 50 models on their model register relating to different components of one third-party system.
Myth #3
With an in-house quant team model validation is “free”
Model validation is a technically demanding and time-consuming exercise and the appropriate staff to perform one can be hard to find. Financial institutions with an in-house quant team likely have qualified people in-house capable of performing model validations but need to consider the opportunity cost of diverting these staff away from potential revenue generating opportunities. We have seen multiple instances of large quant teams being entirely occupied with model validation exercises for months on end causing massive frustration for the businesses they support who would rather have them working on activities that more directly impact revenue generation. Outsourcing model validation to specialists in that field may be a value accretive strategy even for institutions with an in-house quant team.