What does the near-disaster on London’s Millennium Bridge have to teach us about model risk?
“The bridge, inaugurated with great fanfare by the Queen in 2000, filled with people and began to sway so strongly it had to be immediately shut down,” said Ravi Chari, Manager, Americas Risk Practice at the SAS Institute. “When the bridge was modelled during development, the developers did pose the question ‘what is the probability of 10,000 people walking in unison on the bridge?’ And the answer was ‘practically zero’—but that’s exactly what happened on Day One!” Chari was the second of two speakers on model risk management (MRM) at a GARP-sponsored webinar on April 24, 2014.
Chari pointed out that model risk had a different profile prior to the Long-Term Capital Management (LTCM) event in 1998. “Prior to LTCM we had huge losses due to rogue traders or fiduciary-type cases like Barings, Sumitomo, Orange County, and Bankers Trust.” However, the losses at LTCM were due to the failure of very sophisticated models that was used under incorrect circumstances. The Amaranth Advisors case and sub-prime fiasco involved models that were incorrectly developed or applied. (Complex math models are among the top five reasons cited for the financial crisis, he said.)
Besides these headline news items, model risk “has generated a lot of interest especially due to recent regulations surrounding stress testing,” said Chari, providing a timeline that included Dodd-Frank Act Stress Testing (DFAST) and Comprehensive Capital Analysis and Review (CCAR).
“The key requirement is that [a bank] must generate balance sheets and income statements for nine quarters under three scenarios,” said Chari. Look at it as “a hub-and-spoke model,” he suggested. Risk managers or economists forecast macroeconomic variables such as GDP and interest rates using various assumptions. Correlations between variables depend on the scenario being modelled. “These are then aggregated in the hub.”
“There are hundreds of models used during this process,” he said, because “banks are expected to have two models for each material risk—a champion and a challenger model.”
Thus, literally hundreds of models must be examined and validated on a regular basis.
Chari devoted a slide to the interplay between the Supervisory Guidance On Model Risk Management SR11-07 and new regulations such as CCAR.
The new regulatory regime extends well beyond technical calculations. “Before getting into numbers, the regulators want to see the inventory—both champion and challenger models—for all major risks. You must have a narrative for the model that includes [specifying] when the model will break down.” The regulators will also check for existence of a sound model validation process “to show whether the numbers are trustworthy” and will look at the governance for model risk overall.
Clearly, in this new regulatory regime, MRM is a branch of risk management that is coming into its own. Ignore it, and you might build wobbly bridges—or worse. ª
Click here to read about the first presentation on model risk. ª
Click here to view the webinar Model Risk Under the Post-Crisis Regulatory Regime. Chari’s portion begins at slide 12.
Click here to check out the SAS research paper “Time is Precious, so are your Models.”