Catastrophes occur – Are we prepared?

July 23, 2014

Catastrophes occur.

Short term incentives, goals, and resulting business practices tend to devalue preparing for low-frequency high-impact events. In addition, human cognitive biases like those generally called “availability” and “perception distortion” and a host of others, tend to weaken attempts at effective long-term risk analysis as well. Because catastrophes occur, and because recovery requires activities materially different from dealing with more “normal” negative events, we are required to have plans in place to deal with them (or to have made sufficiently-informed decisions not to). In global Financial Services, I believe that major populations of our stakeholders assume that we are doing so.

This category of events includes, but is not limited to earthquakes, floods, droughts, tsunamis, cyclones and more. Some Financial Services organizations have attempted to address these natural and some political risks via geographic distribution of all critical functions — where the loss of any given locality or region would remain below the threshold of “catastrophe.” That approach is not effective against other types of systemic vulnerabilities. Increasingly interconnected global business and technology infrastructure and operations have added new categories of potential catastrophe. It is likely that there are new vulnerabilities that emerge from a greater degree of interdependency and interconnectedness than executive decision-makers understand. The combination of globalization in the Financial Services industry along with Internet-enabled real-time reach is often highlighted as bringing opportunities to hedge risks through investment, vendor, partner, and customer diversity. The potential that it also brings for strategic and enterprise-wide harm is not so well documented.

Internet “plumbing” like DNS or traffic routing are the product of relatively “ancient” architectures, and in some instances, incorporate decades-old code. Successful widespread exploit of Internet of Internet “plumbing” could result in catastrophic impacts on global financial services — virtually all of our markets depend upon real-time or near-real-time Internet connectivity. Sometimes this is a direct impact, but it will almost certainly damage operations somewhere down the supply-chain. Patching, disinfection, throttling, or containment at Internet scales is a challenge — one that we are not generally prepared for. Successful targeted or widespread endpoint exploit via one or another Internet pathogen has the potential for catastrophic impacts — if hostile agent can employ malware to gain partial or total control of all our infrastructure and/or user endpoints, we don’t own our businesses anymore — that kind of asset-transfer is something all financial services leaders need to be aware of. For many of us, even the failure of a single vendor/partner or a network of vendors/partner presenting a common interface could result in materially-negative, even catastrophic consequences. What would happen to your organization if Amazon, Google, Bloomberg, Bank of New York, (pick your large-scale partner) no longer had an effective Internet presence? How would your enterprise continue to function if broad categories of securities trading and/or settlement went dark because a systemic weakness in that “market” was exploited, and “turned-off?”

I believe that for most of us in Financial Services, this topic deserves more attention than it has generally been receiving.

The World Economic Forum [WEF] has been sponsoring some work on this topic that might be a useful resource in any effort to get this effort started, restarted, or enhanced at your organization.

In their 2014 “Global Risks Report” WEF authors argued that a myopic focus on quantitative risk probability measures can disserve organizations. They also warn of how too heavily weighted “intuitive” thinking about risk can also weaken an organization’s ability to deal with potentially-catastrophic risks.

I strongly recommend reading this the 2014 WEF “Global Risks Report,” especially section 2, pages 38 through 47, where it focusses on cyber-risks and strategies for managing global risks.

As a teaser, glance at their quick review of risk management and monitoring strategies below:

Risk-management strategies are guided by a firm’s risk appetite; the level of risk an organization is prepared to accept to achieve its objectives, such as profitability and safety goals. Often, although not always, there is a trade-off between profitability in times of normal operations and resilience in the face of negative events affecting the firm. Examples of risk management and monitoring strategies include:
  • Mitigation measures: Actions taken by the firm to reduce the likelihood and/or consequences of a negative event; for example, designing plants to withstand specific levels of natural disasters such as earthquakes, floods and hurricanes.
  • Accountability measures: Finding ways to incentivize individual employees not to cut corners in ways that would normally be undetectable but would increase a firm’s vulnerability in a crisis, such as failing to maintain back-ups. Some firms hire external consultants to assess how effectively they are mitigating risks identified as priorities.
  • Supply-chain diversification: Sourcing supplies and raw materials from multiple providers in different locations to minimize disruption if one link in the supply chain is broken. Another hedge against sudden unavailability of inputs is to maintain an excess inventory of finished products.
  • Avoiding less profitable risks: Firms may decide to drop activities altogether if they represent a small part of their overall business but a significant part of their risk profile.
  • Transferring the risk: In addition to the range of insurance products available — liability, property, business interruption — some large firms run their own “captive” insurance companies to distribute risks across their own different operations and subsidiaries.
  • Retaining the risk: When insurance is unobtainable or not cost-effective, firms can choose to set aside reserves to cover possible losses from low-probability risks.
  • Early warning systems: Some firms employ their own teams to scan for specific risks that may be brewing, from political crises, for example, to storms off the coast of Africa that may become hurricanes in the US in the next fortnight.
  • Simulations and tabletop exercises: Many firms simulate crisis situations; for example, by making critical staff unexpectedly unavailable and assessing how other employees cope. Such exercises can capture lessons to be integrated into the risk-management strategy.
  • Back-up sites: Many firms are set up so that if one or more factory or office becomes unusable, others are quickly able to assume the same functions.

    [Italics above quoted from: WEF, GRR 2014, page 44]


World Economic Forum – Global Risks Report 2014

Third-Party Security Assessments – We Need a Better Way

July 6, 2014

“According to a February 2013 Ponemon Institute survey, 65% of organizations transferring consumer data to third-party vendors reported a breach involving the loss or theft of their information. In addition, nearly half of organizations surveyed did not evaluate their partners before sharing sensitive data.” [DarkReading]

Assessing the risks associated with extending Financial Services operations into vendor/partner environments is a challenge.  It often results in less-than-crisp indicators of more or less risk.  Identifying, measuring, and dealing with these risks with a risk-relevant level of objectivity is generally not cheap and often takes time — and sometimes it is just not practical using our traditional approaches.  Some approaches also only attempt to deal with a single point-in-time, which ignores the velocity of business and technical change.

There are a number of talented security assessment companies that offer specialized talent, experience, and localized access virtually world-wide.  The challenge is less about available talent, but of time/delay, expense, and risks that are sometimes associated with revealing your interest in any given target(s).

There are also organizations which attempt to replace a repetitive, labor-intensive process with a non-repetitive, labor-saving approach that may reduce operational expenses and may also support some amount of staff redeployment.  The Financial Services Round Table/BITS has worked toward this goal for over a decade.  Their guidance is invaluable.  For those in the “sharing” club, it appears to work well when used applied to a range established vendor types.  It is also, though, a difficult fit for many situations where the candidate vendor/partners are all relatively new (some still living on venture capital) and are still undergoing rapid evolution.  Some types of niche, cloud-based specialty service providers fall easily into this category.  The incentive to invest in a “BITS compliant” assessment for these types of targets seems small, and any assessment’s lasting value seems equally small.

Some challenges are enhanced by increasing globalization – for example, how do we evaluate the risks associated with a candidate vendor that has technical and infrastructure administrative support personnel spread across Brazil, Costa Rica, U.S East & West coasts, Viet Nam, China, India, Georgia, Germany, and Ireland?  Culture still matters.  What a hassle…

None of that alters the fact that as global financial services organizations we have obligations to many of our stakeholders to effectively manage the risks associated with extending our operations into vendor’s environments and building business partnerships.

When the stakes are material – for example during merger or acquisition research – it is easy to understand the importance of investing in an understanding of existing and candidate third-party risks.  There are many other situations where it seems “easy” to understand that a third party security assessment is mandated.  Unfortunately, not all use cases seem so universally clear-cut.

When we are attempting to evaluate platform or vendor opportunities, especially when in the early stages of doing so, the time and expense associated with traditional approaches to full-bore third-party risk assessments are a mismatch.  Performing third-party risk assessments in-house can also reveal sensitive tactical or strategic planning which can negatively impact existing relationships, add unnecessary complexity to negotiations, or, in edge cases, even disrupt relationships with key regulators.  As an industry, we have got to get better at quick-turn-around third-party risk assessments that are “good-enough” for many types of financial services decision-making.

For years, “technicians” have been evaluating Internet-facing infrastructure for signals of effective technology-centric risk management practices – or for their absence.  Poorly configured or vulnerable email or DNS infrastructure, open SNMP services, “external” exposure of “internal” administrative interfaces, SSL configurations, public announcements of breaches, and more have been used by many in their attempts to read “signals” of stronger or weaker risk management practices.  A colleague just introduced me to a company that uses “externally-observable” data to infer how diligent a target organization is in mitigating technology-associated risks.  Based on a quick scan of their site, they tell a good story.*  I am interested in learning about anyone’s experience with this, or this type of service.

*I have no relationships with BitsightTech, financial or otherwise.



“BitSight Technologies Launches Information Security Risk Rating Service.” 9/10/2013

“Bits Framework For Managing Technology Risk For Service Provider Relationships.” November 2003 Revised In Part February 2010.

Shared Assessments.

The company a colleague mentioned to me…

%d bloggers like this: