There is an old business axiom that goes, “If You Can’t Measure it, Don’t Do It.”
This is just like breathing for any serious manufacturing. ISO-9000 quality management processes dominate there. And serious marketing appears to have implemented schemes for testing, measuring, and learning from almost everything they do. They have become fiendishly effective at causing predicted consumer behaviors.
Information security practices, and broader technology infrastructure and operations risk management have not yet been rationalized in a manner analogous to manufacturing and marketing. Too often we depend largely on employee orientation signatures on “statements of understanding,” periodic security awareness reminders, policies and standards, a disjointed range of tools and technologies, along with selective monitoring and reporting. We have not been able to accurately or efficiently identify probabilities of loss associated with given risks or to effectively prioritize our investments in mitigating specific risks That situation probably results in both unproductive business investments and the assumption of irrational risk loads. Neither well serve the needs of our customers, investors, and partners.
I have recently started reading with a little more focus about “prediction markets” and the broader “collective intelligence.” I’ll admit up front that I have not really believed in either of these in the past. It seemed to me that only those individuals having specialized training, relatively extensive experience, along with some material level of leadership and maybe even wisdom were going to be helpful in making the really important decisions about risk probability and risk mitigation strategies in complex business environments. Maybe I was wrong.
In many corporations, thousands or tens of thousands of employees, contractors, partners, vendors, and others associates represent a diverse pool of experience and talent. Prediction markets promoters offer a range of examples where this methodology has been successful. Examples range from predicting presidential elections, casualty counts, printer and movie ticket sales, FDA approval of new drugs, and project completion dates (see: http://www.rmmag.com/Magazine/PrintTemplate.cfm?AID=3611).
Prediction markets are where populations of people buy and sell stocks representing the possible answers to questions. When addressing effective questions and ranges of answers, a diverse group of people seem to often be better at many types of prediction than individuals or small groups of like-minded, single-discipline employees.
Prediction markets may be able to react and adapt to changing environments more effectively than do small groups of professional specialists.
Most prediction markets incorporate broad data transparency and stakeholder anonymity, circumventing office politics, spin, organizational stovepipes, and more.
Prediction markets appear to offer cost-effective opportunities for collaboration on risk analysis across organizational boundaries.
Setting up and managing prediction markets would force us to ask specific questions about risk. For many participants this would be a powerful way to transform risk fron an abstract concept to something meaningful to their everyday involvement in company operations.
Prediction market outputs might initially augment existing risk management practices.
Why can’t we seem to accurately or efficiently identify probabilities of loss associated with given risks, or to effectively prioritize our investments in mitigating specific risks? The MIT Center for Collective Intelligence reports that there are a number of forces that might be relevant. They include:
- Biases involving fixating on some initial fact, group favoritism, herd behavior, polarization, or a tendency to accept risk in familiar situations or in groups can degrade the quality of risk analysis.
- The shared emotional connection that often comes with membership in a group can also lead to bias.
- Failure to comunicate and retain contextual information across geographically distributed teams.
- Unevenly distributed information.
- Differences in the speed of access to information.
- Difficulty in interpreting the meaning of silence.
- Difficulty in achitving agreement when diverse viewpoints exist.
- Agreeing from different perspectives.
- Sensitivity to rank, and an accompanying tendency for some to do what they are told.
- Ineffective communications across cultural boundaries.
- The propensity for an individual’s self interest to outweigh the potential gain of collective action.
- Weaknesses in the implementation of collaboration infrastructure and operations.
- Overt manipulation by insiders or elites.
- Fixing on extremes.
- Overly restrictive or unclear specifications.
(See the Handbook of Collective Intelligence)
New communication and collaboration technogies can enable much larger and diverse groups to work together in ways that have not been possible before. The MIT group argues that this enables a vast expansion of the phenomenon of “collective intelligence.” The key characteristics include:
- Diversity. (The lack of diversity increases the bias in collective decision-making.)
- Formal and informal structure.
- Modularization of tasks.
- Dense communications structure.
- Incentives for contribution.
- Shared vocabulary and other infrastructure.
- The “Power of the Edge,” or better opportunities for bottom-up information flow from the edges of an organization.
- The “Power of an Ecosystem” to identify interesting features in the external environment, where one highly regimented view often limits situation awareness.
This list of factors appears to off-set many of the problems that seem to be key to degrading our risk analysis capabilities.
Maybe there is a role for collective intelligence and prediction markets in corporate information security and broader technology infrastructure and operations risk management. Have you been using or experimented with prediction markets in your risk analysis efforts?
— References —
International Organization for Standardization, ISO 9000 Table of Contents: http://www.iso.org/iso/iso9000_2007_toc.pdf
Taking The Pulse of Your Company.” by Adam Siegel: http://www.rmmag.com/Magazine/PrintTemplate.cfm?AID=3611
MIT Center for Collective Intelligence: http://cci.mit.edu/
Handbook of Collective Intelligence: http://scripts.mit.edu/~cci/HCI/index.php?title=Main_Page