Normal Accidents

Normal Accidents

Charles Perrow is regarded as a pivotal figure in the theory of why and how things fail. He served as a sociology professor at Yale and Stanford and was primarily focused on the influence of large organisations on society – his most referenced work is “Complex Organisations: A Critical Essay,” released in 1972. 

In this newsletter, we’re exploring ideas from the excellent book that he wrote whilst examining the causes of the Three Mile Island accident in 1979: “Normal Accidents: Living with High-Risk Technologies”. This was published in 1984, and effectively predicted the Chernobyl Disaster of 1986.

Perrow wrote, “If interactive complexity and tight coupling —system characteristics — inevitably will produce an accident, I believe we are justified in calling it a normal accident, or a system accident.”

He introduced the concept of “Normal Accidents“, which he also refers to as system accidents. These are near-inevitable catastrophic failures in highly complex and tightly coupled systems. In these complex systems, despite efforts to prevent them, component failures arise and can interact unpredictably with other components, particularly if components are dependent upon each other. Perrow also argues that many measures intended to bolster system resilience can inadvertently increase its complexity, leading to new potential failure states. This phenomenon is prevalent in software and technology (I’ve experienced and been responsible for it myself); efforts to enhance system robustness – through added redundancy, availability checks, failsafe mechanisms and protections – can actually render them more brittle. This added complexity can also impede prompt diagnosis of complex failures, making them not only more likely, but harder to resolve.

Perrow points out that mistakes made by human operators aren’t the sole cause of accidents. Many failures stem from organisational issues rather than technological ones, and major accidents often start with seemingly trivial events. These small incidents can unpredictably snowball through complex and tightly coupled systems, leading to disaster. He argued that technological failures weren’t just isolated incidents of malfunction, human error, or bad luck. Instead, they’re often the result of complex, interdependent systems, with organisational, cultural, and management factors playing a significant role.

Essentially, we can’t simply blame a machine breaking down, a person making a mistake, or an unforeseeable event for a disaster. Technology disasters are far more complex than that.

Complexity, Coupling and Catastrophe

Perrow outlines three conditions that make any system more prone to suffering a “Normal Accident”:

• The system is highly complex: Having lots of components with unpredictable interactions increases the likelihood of failures.
• The system is tightly coupled: If elements within the system are closely connected and dependant, a failure in one part has a “prompt and major” impact on others.
• The system has catastrophic potential: When the stakes are high, a failure can lead to severe consequences. 

In the book, Perrow warned of a likely major nuclear catastrophe. He attributed this risk to the intricate complexity and closely interlinked components of nuclear reactors. When these components fail, they can interact unpredictably, causing failure cascades that are challenging to comprehend, particularly under time pressure. Perrow even noted “unless we are very lucky, one or more [serious nuclear accidents] will appear within the next decade and breach containment“. He only needed to wait two years before his prediction came true with the Chernobyl disaster in 1986. Subsequent disasters such as Fukushima also followed the Perrowian script of high complexity combined with tight coupling of system components.

An illustration of Normal Accident Theory and the relationship between complexity and coupling, with various domains represented as examples. Adapted from Etkin and Timmerman, 2013.

How do we mitigate against Normal Accidents?

What can we learn from Perrow? There are a number of tactics that can mitigate the risk of catastrophe in this kind of system:

1. Reduce complexity. It’s very easy to add complexity to a system but much harder to make it simpler. For example, software engineers are often rewarded for adding features and functions, more rarely for removing them. Incentivise simplicity over complexity.
2. Reduce coupling and dependencies. Whether we’re dealing with technological systems or social systems, ensure that if one component fails, others can continue to function as normal or in a safe-but-degraded state. 
3. Move authority to where the information is. People and teams who are empowered to make rapid decisions in high-pressure situations will be able to mitigate and prevent catastrophe far more effectively than if they need to escalate decisions up a hierarchy.
4. Create environments of psychological safety. Where people feel safe to raise concerns about potential risks, failures, and ideas for improvement, both the technological and social systems will be more resilient to shocks and less likely to “drift into failure”.

Drift Into Failure

Sidney Dekker later built on the idea of Normal Accidents with his concept of “Drift into Failure” (DIF) which describes a slow and gradual descent into catastrophe. Driven by production pressure, competition and scarcity, organisations find themselves inching closer to their safety limits, and social patterns can make taking growing risks seem normal. The incentive to push the boundaries can often overshadow the lurking and growing dangers. An example is the “normalisation of deviance” that led to the Challenger disaster – (see Vaughan, 1996). The Drift Into Failure concept suggests that organisations may end up failing not because something goes wrong, but rather because they appear to be performing well, and gradually adopting (or simply maintaining) high-risk practices without realising the potential for disaster. This drift into disaster isn’t a fluke: it’s a natural outcome of how sociotechnical systems function. 

Humans are the true adaptive element of a system

The principles outlined by Perrow are evident in aviation. Even as aircraft technical reliability improved, it didn’t necessarily lead to fewer accidents (as Hagen highlighted in 2013). Perrow argued that depending solely on technical fixes for safety simply increases the complexity that humans have to navigate. This increased complexity, combined with tightly coupled components, amplifies the potential for catastrophic failures. In reality, it is only humans that can adequately prepare and respond to disasters. Alongside technological developments, the advent of Crew Resource Management and Human Factors training for air and ground crew is ultimately what has made aviation so safe over the past few decades. 

In the safety domain, Perrow wasn’t regarded as “one of us” (due in significant part due to his anti-nuclear power stance), but his ideas have stood the test of time. His theories hold particular relevance for today’s global technological systems and cloud computing – which were preceded by Perrow’s theories on complexity and coupling. The fields of resilience engineering, human factors, and learning from incidents also possess strong Perrowian foundations.

Perrowian Relevance Today

As we develop ever-increasing complexity and interconnectedness in our organisations and world in general, the insights of Charles Perrow’s “Normal Accidents” have never been more relevant. We should work to simplify systems, empower people at the sharp end of work, and create cultures that foster psychological safety and open communication. Perrow’s work continues to remind us that the prevention of catastrophe doesn’t only lie in machinery and technology, but in the very fabric of how we communicate with each other and how we organise, manage, and understand the systems we create.

References:

Dekker, S. Drift into failure, 2014. https://safetydifferently.com/wp-content/uploads/2014/08/SDDriftPaper.pdf

Etkin, D. and Timmerman, P., 2013. Emergency management and ethics. International Journal of Emergency Management, 9(4), pp.277-297. https://www.researchgate.net/publication/264812137_Emergency_management_and_ethics

Hagen, J. Confronting Mistakes, 2013. Lessons from the Aviation Industry when Dealing with Error. https://amzn.to/3E8P3FW

NASA. The Cost of Silence: Normalization of Deviance and Groupthink, 2014. https://sma.nasa.gov/docs/default-source/safety-messages/safetymessage-normalizationofdeviance-2014-11-03b.pdf

Perrow, C. 1984. Normal Accidents – Living with High Risk Technologies https://amzn.to/3QRm40S

Price, M.R. and Williams, T.C., 2018. When doing wrong feels so right: normalization of deviance. Journal of patient safety, 14(1), pp.1-2. https://pubmed.ncbi.nlm.nih.gov/25742063/

Vaughan, D. 1996. The Challenger Launch Decision – Risky Technology, Culture, and Deviance at NASA: Risky Technology, Culture, and Deviance at NASA https://amzn.to/3QPRNPS 

More Psychological Safety Workshops!

We have places left on our upcoming workshops in October and November – you can choose from:

• Intermediate: covering the essential theory and practice to help you level up your knowledge and practice.
• In-depth: covering cultural impacts on psychological safety, complexity, sociotechnical and more.
• Practices: From Empathy Mapping to TRIZ. Practical things to do with groups and teams.
• Leadership: For managers and senior leaders: we cover effective management practices, senior leadership, reporting, 1-1s, work design and flow, managing neurodiverse people and much more. 

 Find out more and register here.

Psychological Safety at work

We cannot fully discuss psychological safety without discussing intersectionality and privilege. Here’s a fantastic guide to intersectionality from UN Women. This guide provides a fantastic description of intersectionality, a description of various enablers and how to apply them, with clear examples and instructions. 

AI and LLM tools are have taken over workplaces in a very short period of time. Many of us now use these tools to help with all sorts of tasks, from proof-reading to translation, and data analysis to research design. However, accuracy and quality issues notwithstanding, these tools are subject to significant biases, as this video from LIS, The London Interdisciplinary School shows. Of course, the algorithms and models are only returning the data they’ve been given, but the scale and speed that they’re being adopted means these biases are only being calcified into our practices and made worse by our tools, not better.

Business leaders have historically drawn inspiration from military strategies but have overlooked lessons from non-violent social and political movements. This excellent article by Greg Satell emphasises the importance of learning from both failures and successes, understanding multiple perspectives, and developing strategies to overcome resistance to change. 

Here’s a great podcast episode from the folks at BBC Analysis: “Does work have to be miserable?”. In this, Pauline Mason considers the impact the pandemic had on the workplace, how suddenly many of us transitioned to different (and often better) ways of working, and how “job design” focuses on people, their skills, their knowledge and how they interact with each other and technology in the workplace. “Good work is about much more than money.“

Finally, I’m just going to leave this here:

New stickers!

There are three ways that you can get your hands on some of our psychological safety stickers!

1. Buy some at our online shop
2. Attend a workshop
3. Promote psychsafety.com on social media and send us your postal address!

This week’s poem:

“From the soul’s point of view, you come to appreciate that each one of us is living out his or her own karma. We interact together, and those interactions are the grist for each other’s mill of awakening. From a personality point of view, you develop judgment, but from the soul’s point of view, you develop appreciation. This shift from judging to appreciating — to appreciating yourself and what your karmic predicament is, and who other beings are with their own karma — brings everything into a simple loving awareness. To be free means to open your heart and your being to the fullness of who you are because only when you are resting in the place of unity can you truly honour and appreciate others and the incredible diversity of the universe.”

From “Polishing the Mirror: How to Live from Your Spiritual Heart” by Baba Ram Dass