Ep 112 How biased are incident investigators?

Today’s paper, “Multiple Systemic Contributors versus Root Cause: Learning from a NASA Near Miss”  by Katherine E. Walker et al, examines an incident wherein a NASA astronaut nearly drowned (asphyxiated) during an Extravehicular Activity (EVA 23) on the International Space Station due to spacesuit leakage. The paper introduces us to an innovative and efficient technique developed during Walker’s PhD research. In this discussion, we reflect on the foundational elements of safety science and how organizations are tirelessly working to unearth better methods for analyzing and learning from safety incidents. We unpack the intricate findings of the investigation committee and discuss how root cause analysis can sometimes lead to the unintended consequence of adding more pressure within a system. A holistic understanding of how systems and individuals manage and adapt to these pressures may provide more meaningful insights for preventing future issues.Wrapping up, our conversation turns to the merits of the SCAD technique, which champions the analysis of accidents as extensions of normal work. By examining the systemic organizational pressures that shape everyday work adaptations, we can better comprehend how deviations due to constant pressures may lead to incidents. We also critique current accident analysis techniques and emphasize the importance of design improvement recommendations. Discussion Points:History and current state of accident investigationSystemic solutions in safetyTraditional root cause analysis challenged by new perspectivesNASA's 2013 EVA 23 space walk incident examinedOrganizational pressures and their impact on safetySCAD technique for accident analysis efficiencyShift from tracing causes to understanding work adaptationsEmphasis on normal work analysis for accident preventionCritique of NASA's administrative processes in safetyCognitive biases and challenges in accident investigationsContinuous evolution of safety practices Practical takeaways -how do you go beyond the immediate events to find broader systems and broader learnings?Canging language away from causes to talk about pressures and contributorsThe answer to our episode’s question is, “Yeah, it probably helps, but still doesn't fix the problem that we're facing with trying to get useful system changes out of investigations.”Quotes:“We've been doing formal investigations of accidents since the late 1700s early 1800s. Everyone, if they don't do anything else for safety, still gets involved in investigating if there's an incident that happens.” - Drew“If you didn't have this emphasis on maximising crew time they would have been much more cautious about EVA 23” - Drew“Saying that there's work pressure is not actually an explanation for accidents, because work pressure is normal, work pressure always exists.” - Drew“One of the things that is absent from this technique through and they call it an accident analysis method is there is no commentary in the paper at all about how to design improvements and recommendations.” - DavidResources:The Paper: NASA Near MissThe Safety of Work PodcastThe Safety of Work on LinkedInFeedback@safetyofwork

Using the Waterfall incident as a striking focal point, we dissect the investigation and its aftermath, we share personal reflections on the implementation of safety recommendations and the nuances of assessing systems designed to protect us. From the mechanics of dead man's systems to the critical evaluation of managerial decisions, our dialogue exposes the delicate balance of enforcing safety while maintaining the practicality of operations. Our aim is to contribute to the ongoing conversation about creating safer work environments across industries, recognizing the need for both technological advancements and refined human judgment.  Discussion Points:Drew loves a paper with a great nameThe circumstances surrounding the Waterfall rail accidentHow the “dead man system” works on certain trainsRecommended changes from investigation committeesIn the field of safety, we seem more certain about our theoriesExploration of narratives and facts in accident investigationsDead man's system and Waterfall derailment's investigationPost-accident list of operator failuresSafety theories and organizational fault correlation critiquedEvolution of railway safetyDiscussion on managerial decisions amidst imperfect knowledgeThe importance of context in incident investigationsSafety management systems and human judgmentInsights on enhancing organizational safetyTheoretical conclusionsPractical takeawaysThe answer to our episode’s question is, “yes, keep it in mind as a digital tool” Quotes:“I find that some of the most interesting things in safety don't actually come from people with traditional safety or even traditional safety backgrounds.”- Drew“Because this is a possible risk scenario, on these trains, we have what's called a ‘dead man system.” - David“Every time you have an accident, it must have objective physical causes, and those physical causes have to come from objective organisational failures, and I think that's a fairly fair representation of how we think about accidents in safety.” - Drew“They focused on the dead man pedal because they couldn't find anything wrong with the design of the switch, so they assumed that it must have been the pedal that was the problem” - DrewResources:The Paper: Blaming Dead MenThe Safety of Work PodcastThe Safety of Work on LinkedInFeedback@safetyofwork

Using the paper, “Digital Twins in Safety Analysis, Risk Assessment and Emergency Management.” by Zio and Miqueles, published in the technical safety journal, Reliability Engineering and System Safety, we examine intricate simulations that predict traffic flows to emergency management tools that plan safe evacuation routes, and we delve into how these virtual counterparts of physical systems are redefining risk assessments and scenario planning.As we navigate the world of operational safety, we discuss the diverse array of models—from geometric to sophisticated hybrid simulations—and their groundbreaking applications in forecasting fire spread and optimizing evacuation procedures. These digital twins aren't just theoretical concepts; they're powerful, real-time lifesavers in emergency situations, emblematic of the future of safety science.  Discussion Points:What are digital twins and how are they used?Use of digital twins is de rigueur in traffic flow, fire engineering, water flow structureIdentifying all recent papers written on digital twinsVirtual simulations offer advanced risk assessment capabilitiesOverview of tasks and functions identified, industries - construction, naval engineering, manufacturingTechnical discussion on digital twin creation and maintenanceSix key challenges of digital twinningSmart paint innovation improves virtual model accuracyCybersecurity risksReal-time operational safety monitoringDigital twins promise improved safety and operational efficiencyEmergency management potentially bolstered by real-time simulationsPractical takeawaysIndustry practice may surpass academic digital twin findingsThe answer to our episode’s question is, “yes, keep it in mind as a digital tool” Quotes:"Ideally, a digital twin is a complete virtual copy of a product or service that is an electronic simulation that is completely accurate compared to that real product or service.”- Drew“One of the first documented digital twins was in the aerospace industry -  NASA [used it] during the Apollo 13 program.” - David“this idea of having a complete digital picture of the thing that you're building is becoming fairly common, so that  lends itself very much towards using it for things like digital twins.” - Drew“we may never quite know exactly how different the digital twin is from the physical object itself. That’s the challenge.” - David Resources:The PaperThe Safety of Work PodcastThe Safety of Work on LinkedInFeedback@safetyofwork

Ben's expertise guides us through an analysis of audit reports and accident investigations, laying bare the counterfactual reasoning that often skews post-incident narratives. It's an eye-opening examination that calls for a reimagined approach to audits, one that aligns with the genuine complexities of organizational culture and safety. Together, we confront the silent failure of safety audits and management systems, debating the need for a fundamental shift in how these are designed and conducted to truly protect workers. Join us for this critical dialogue that challenges preconceptions and seeks to reforge the link between safety audits and the real work of keeping people safe. Discussion Points:Ben’s background and papers authoredThe reality of safety auditsSafety plans - often perceived as comfort, not changeDocumentation versus actual safety"Audit masquerade" reveals gaps between theory and practiceExamination of 327 audit corrective actions and their efficacyAdministrative intentions vs. practical safetyThe weak connection between audits and physical safety improvements Concerns about evaluating work practices Audits can ‘fail silently’, giving false security assurancePractical steps for ensuring audit effectivenessA critical examination of safety management systemsKey takeawaysBen’s answer to our episode’s question is, “Possibly, but it depends.” Quotes:"Some audits were very poorly calibrated to actually exploring and eliciting work - day-to-day work and operational risk.." - Ben“you've got to pick and choose what to pay attention to. So unless something is really standing out as needing attention, then it can be hard to be curious and to notice these weak signals.” - David“I'm proud to work in safety. I'm proud to call myself a sector professional. What really drove me to understand these systems was my love for safety, and I had just become so disillusioned with the amount of safety work I had to do. It wasn't helping.” - Ben“Audits, like most activities, are very socio-political. There's a lot of vested power and conflicting interests.” - Ben Resources:Paper: Audit masquerade: How audits provide comfort rather than treatment for serious safety problemsPaper: How audits fail according to accident investigations: A counterfactual logic analysisBen Hutchinson LinkedInThe Safety of Work PodcastThe Safety of Work on LinkedInFeedback@safetyofwork

Safety isn't one-size-fits-all, especially for subcontractors who navigate multiple sites with varying rules and equipment. This episode peels back the layers on the practical safety management challenges subcontractors endure, revealing how transient work complicates the integration of safety protocols. We scrutinize the institutional oversights and fragmented safety systems that often overlook the needs of these critical yet vulnerable players in the industry. Our conversation isn't just about identifying problems; it's an urgent call to action for better practices and a safer future for all involved in subcontracting work. Discussion Points:The vagaries of subcontracting workBackground on the paper being discussedFindings presented in the paperInstitutional safety vs. the subcontractor’s workExpertise in the work does not equal expertise in safetyCommunication and safety work activitiesInstitutional safety mechanismsDangerous environments and lack of safety knowledge in that environmentSubcontractors in the mining industry and the many layers and risksSafety rules are perceived differently by subcontractorsFinancial and other burdens to following safety protocols for subcontractorsKey takeawaysThe answer to our episode’s question –the short answer in some of it is that there are lots of filtered and missing communication towards contractors' gaps in situational specific expertise that don't get identified and just our broad safety management systems and arrangements that don't work well for the subcontractor context. Quotes:"Subcontracting itself is also a fairly undefined term. You can range from anything from large, labour -higher organisations to what we typically think in Australia of a small business with maybe one to four or five employees." - Drew “All of the normal protections we put in place for safety just don't work as well when there are contract boundaries in place.” - Drew“the subcontractor may be called in because they've got expertise in a particular type of work, but they're in an environment where they don't have expertise.” - Drew Resources:Link to the PaperThe Safety of Work PodcastThe Safety of Work on LinkedInFeedback@safetyofwork

Lastly, we delve into the role of leadership in addressing psychosocial hazards, the importance of standardized guidance for remote work, and the challenges faced by line managers in managing remote workers. We wrap up the episode by providing a toolkit for managers to effectively navigate the challenges of remote work, and highlight the need for tailored safety strategies for different work arrangements.  Discussion Points:Different work-from-home arrangementsSafety needs of work from homeChallenges of remote worker representationUnderstanding and managing psychosocial risksLeadership and managing technical risksRemote work challenges and physical presencePractical takeaways and general discussionSafety strategies for different work arrangementsThe answer to our episode’s question – the short answer is that there definitely isn't a short answer. But this paper comes from a larger project and I know that the people who did the work have gathered together a list of existing resources and toolboxes and, they've even created a few prototype tools and training packagesQuotes:"There's a risk that we're missing important contributions from workers with different needs, neurodiverse workers, workers with mental health issues, workers with particular reasons for working at home and we’re not going to be able to comment on the framework and how it might affect them." - Drew “When organizations' number of incident reports go up and up and up and we struggle to understand, is that a sign of worsening safety or is that a sign of better reporting?” - David“They do highlight just how inconsistent organisations approaches are and perhaps the need for just some sort of standardised guidance on what is an organisation responsible for when you ask to work from home, or when they ask you to work from home.” - Drew“I think a lot of people's response to work from home is let's try to subtly discourage it because we're uncomfortable with it, at the same time as we recognise that it's probably inevitable.” - Drew Resources:Link to the PaperThe Safety of Work PodcastThe Safety of Work on LinkedInFeedback@safetyofwork

The conversation stems from a review of a noteworthy paper from the Academy of Management Review Journal titled "The Paradox of Stretch Goals: Organizations in Pursuit of the Seemingly Impossible," which offers invaluable insights into the world of goal setting in senior management. Discussion Points:The concept of seemingly impossible goals in organizationsControversial nature and impact of ‘zero harm’The role of stretch goals in promoting innovationPotential negative effects of setting stretch goalsPsychological effects of ambitious organizational targetsParadoxical outcomes of setting seemingly impossible goalsThe role of emotions in achieving stretch goalsFactors that contribute to the success of stretch goalsReal-world examples of successful stretch goal implementationCautions against blind imitation of successful stretch goal strategiesThe concept of zero harm in safety initiativesNeed for long-term research on zero harm effectivenessThe answer to our episode’s question – they're good when the organization is currently doing well enough, but stretch goals are not good when the organization is struggling and trying to turn a corner using that stretch goal. Quotes:"The basic idea [of ‘zero harm’] is that companies should adopt a visionary goal of having zero accidents. Often that comes along with commitment statements by managers, sometimes by workers as well that everyone is committed to the vision of having no accidents." - Drew “I think organizations are in this loop, where I know maybe I can't achieve zero, but I can't say anything other than zero because that wouldn't be moral or responsible, because I'd be saying it's okay to hurt people. So I set zero because it's the best thing for me to do.” - David“The “stretch goal” was credited with the introduction of hybrid cars. You've got to have a whole new way of managing your car to get that seemingly impossible goal of doubling your efficiency.”-  Drew Resources:Link to the PaperThe Safety of Work PodcastThe Safety of Work on LinkedInFeedback@safetyofwork

You’ll hear David and Drew delve into the often overlooked role of bias in accident investigations. They explore the potential pitfalls of data collection, particularly confirmation bias, and discuss the impacts of other biases such as anchoring bias and hindsight bias. Findings from the paper are examined, revealing insights into confirmation bias and its prevalence in interviews. Strategies for enhancing the quality of incident investigations are also discussed, emphasizing the need to shift focus from blaming individuals to investigating organizational causes. The episode concludes with the introduction of Safety Exchange, a platform for global safety community collaboration. Discussion Points:Exploring the role of bias in accident investigationsConfirmation bias in data collection can validate initial assumptionsReview of a study examining confirmation bias among industry practitionersAnchoring bias and hindsight bias on safety strategiesRecognizing and confronting personal biases Counterfactuals in steering conversations towards preconceived solutionsStrategies to enhance the quality of incident investigationsShifting focus from blaming individuals to investigating organizational causesSafety Exchange - a platform for global safety communityThe challenges organizations face when conducting good quality investigationsStandardization, trust, and managing time and production constraintsConfirmation bias in shaping investigation outcomesTechniques to avoid bias in accident investigations and improve their qualitySafety Exchange - a safe place for open discussionSix key questionsThe answer to our episode’s question – Very, and we all are as human beings. It does mean that we should probably worry more about the data collection phase of our investigations more than the causal analysis methodology and taxonomy that we concern ourselves with Quotes:"If we actually don't understand how to get a good data collection process, then it really doesn't matter what happens after that." - David "The trick is recognizing our biases and separating ourselves from prior experiences to view each incident with fresh eyes." - Drew"I have heard people in the industry say this to me, that there's no new problems in safety, we've seen them all before." - David"In talking with people in the industry around this topic, incident investigation and incident investigation quality, 80% of the conversation is around that causal classification taxonomy." - David Resources:Link to the PaperThe Safety of Work PodcastThe Safety of Work on LinkedInFeedback@safetyofwork

The research paper discussed is by Anita Tucker and Sarah Singer, titled "The Effectiveness of Management by Walking Around: A Randomised Field Study," published in Production and Operations Management.  Discussion Points:Understanding senior leadership safety visits and management walkaroundsBest practices for safety management programsHow management walkarounds influence staff perceptionResearch findings comparing intervention and control groupsConsequences of management inactionEffective implementation of changes Role of senior managers in prioritizing problemsImpact of patchy implementationHow leadership visits affect staff perceptionInvestigating management inaction Effective implementation and consultationKey Takeaways:The same general initiative can have very different effectiveness depending on how it's implemented and who's implementing itWhen we do any sort of consultation effort, whether it's forums, walkarounds, reporting systems, or learning teams, what do we judge those on? Do we judge them on their success at consulting or do we judge them on their success at generating actions that get taken?The answer to our episode’s question – Your answer here at the end of our notes is sometimes yes, sometimes no. It depends on the resulting actions. Quotes:"I've definitely lived and breathed this sort of a program a lot during my career." - David"The effectiveness of management walkarounds depends on the resulting actions." - David"The worst thing you can do is spend lots of time deciding what is a high-value problem." - Drew"Having the senior manager allocated really means that something serious has been done about it." - Drew"The individual who walks around with the leader and talks about safety with the leader, thinks a lot better about the organization." - David Resources:Link to the PaperThe Safety of Work PodcastThe Safety of Work on LinkedInFeedback@safetyofwork

The paper reviewed in this episode is from the Journal of Applied Psychology entitled, “A meta-analysis of personality and workplace safety: Addressing unanswered questions” by Beus, J. M., Dhanani, L. Y., & McCord, M. A. (2015). Discussion Points:Overview of the intersection between psychology and workplace safetyHow personality tests may predict safety performanceAccident proneness theory to modern behaviorismResearch on personality and safety performancePersonality traits influencing work behaviorsThe influence of institutional logicPersonality tests for safety performanceThe need for further research and standardized measurement methodsExamining statistical evidence linking personality to safety performancePersonality traits and their impact on work behaviorAnalysis of research findings on personality and safety performanceThe practical implications of the research findingsThe intriguing yet complex relationship between personality and safetyTakeaways:While not total bunk, we definitely don't understand the impact of personality on safety nearly enough to use it as a tool to predict who will or won't make a safe employeeThere are lots of different ways that we could use personality to get some insights and to make some contributionsWe need people using those measurements to find out more about the relationship between personality and behavior in different situations in different contexts with different choices under different organizational influences.The answer to our episode’s question – Maybe. It depends. Sometimes, in some places not yet. I don't want to say no, but it's not yes yet either. Quotes:I have to admit, before I read this, I thought that the entire idea of personality testing for safety was total bunk. Coming out of it, I'm still not convinced, but it's much more mixed or nuanced than I was expecting.  - DrewIf there was a systemic trend where some people were genuinely more accident prone, we would expect to see much sharper differences between the number of times one person had an accident and all people who didn't have accidents. - DrewI think anything that lumps people into four or five categories downplays the uniqueness of each individual. - DavidThere are good professionals in HR, there's good science in HR, but there is a huge amount of pseudo-science around recruiting practices and every country has its own pseudoscience. - Drew Resources:Link to the Paper The Safety of Work PodcastThe Safety of Work on LinkedInFeedback@safetyofwork