Human performance and system safety: Alternative perspectives on safety assessment

Abstract

Discussion Panel Abstract: The recent Boeing 737MAX accidents crystalized for the public the complexity of anticipating system and operator performance and developing a system design that prevents catastrophic outcomes. The operational situations, progression of flightcrew actions, and system behaviors that led to the two accidents had not been anticipated by the manufacturer or the regulator. These accidents were only the most recent examples of our failure to anticipate and manage operational complexities and operator performance. The art and science of human factors has yet to perfect risk assessment (or safety assessment) for complex systems. In the not-so-distant past, system risk assessment made estimates of human error probabilities (HEPs) for specific operational tasks, which were combined with estimated equipment failure rates to produce an overall risk estimate. Indeed, these Human Reliability Analysis (HRA) techniques have evolved over decades and are still being developed (e.g., IDHEAS-ECA, Xing et al., 2020), partly because they satisfy the need for a simple quantitative threshold that can be used by industry and regulators: if risk probability is too high, change the design or some other aspect of operations. Through the years, there have been critiques of the HRA approach (e.g., Hollnagel, 1998) that led to revisions, such as focusing on cognitive functions instead of operator tasks, but not to the basic quantitative risk-estimation approach. Other approaches to assessing risk/safety have wandered down other paths: attempting to capture system complexity from an operator’s perspective (Roth, Mumaw, Lewis, 1994; Nuclear Regulatory Commission, 2000), or better documenting the many ways in which system operators manage complexity daily to find ways to improve their capacity (Hollnagel, Woods, & Leveson, 2006). These approaches have used different measures than HEPs; e.g., measures of operator performance, measures of interface usability/design, measures of task complexity, and the analysis of system constraints. In this panel, we offer different perspectives on risk/safety assessment as it relates to operator performance in complex systems. Foundational to assessment is deciding the nature of safety and the role of operator performance. Another important question is, as you move away from simple quantitative measures, how do you establish safety thresholds? That is, what guidance can we give to industry and regulators regarding how to measure safety and how to decide that action is required on the basis of safety.