Quantifying Levels of Influence and Causal Responsibility in Dynamic Decision Making Events

Abstract

Intelligent systems support human operators’ decision-making processes, many of which are dynamic and involve temporal changes in the decision-related parameters. As we increasingly depend on automation, it becomes imperative to understand and quantify its influence on the operator’s decisions and to evaluate its implications for the human’s causal responsibility for outcomes. Past studies proposed a model for human responsibility in static decision-making processes involving intelligent systems. We present a model for dynamic, non-stationary decision-making events based on the concept of causation strength. We apply it to a test case of a dynamic binary categorization decision. The results show that for automation to influence humans significantly, it must have high detection sensitivity. However, this condition is insufficient since it is unlikely that automation, irrespective of its sensitivity, will sway humans with high detection sensitivity away from their original position. Specific combinations of automation and human detection sensitivities are required for automation to have a major influence. Moreover, the automation influence and the human causal responsibility that can be derived from it are sensitive to possible changes in the human’s detection capabilities due to fatigue or other factors, creating a “Responsibility Cliff.” This should be considered during system design and when policies and regulations are defined. This model constitutes a basis for further analyses of complex events in which human and automation sensitivity levels change over time and for evaluating human involvement in such events.