The origins of unpredictability in life outcome prediction tasks

Abstract

Why are some life outcomes difficult to predict? We investigated this question through in-depth qualitative interviews with 40 families sampled from a multidecade longitudinal study. Our sampling and interviewing process was informed by the earlier efforts of hundreds of researchers to predict life outcomes for participants in this study. The qualitative evidence we uncovered in these interviews combined with a mathematical decomposition of prediction error led us to create a conceptual framework. Our specific evidence and our more general framework suggest that unpredictability should be expected in many life outcome prediction tasks, even in the presence of complex algorithms and large datasets. Our work provides a foundation for future empirical and theoretical work on unpredictability in human lives.