Conversations that Count: Lessons From Evaluating a Men’s Digital Mental Health Response During COVID-19

Abstract

This paper details disruption and innovation in digital evaluation practice at Movember, as a result of the COVID-19 pandemic. The paper examines a men’s digital health intervention (DHI) – Movember Conversations – and the product pivot that was necessary to ensure it could respond to the pandemic. The paper focuses on the implications of the pivot for the evaluation and how the evaluation was adapted to the COVID-19 exigencies. It details the redesign of the evaluation to ensure methods wrapped around the modified product and could deliver real-time, practical insights. The paper seeks to fill knowledge gaps in the DHI evaluation space and outlines four key principles that support evaluation re-design in an agile setting. These include a user-centred approach to evaluation design, proportionate data collection, mixed (and flexible) methodologies, and agile evaluation reporting. The paper concludes with key lessons and reflections from the evaluators about what worked at Movember, to support other evaluators planning digital evaluations.