Development of the Individual Participant Data (IPD) Integrity Tool for assessing the integrity of randomised trials using individual participant data

Abstract

ABSTRACTIntroductionIncreasing concerns about integrity in medical research have prompted the development of tools to detect untrustworthy studies. Existing tools focus on evaluating aggregate or published data, though some trustworthiness issues may only be detected upon scrutiny of individual participant data (IPD). To address this, we developed the IPD Integrity Tool for detecting integrity issues in randomised controlled trials with IPD available. This manuscript describes the development of this tool.MethodsWe conducted a literature review to collate and map existing integrity items. These were discussed with an expert advisory group, and agreed items were included in a standardised tool and automated where possible. We piloted this tool in two IPD meta-analyses, and conducted preliminary validation checks on 13 datasets with and without known integrity issues in a blinded manner.ResultsThe literature review identified 120 integrity items: 54 could be conducted at the publication or aggregate data (AD) level, 48 required IPD, and 18 were possible with aggregate data, but more comprehensive with IPD. Based on these items, an initial reduced tool was developed in a consensus process involving 13 advisors with different backgrounds (countries, profession, education). This initial tool included 11 items across four domains for AD, and 12 items across 8 domains requiring IPD. The tool was iteratively refined throughout piloting on two IPD meta-analyses including a total of 116 trials (73 with IPD, and 43 with only AD available), and preliminary validation using an additional 13 datasets. All five studies with known integrity issues were accurately identified during validation. The final version of the tool included seven domains with 13 items for AD and eight domains with 18 items requiring IPD.ConclusionsThe quality of evidence informing health care relies on trustworthy data. This manuscript describes the development of a tool to enable researchers, editors, and other stakeholders to detect integrity issues in randomised trials using IPD. Detailed instructions on the application of this tool will be published subsequently.