Quality of Reporting in Systematic Reviews of Implantable Medical Devices

Over the past decade, advances in the technologies used in medical devices have profoundly transformed clinical practice and patient management. According to a recent position paper, studies of medical devices typically do not address in sufficient detail study design features and factors that affect safety and effectiveness, including standardization of outcomes and endpoints, evaluation of device-operator interactions, and evaluation of characteristics of the clinical practice setting. The evaluation of medical devices poses a number of methodological challenges, and empirical research evaluating such devices has not been performed. The challenges in conducting assessments of medical devices are secondary to the rapid evolution of technology, practitioner input into design, operator learning curve, and variation in the skill with which they are used. These features can potentially impact study outcomes. Systematic reviews have an established role in medical practice and research. Clinicians use systematic reviews to keep abreast of current research and to provide information on the effectiveness of competing interventions. Agencies including the Centers for Medicare and Medicaid in the United States and the National Institute of Clinical Excellence in the United Kingdom use systematic reviews to prioritize funding or coverage decisions. Health care organizations use systematic reviews to shape policy and inform clinical practice guidelines. Despite there being many published systematic reviews of medical devices, no empirical evaluation of the reviews has been performed. A thorough empirical appraisal of systematic reviews is needed to critically assess current practices and identify issues and gaps in reporting. The information generated from an empirical appraisal of systematic reviews of medical devices could be used to develop new items in reporting guidelines and improve the conduct and the quality of reporting of systematic reviews of medical devices. The strength of systematic reviews and meta-analyses relies in part on the clarity of the reporting. Reporting standards may serve to improve the quality of systematic reviews and ultimately to benefit evidence-based patient management and treatment decisions. There have been attempts to improve the general quality of reporting of systematic reviews through guidelines. Current reviews indicate that adherence to existing systematic review reporting guidelines has generally been low in systematic reviews published in high-impact journals or leading medical journals. Existing systematic review reporting guidelines include the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and Meta-analysis of Observational Studies in Epidemiology (MOOSE) guidelines. While these represent consensus guidelines to improve the quality of reporting in systematic reviews in general, they do not provide guidance for reporting or analyses of variables unique to the field of medical devices. Standardized guidance for researchers conducting systematic reviews on device-related topics could benefit the users of these reviews. The following Key Questions were formulated in consultation with the TEP and AHRQ. For each of the proposed Key Questions, with input from the TEP, we operationalized our analysis by creating specific items that could be answered by the systematic reviews. 1. How are items such as literature searches, study selection, and results reported in published systematic reviews of implantable medical devices? 2. How do published systematic reviews of implantable medical devices report device- or operator-specific information? 3. How is the reporting of heterogeneity handled in published systematic reviews of implantable medical devices? 4. What are the limitations and issues related to reporting of the quality and generalizability in published systematic reviews of implantable medical devices?