Value in Health                  
Vol 17 | No. 2 | March 2014 | Pages 141-306
http://www.valueinhealthjournal.com/current

A Questionnaire to Assess the Relevance and Credibility of Observational Studies to Inform Health Care Decision Making: An ISPOR-AMCP-NPC Good Practice Task Force Report  
Marc L. Berger, Bradley C. Martin, Don Husereau, Karen Worley, et al.
Abstract 
Evidence-based health care decisions are best informed by comparisons of all relevant interventions used to treat conditions in specific patient populations. Observational studies are being performed to help fill evidence gaps. Widespread adoption of evidence from observational studies, however, has been limited because of various factors, including the lack of consensus regarding accepted principles for their evaluation and interpretation. Two task forces were formed to develop questionnaires to assist decision makers in evaluating observational studies, with one Task Force addressing retrospective research and the other Task Force addressing prospective research. The intent was to promote a structured approach to reduce the potential for subjective interpretation of evidence and drive consistency in decision making. Separately developed questionnaires were combined into a single questionnaire consisting of 33 items. These were divided into two domains: relevance and credibility. Relevance addresses the extent to which findings, if accurate, apply to the setting of interest to the decision maker. Credibility addresses the extent to which the study findings accurately answer the study question. The questionnaire provides a guide for assessing the degree of confidence that should be placed from observational studies and promotes awareness of the subtleties involved in evaluating those.

Questionnaire to Assess Relevance and Credibility of Modeling Studies for Informing Health Care Decision Making: An ISPOR-AMCP-NPC Good Practice Task Force Report
J. Jaime Caro, David M. Eddy, Hong Kan, Cheryl Kaltz, et al.
Abstract 
The evaluation of the cost and health implications of agreeing to cover a new health technology is best accomplished using a model that mathematically combines inputs from various sources, together with assumptions about how these fit together and what might happen in reality. This need to make assumptions, the complexity of the resulting framework, the technical knowledge required, as well as funding by interested parties have led many decision makers to distrust the results of models. To assist stakeholders reviewing a model’s report, questions pertaining to the credibility of a model were developed. Because credibility is insufficient, questions regarding relevance of the model results were also created. The questions are formulated such that they are readily answered and they are supplemented by helper questions that provide additional detail. Some responses indicate strongly that a model should not be used for decision making: these trigger a “fatal flaw” indicator. It is hoped that the use of this questionnaire, along with the three others in the series, will help disseminate what to look for in comparative effectiveness evidence, improve practices by researchers supplying these data, and ultimately facilitate their use by health care decision makers.