New England Journal of Medicine
February 10, 2011  Vol. 364 No. 6
http://content.nejm.org/current.shtml

Perspective
Developing the Sentinel System — A National Resource for Evidence Development
R.E. Behrman and Others

[Free Full-Text]
The Food and Drug Administration (FDA) now has the capacity to “query” the electronic health information of more than 60 million people, posing specific questions in order to monitor the safety of approved medical products. This pilot program, called Mini-Sentinel, uses a distributed data network (rather than a centralized database) that allows participating health plans and other organizations to create data files in a standard format and to maintain possession of those files. These organizations perform most analyses of their own data by running computer programs distributed by a coordinating center, and they provide consistent summarized results for the FDA’s review.1 The principles and practices involved in this effort to improve the safety of medical products can inform other uses of electronic health information to answer additional important questions about health and health care.

When the FDA announced the Sentinel Initiative in May 2008, it established a vision and objectives for the program, including the development of the Sentinel System, which will eventually be able to search the electronic health data of a minimum of 100 million patients.2 Laying the groundwork for that system has required an extraordinary range of input from public and private organizations. Under a cooperative agreement with the FDA, the Engelberg Center for Health Care Reform at the Brookings Institution has been convening an ongoing series of discussions among stakeholders to address the near- and long-term challenges inherent in implementing the Sentinel System.3 In 2009, the FDA gave the Harvard Pilgrim Health Care Institute the lead role in fulfilling a 5-year contract to establish a system — the Mini-Sentinel — for developing and testing approaches and methods that could be used to inform the structure and operations of the full Sentinel System. The institute is now leading a diverse partnership of approximately 200 epidemiologists, clinical content experts, statisticians, and data specialists from 27 institutions that are participating in this pilot system (www.minisentinel.org).

Through the Mini-Sentinel, capabilities are being developed for actively monitoring the safety of approved medical products using the electronic health information in claims systems, inpatient and outpatient medical records, and patient registries. The Mini-Sentinel builds on the work of the Vaccine Safety Datalink project (managed by the Centers for Disease Control and Prevention), the HMO Research Network, the Population Medicine Distributed Research Network (PopMedNet, funded by the Agency for Healthcare Research and Quality), and the Observational Medical Outcomes Partnership, among others.4

In the first year of the Mini-Sentinel project, its leaders established a network of data partners and a system with robust patient-privacy policies that could be used in querying the network’s databases. The initiative’s distributed data network allows each data partner to maintain physical and operational control over its own patient-level data, while providing the aggregated information needed to address the FDA’s questions. Source data reside behind the data partners’ institutional firewalls, where they are transformed into a standard format. This approach allows each data partner to answer the FDA’s queries by executing standardized computer programs distributed by the Mini-Sentinel Operations Center. A typical result might include the number of new users of a product who experience a particular outcome, grouped according to age, sex, other treatments, and health status. This use of distributed analysis — whenever possible — eliminates or greatly reduces the exchange of protected health information. The data partners can obtain full-text medical records when necessary to confirm diagnoses or exposures and to determine the existence or severity of risk factors.

The initial focus of Mini-Sentinel has been on developing the ability to use claims data. In the next year, laboratory-test results and vital signs, derived from electronic health records and clinical laboratory records, will be added. The partnership is also evaluating procedures whereby Mini-Sentinel data partners will be able to link to data held by other organizations, such as state immunization registries and device registries.

The FDA will soon begin to actively monitor the data, seeking answers to specific questions about the performance of medical products, such as the frequency of myocardial infarction among users of oral hypoglycemic agents (a topic selected because it has been difficult to identify drug-induced myocardial infarction through existing prospective surveillance mechanisms). The FDA will also monitor the occurrence of adverse events associated with select routinely administered vaccines. Using the Mini-Sentinel system, the FDA will also be able to obtain rapid responses to new questions about medical products and, eventually, to evaluate the health effects of its regulatory actions. This monitoring portfolio will expand as the FDA and its collaborators acquire experience and develop operational efficiencies and as additional data resources become available.

The distributed-database-and-analysis model and the infrastructure of the Mini-Sentinel data network can be extended to other forms of evidence development. Provisions in the economic stimulus and health care reform legislation, and a recent report from the President’s Council of Advisors on Science and Technology,5 envision expanded use of electronic health information for other types of public health surveillance, quality measurement, comparative effectiveness research, and biomedical research — all of which are essential to improving the country’s health and health care delivery system.

Issues relevant to other secondary uses of electronic health information include recruitment of appropriate data partners, development and refinement of analytic methods, implementation of standards to ensure that analytic methods are consistent across the data sources, and above all, protection for the rights and privacy of patients. Data privacy and security are top priorities that were key considerations in the decision to build Mini-Sentinel as a system that uses a distributed data system and distributed analysis whenever possible. The committed collaboration among representatives of patients and consumers, health care professionals, Mini-Sentinel’s data partners and safety scientists, and the medical-products industry has been essential to the Sentinel Initiative’s progress.

It is particularly challenging to establish appropriate governance for a distributed data network that can support multiple secondary uses for health information. The current infrastructure is supported by a single federal agency, the FDA, and all the data are provided by private organizations, yet potential users of such a system reside not only broadly in government but also in academia, the private sector, and other user communities. To facilitate the development of this infrastructure into a national resource, this distributed system may ultimately be best managed by a consortium of interested parties operating as a public–private partnership. For example, specialized network-coordinating centers might rely on a consistent infrastructure to use the same sources of health information for various purposes, including public health uses, effectiveness research, quality measurement, and health services research.

The envisioned Sentinel System will build on the knowledge, partnerships, data resources, privacy protections, and technical capabilities that are being developed in the Mini-Sentinel program. Success in the form of improved safety of medical products will depend on the continued engagement of all concerned stakeholders and on ensuring that patients, consumers, and health care providers understand that all medical products pose risks and that postmarketing surveillance is critical to expanding the limited evidence base that exists when products are approved. Success also depends on the continued development of surveillance methods and on increasing the workforce of scientists who are trained to develop and interpret this evidence effectively.

Health care data represent a precious resource that must be used to the fullest possible extent to promote the public health, while the rights of patients and consumers are protected. As an early working model for secondary uses of data produced in the routine delivery of health care, the Sentinel System can and should become a national resource for evidence development and a cornerstone of a learning health care system.

This article (10.1056/NEJMp1014427) was published on January 12, 2011, at NEJM.org.