---

Press the print button on your browser.
Click here to return to the previous page.

Report 3 of the Council on Scientific Affairs (I-02)
Full Text

Quality improvement projects and human subjects research

Note: This report represents information on this subject as of December 2002.

Full text

Resolution 807 (A-02), introduced by the Renal Physicians Association and adopted at the 2002 Annual Meeting, asks that the American Medical Association, (AMA) call upon:

1. The Office for Human Research Protections to develop clear guidelines to differentiate between quality improvement and human subjects research;
2. The Centers for Medicare & Medicaid Services (CMS) to develop a process to ensure that all QIPs (Quality Improvement Projects) and other studies performed by the End Stage Renal Disease Networks and other CMS contracted Quality Improvement Organizations are reviewed and certified as exempt studies under the federal regulations covering human subjects research protection; and
3. The Centers for Medicare & Medicaid Services to indemnify the volunteer members of the Medical Review Boards from responsibility for having participated in QIPs developed in accordance with CMS instructions that were not in compliance with federal regulations covering human subjects research protection; and that the AMA
4. Study the issue of the relationship between quality improvement projects and human subjects research and the potential impact of defining quality improvement projects as human subjects research on improving the quality of medical care by and within the private sector and issue a report at the 2002 Interim Meeting.

This report of the Council on Scientific Affairs (CSA) examines and offers recommendations on the relationship between QIPs and human subjects research.

Introduction

Human subjects research is critical to assess how advances in medical science impact patient care. Clinical quality improvement (QI) initiatives are critical to ensure that patients are receiving care consistent with current standards of medical practice. These two categories of activity may overlap, but they have notable differences. Human subjects research may produce knowledge with long-term significance, whereas QI initiatives are intended to provide quick feedback to practitioners. While research to yield new knowledge is conducted at the discretion of the investigator and may therefore be regarded as optional (from an individual perspective), the commitment to improve performance for the betterment of public health is fundamental to the principles of medical ethics and professionalism.^1,2 Quality improvement may thus be seen as obligatory for medical professionals. This view is consistent with the recent studies that have described the current state of health care in the United States,³ the urgent need for improvements in the quality of care,^4,5 and the potential effectiveness of QI initiatives.⁶

A discourse is under way in the professional literature on whether particular QI initiatives should be considered human subjects research, and therefore subject to review by an Institutional Review Board (IRB), and possibly, per federal regulations, require patient informed consent. Intensifying the discussion is the release of the Health Insurance Portability and Accountability Act (HIPAA) final Privacy Rule and the need for all parties to interpret this Act for their particular situations. Awareness of the need to assure protections for patients also is heightened by the 1999 case of a young research volunteer who died in a gene transfer trial, reportedly because he was not protected from unacceptable research risks.⁷ Also in 1999, the former Federal Office for Protection from Research Risks and the Food and Drug Administration (FDA) suspended the human research programs of several major research universities because of noncompliance with federal regulations.⁷

Defining the requirements for independent review and patient consent for QI initiatives is timely, given the continuous increase in clinical QI initiatives by individual physicians, physician groups, hospitals, and health plans, as well as quality improvement and accreditation organizations. The issue is relevant in both the public and private sectors. Professional journal editors also must grapple with the distinction in completing their own responsibilities, professional and ethical, to ensure that initiatives described in manuscripts submitted for publication have undergone appropriate human subjects protection review.

The distinction between human subjects research and QI initiatives can be blurry, at times; however, the discussion is evolving. To frame the issues, this report reviews definitions of research and QI, the current human subjects federal regulations, and the recent HIPAA Privacy Rule; describes current QI initiatives in both the public and private sectors; summarizes viewpoints of experts in the field; and proposes specific questions. Recommendations for developing a framework are provided.

Whether a QI activity is ultimately classified as human subjects research or not, an overarching consideration for all health care providers and organizations must be to ensure that the patient population is adequately protected from risks, whether those risks be to patients’ medical health or privacy. At the same time, these patient protections should not place undue restrictions on QI initiatives, which hold the potential to improve care.

Methods

Literature searches were conducted in PubMed/MEDLINE and the JAMA/AMA Archives for discussion and recommendations on the QI/human subjects research issues, using the search terms "quality improvement initiative," "quality improvement project," "human subjects research," and "Office for Human Research Protections." Correspondence pertaining to the ambiguous classification of QI projects of the End Stage Renal Disease (ESRD) Networks was reviewed.

Staff at the Office for Human Research Protections (OHRP), the Centers for Medicare and Medicaid Services (CMS), selected CMS Quality Improvement Organizations (QIOs) and private sector groups engaged in QI initiatives/research were contacted. Experts in medical ethics, research, and public policy were also consulted. Without exception, each individual was acutely aware of the issue of whether particular QI activities should be considered research and indicated that his or her respective organization is evaluating the issue and formulating its opinions.

Background

AMA Policy [Editor’s note: The discussion below reflects AMA policy at the time this report was written (December 2002.)]

Current AMA policy acknowledges the value of QI projects and the importance of adequate protections in human subjects research, but does not address the differentiation of QI projects that are research from those that are not. Policies H-340.902 and H-340.904 (AMA Policy Database) support and encourage professional involvement in CMS Quality Improvement Projects (QIPs) at the local and state levels. Policy H-460.916 encourages education in government regulations on human subjects research for all investigators and relevant staff at institutions conducting research.

The importance of safeguarding patient privacy and the confidentiality of personal medical data has been consistently upheld and reinforced in AMA policy. Policies H-315.978, H-315.983, H-315.984, and H-460.919 are particularly relevant to the proper use and disclosure of patient-specific data in research and QI initiatives.

Definitions

Definitions of research, human subject, and QI serve as a starting point for discussion. United States Department of Health and Human Services (DHHS) regulations and the Office for Human Research Protections (OHRP) define research as follows:

The DHHS regulations define a human subject as:

QI initiatives (which may or may not involve human subjects) have been defined elsewhere as:

These definitions leave room for varying interpretations. One author believes most QI activities are not generalizable and therefore should not be considered research.¹⁰ Another author suggests, "the results of virtually all QI interventions are generalizable, to lesser or greater degrees."⁹ This raises the question of whether the latter interpretation requires that all QI activities involving human data be considered human subjects research.

For example, the AMA-convened Physician Consortium for Performance Improvement (The Consortium) advocates that physicians implement Consortium flowsheets into their practices. A flowsheet is designed to collect patient-specific data (obtained primarily, although not exclusively, through interaction with patients), much like data collected in medical records. The flowsheet serves as both a data collection vehicle and an intervention to assist physicians in documenting and practicing evidence-based care. As part of a QI activity, a physician practice may implement the flowsheets without patient consent, considering the activity part of practice operations. However, with repeated measurements, the physicians also may evaluate the usefulness of flowsheets in improving care. Even though the physicians may not have originally intended their results to be generalizable beyond their practice, the results may be useful to other physician practices considering use of the flowsheets. If the physicians publish their work to inform others, using only aggregate data, are they placing patient privacy at risk? Should the patients have been informed a priori?

The definition for QI likewise is ambiguous in terms of the risk to patients. For example, "interventions linked to assessment" to improve "complex systems" may, in certain situations, present risks to a patient. For example, physicians in the United Kingdom evaluated 96,127 fetuses for nuchal-translucency thickness without patient consent. The authors contend that an ethics committee review was not necessary because their work was an audit of results from a clinical service, not a research study. Their work was published in Lancet, leading to professional and public debate.^11,12

To date, no attempt at wordsmithing the definitions of QI and human subjects research has resolved the challenge of differentiating one activity from the other. A comprehensive framework is needed.

Federal regulations

In the United States, federal regulations for the protection of human subjects apply to research that is conducted or supported by the DHHS or is conducted by an institution and subject to the DHHS regulations under an applicable assurance of compliance. Therefore, the regulations apply to all federally funded research studies and those privately funded research studies that are regulated by a federal agency such as the FDA.¹³ For example, pharmaceutical clinical trials, although mostly privately funded, are subject to FDA human subject protection regulations (Title 21 Code of Federal Regulation 56) . An institution may also agree voluntarily in its assurance of compliance to apply the regulations to non-federal research, which may include non-federally funded, non-FDA-regulated research.

The DHHS issued a revised version of the federal "Protection of Human Research Subjects" policy (Common Rule) in August 2001. Additional protections for pregnant women, human fetuses, and neonates involved in research (Subpart B) were added to the policy in December 2001.⁸

The Common Rule requires all covered, non-exempt studies to be submitted to an IRB for review and approval. This policy also defines six categories of exemption from IRB review. The determination of exemption may be made by an institution’s IRB or by some authority other than the investigator. The OHRP recommends that institutions adopt clear procedures for making these determinations.¹⁴ Listed below are five of the six exemptions (exemption 6 is not relevant to this discussion):

For research not determined to be exempt from IRB review, the Common Rule defines criteria by which the research may qualify for an expedited review procedure, which consists of a review by the IRB chair or by one or more experienced reviewers designated by the chair. Qualification for expedited review, in essence, indicates that the researchers have proven minimal risk and have an acceptable plan to ensure patient privacy within their research protocol and the entire course of the study. Additionally, non-exempt research – whether subject to expedited review or review by a full IRB – may receive a waiver from obtaining patient informed consent. Provided that other specific criteria are met, informed consent may be waived by the IRB for certain studies in which the IRB believes that patients would not be harmed by the project and that receiving informed consent would be too cumbersome and financially burdensome for the researchers.¹³ The research proposal is considered to be for the betterment of society, and the extra burden of obtaining consent would hinder the completion of the protocol.

A 1997 study in Minnesota, where a state law requires patient informed consent to access medical records for research, demonstrated the impact of securing consent on producing timely and valid research.¹⁵ The proposed study was part of an FDA post-marketing safety surveillance program to evaluate adverse events associated with a pain medication. It required retrospective review of medical records. Of 140 Minnesota health plan members asked to participate, 52% (73) responded and 19% (26) returned a signed consent form. For 132 plan members enrolled in states where consent is not required, health care providers granted access to medical records for 93% (123) of the members. The Minnesota legislature subsequently amended the state law to permit implied authorization for the use of medical records if the patient has not responded to a request for consent within 60 days following two good-faith attempts.

Institutional Review Boards (IRBs)

According to the Division of Assurances and Quality Improvement (DAQI) of the OHRP, the United States has 1,951 registered IRBs as of August of 2002.¹⁶ Some have suggested IRBs are already overwhelmed by the number of research protocols they are required to review to ensure safeguards for human subjects.¹⁷ The 1998 inspector general’s report states that "increased workload coupled with resource constraints, causes problems for IRBs and threatens the adequacy of their reviews."¹⁸ In today’s environment, the funding, technology, and development of new clinical tools are continually creating the need for more human subjects for research studies. These trends are slowly diminishing the differences between medical care and research. Institutional Review Boards are not charged to assess the implications of these continuing trends within research; rather, they are charged to assess and examine individual research projects.¹⁹

The FDA conducts routine inspections of IRBs, and the OHRP has begun a program of not-for-cause compliance oversight evaluations.²⁰

Additionally, in response to the highly publicized breaches in patient safety described above, the Secretary of DHHS is making changes in the research review system. The Secretary requested that the Institute of Medicine (IOM) assure that all investigators and key staff of federally funded human subjects research obtain certification of participation in a class on human subjects research.²¹ In response to another request from the DHHS Secretary, the IOM is proposing a new system, designed to be broader than IRBs and named Human Research Participant Protection Programs (HRPPPs).²²

Human Research Participant Protection Programs are considered a "total system" of human participants protections. An HRPPP includes the research organization, study investigators, the IRB, and a system of HRPPP performance assessment. This total system comprises four functional principles (which are also regulatory requirements of the Common Rule): "(1) to ensure that research design is sound and that a study’s promise for augmenting knowledge justifies the involvement of human participants; (2) to assess the risks and benefits of a study independently of the investigators who carry out the research; (3) to ensure that participation in research is voluntary and informed; and (4) to ensure that participants are recruited equitably and that risks and benefits are fairly distributed."⁷

The IOM also proposes that all HRPPPs, including IRBs, be subject to an accreditation process.

The National Committee for Quality Assurance (NCQA) developed a set of accreditation standards for IRBs under contract to the US Department of Veterans Affairs (VA). The Public Responsibility in Medicine and Research (PRIM&R) also developed a set of standards. The IOM concluded that NCQA’s accreditation standards were more suitable than PRIM&R’s and therefore suggests the NCQA’s standards be pilot studied beyond VA facilities and academic medical centers.

Health Insurance Portability and Accountability Act (HIPAA)

The Health Insurance Portability and Accountability Act is a federal law enacted in 1996 to, in part, establish standards and requirements for the maintenance and transmission of health information that identifies individual patients. The Privacy Rule, published in final form in August 2002, has implications for QI and human research initiatives.

Due to the scope of HIPAA and very specific applicability pursuant to definitions in the Privacy Rule, a review of the impact of HIPAA on QI and human research is beyond the scope of this report. However, regardless of how QI and human research are ultimately characterized or distinguished by the scientific community, human research activities (however defined) that involve the use or disclosure of protected health information are subject to HIPAA and the Privacy Rule.

Peer-reviewed journals

Investigators and their institutions are primarily responsible for protecting human subjects during research. Professional medical journals, however, also have a responsibility to ensure that published research meets current ethical standards.²¹ Two examples demonstrate the current varying opinions among journals.

The editor of the Archives of Pediatrics and Adolescent Medicine took a strong stand on the topic in a July 2002 editorial. The journal received a number of manuscripts without documentation of an IRB review. In response, the editor states that "our practice at the Archives will be that all studies involving data derived from humans, whether they be medical chart reviews or randomized controlled trials, require either review by an IRB or a decision by the investigator’s department, college, or IRB that the research is exempt." ²¹

The Journal of the American Medical Association (JAMA) published an article including state-level values for the Medicare quality of care measurement program in October 2000. The data originally were compiled from administrative data and medical records. The article included no reference to consideration of whether IRB review of the program might be appropriate or required.²³ The JAMA editor felt the aggregate data (aggregated by state) posed no risk to patients and did not require an IRB review.²⁴

Yank and Rennie²⁵ examined articles published in five professional medical journals (including JAMA) between 1995 and 1999 to assess trends in the reporting of ethical protections. The study found that all of the journals examined have improved their reporting on both informed consent and ethics committee or IRB approval, but that 9 percent of clinical trials published after 1997 still report neither of these protections.

Public sector example: Quality improvement projects of the QIOs and the ESRD networks

The CMS covers 75 million beneficiaries through its Medicare and Medicaid programs, making it the largest public payer of health care in the United States.²⁶ The CMS network of 53 Quality Improvement Organizations (QIOs, formerly Peer Review Organizations) is designed to monitor and improve utilization and the quality of health care for Medicare beneficiaries.

In 1992, QIOs began collecting and analyzing condition-specific data on clinical care, identifying improvement opportunities, intervening to foster improvement, and remeasuring to evaluate the success of the intervention.²⁷ Data are collected from several sources, including Medicare claims data and medical records from physician offices, hospitals, nursing homes, and home healthcare agencies. To facilitate these projects, QIOs have access to patient-specific data in the Medicare claims files and the Medicare enrollment database for their respective states. Data are analyzed and aggregated for reporting back to hospitals and other provider entities. Confidentiality of information is safeguarded by reporting back aggregated provider-specific and, when necessary, patient-specific data only to the provider of the services. Disclosure of provider-specific data to any other entity can occur only with written permission from the provider or with a thirty-day notice allowing the provider to comment. Patient-specific data are never disclosed to entities other than to the provider of the service to that patient.

A provider claiming Medicare payment must permit a QIO to examine its operations and records that are pertinent to health care services furnished to Medicare beneficiaries. In order to facilitate the objectives of individual quality improvement projects, QIOs may process non-Medicare patient data when these data are voluntarily supplied by a provider. The same privacy rules as described above are applied to the non-Medicare patient data.²⁸

According to the CMS, QIOs, although federally funded, are not required to seek IRB approval for certain research and demonstration activities. The QIOs function under the presumed continued exemptions provided by the original peer review statutes.²⁹ More specifically, QIOs are funded as "demonstration projects" examining public benefit or service programs and therefore, according to the CMS, fall under the "Exemption of Certain Research and Demonstration Projects from Regulations for Protection of Human Research Subjects" (Exemption 5 described earlier in this report). Therefore, CMS typically does not scrutinize the QI initiatives of the QIOs for characteristics of research.³⁰

The End Stage Renal Disease (ESRD) Networks are under contract with the CMS to implement QI projects for dialysis facilities; the Networks are funded and function similarly to the QIOs. The Networks are charged under the Social Security Act (SSA) to conduct on-site review of dialysis facilities using standards of care to ensure proper medical care, to collect and analyze data, and to prepare reports for the ESRD program. The recent conflicting interpretations of an ESRD Network QI project, which are described in correspondence shared with the AMA by the Renal Physicians Association (and which led to the introduction of Resolution 807, A-02), demonstrate the differing opinions of QI projects and human subjects research.³¹

In a review of studies conducted and published by university staff, the University of Pittsburgh IRB questioned the absence of IRB review and approval of an ESRD Network QI project. In an October 2001 inquiry to the CMS, the IRB noted that the ESRD project appeared to have been designed as a systematic investigation, presumably to "develop or contribute to generalizable knowledge," and thus could be classified as research. In response to the inquiry, a CMS official expressed the opinion that, given the charge to the ESRD Networks, the patient confidentiality protections provided under the SSA, and the QI, non-research orientation of ESRD projects, those projects are not research and are not subject to human subjects protection review.

A subsequent inquiry from the University of Pittsburgh IRB to the OHRP, however, generated a contradictory response. An April 2002 letter from the OHRP determined that activities included in the ESRD project in question did appear to match the OHRP definition of human subjects research. The OHRP letter further noted that the earlier, contradictory determination from the CMS "was apparently not made in collaboration with OHRP." ³²

Given the circumstances described above, the Renal Physicians Association expressed concerns in Resolution 807 (A-02) that the volunteer physician members of the ESRD Networks’ medical review boards had been placed in professional jeopardy for conducting human subjects research (as determined by the OHRP) without prior IRB review and approval.

In light of this situation and other circumstances, CMS officials are keenly aware of the need to evaluate their policies regarding IRB review. The QIO and ESRD Network activities described above are funded by the CMS as "demonstration projects" examining public benefit or service programs; the CMS believes that such demonstration projects fall under Exemption 5 for IRB review (described above). Even if the projects do fall under this exemption category, differences of opinion exist on who should make that determination. The authority to determine exemptions for DHHS activities has been delegated to the Director of the OHRP,²⁰ but the CMS believes it has the authority to determine exemptions for its own projects.

Private sector examples of QI initiatives/research

Three national organizations in the private sector—the NCQA, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO), and the AMA—are conducting or promoting QI activities, and each are evaluating whether their activities require IRB review. Also, physicians, health plans, and hospitals routinely conduct their own, independent QI activities, each of which is subject to a similar evaluation.

Health plans seeking NCQA accreditation must provide to the NCQA de-identified, aggregate patient data on HEDIS^® (Health Plan Employer Data and Information Set) performance measures (eg, the number of patients within a defined eligible population who received an ambulatory prescription for beta blockers after hospitalization for acute myocardial infarction). Health plans construct the measures from claims data provided by physicians and by abstracting data from patient medical records. The NCQA considers this activity as part of health plan operations, as defined by the HIPAA Privacy Rule, and therefore not subject to IRB review. The NCQA’s accreditation standards (eg, Standard RR 6, Privacy and Confidentiality)³³ require that health plans clearly inform members at enrollment (and thereby obtain consent) that data can be used for internal operations.

The field testing of new HEDIS measures is regarded by the NCQA as one activity that does require an IRB review. In one recent instance, however, the IRB determined that formal evaluation of the field test was unwarranted because the test employed a retrospective review methodology that involved no patient identifiable data.³⁴

As part of its ORYX program, the JCAHO requires hospitals to submit data to a vendor, which in turn aggregates and sends the data to the JCAHO. As with NCQA access to HEDIS data, the JCAHO receives only de-identified, aggregate patient data for the ORYX measures. ORYX vendors, however, may have access to identifiable patient information. (One academic medical center confirmed that it submits patient identifiable information to its vendors, which then aggregate the data for submission to JCAHO.³⁵) Should the hospital, however, seek IRB review for this activity? Does this activity fall within the health care operations regulated by HIPAA? The JCAHO hospital accreditation standards (eg, Standards RI.1.2.1 and RI.3)³⁶ mandate that each hospital’s informed consent processes must include disclosure of the potential risks, benefits, and procedures to be followed to all patients asked to participate in research projects; patients’ rights must also be protected while the research is conducted.

As described above, The Consortium advocates the use of flowsheets for the prospective collection of patient care data. If the flowsheets are used by physicians solely for review of treatment and to facilitate point-of-care improvements, their use logically falls in the realm of QI activities. It is also arguable that the flowsheets are educational tools and their use is therefore not subject to IRB review on the basis of the educational exemptions defined by the Common Rule. (The Common Rule does, however, limit the educational exemptions to "research conducted in established or commonly accepted educational settings, involving normal educational practices" or, under certain circumstances, "research involving the use of educational tests." Also, see the randomized controlled trial [RCT] exemption example cited in the Discussion section of this report.) Physicians may even choose to target their QI efforts by using the flowsheets only for a selected group of patients. If, however, a physician uses the flowsheets randomly within the practice, some patients who could benefit from the QI initiative may be deprived of the benefit and the activity could be interpreted by some as research. Moreover, if the Consortium implements a well-designed study to evaluate its QI initiatives, the evaluation study is likely to be considered research.

Independent of the AMA, NCQA, and JCAHO initiatives described above, individual physicians, hospitals and other health care organizations often collect patient care data as part of their own QI efforts. The issues described previously are also relevant to these independent efforts: (1) Are all patients included in the QI effort or is there a "control" group that potentially will not receive "standard" care? (2) Are there other research aspects of the effort that may require independent review? (3) Is there intent to publish the results of the QI effort? (4) Will patient privacy be placed at risk?

The table (PDF, 330KB) attached to this report summarizes QI activities in the private sector, including scenarios for the collection and use of patient care data for HEDIS, ORYX, and Consortium performance measures, and categorizes each activity as either "Research" or "QI Initiative" based on current regulations and interpretations.

Discussion

Several strategies have been suggested to determine whether QI initiatives should be subject to federal human subjects regulations. Each proposal should be viewed in terms of how it determines whether a project places risks on a patient’s medical health or privacy.

All QI is research

As noted above, one journal has instituted a policy whereby all studies involving data derived from humans, whether they be medical chart reviews or RCTs, will be considered research, requiring either review by an IRB or a decision by the investigator’s department or college. This "black and white" approach, while erring on the side of protecting patients from research risks, could result in a significant decrease in the publication of and, indirectly, the initiation of QI studies. Furthermore, IRBs seem to already be overburdened, and, without a clear framework for evaluating QI studies, different IRBs likely would render different decisions. Moreover, many QI activities pose no greater risk than those incurred during routine patient care.

Lynn ³⁷ notes that the documentation requirements and slow feedback rhythm of IRBs are inconsistent with the informality and rapid feedback that characterize QI. She concurs with the above speculation that overwhelming the research review system with QI studies has already delayed or stopped many of these studies and threatens to impede QI efforts nationally.

Another approach is to consider a QI project as research, and requiring IRB review, if the investigator intends to produce generalizable knowledge and publish results. (This approach contrasts with the Common Rule, which defines research on the basis of the design of the study.) As discussed above and noted by Casarett et al,⁹ an investigator may not initially consider the results of a QI project to be generalizable beyond his or her practice site, but may later consider publication worthwhile. Intent is difficult to define and may change over time. Perhaps more importantly, this proposal does not directly address risks to patients. The case described above, where JAMA published a paper by the CMS reporting on clinical measures at the state level, is a prime example. The CMS intended to publish these aggregated results. However, it seems quite reasonable that the JAMA editor considered the collection and publication of these state-level data to pose no undue risks to individual patients.

Criteria for classifying a QI initiative as research

Another strategy calls for defining specific criteria for classifying a QI activity as research. Brett and Grodin³⁸ propose distinctions between conventional research and health services research, which also could apply to the CSA’s discussion of research and QI initiatives. Brett and Grodin suggest a project be considered as research if: (1) the goal of the investigator stretches beyond patient-specific and institution-specific problems; (2) participants in the activity are recruited in a manner that deviates from traditional patient care; or (3) the evaluated entity departs from accepted standards of medical care. The OHRP notes, however, that failure to meet these criteria does not necessarily mean the activity is not research.²⁰

Brett and Grodin’s third criterion begins to address the primary issue of patient risk. However, defining "accepted standards of medical care" may be problematic. Consider again a study where physicians are randomly assigned to use The Consortium’s flowsheet for documenting care for chronic stable coronary artery disease. The data entered onto the flowsheet should be routinely entered into the medical record; the flowsheet serves only as a reminder. The control group therefore would include physicians providing routine care without a flowsheet. One could argue that the patients seen by physicians in the control group will receive standard care but will not have an opportunity to receive "best practice," which the flowsheet may prompt. (This scenario assumes that the use of the flowsheet is beneficial and not detrimental to care.) Is routine care or the potential for best practice considered "accepted standards of medical care?"

Casarett and colleagues⁹ propose that QI projects be reviewed and regulated as research if one of two sequential criteria are met:

Two reviewers of this proposal questioned the first criterion, giving the example of retrospective chart reviews that may not benefit those patients already cared for, but may benefit future patients. They argue the emphasis should not be on "benefit" but rather "harm."³⁹ The second criterion is closer to the notion of whether more than minimal risk to the patient is involved. Cretin and colleagues,⁴⁰ speaking against these criteria as discouraging QI initiatives, reverse the question:

Classifying study designs as QI or research

Another approach calls for developing a list of study designs and classifying each as constituting either QI or research, the latter requiring IRB review. For example, a hospital may implement a Plan-Do-Study-Act (PDSA) cycle to reduce patient falls and track progress using a run chart for initial and subsequent measurements. In this scenario, no single patient is placed at undue risk. Furthermore, this study design may not have the statistical rigor to warrant publication. Most would agree that this study, as an isolated initiative, does not require IRB review.

Conversely, it may be tempting to classify all RCTs as research requiring review. However, exceptions may also exist here. Soumerai and colleagues⁴¹ conducted an RCT to evaluate the effect of local medical opinion leaders on the use of lifesaving drugs for patients with an acute myocardial infarction. The researchers randomly assigned hospitals to receive either mailed feedback on the use of lifesaving drugs or mailed feedback plus small group discussions led by medical opinion leaders. Although this study design was an RCT, the study was considered to be research exempt from human subjects committee review "...because it used chart reviews to evaluate educational activities aimed at increasing adherence to accepted standards of care" and thus qualified for Exemption 1 (described above).⁴² The OHRP, on the other hand, regards this particular study as non-exempt research.²⁰

Institutional review board or new entity review for waiver of consent

Another approach is for IRBs to consider all QI studies as research, but to grant waivers of informed consent. As described previously, informed consent may be the rate limiting factor in a QI study designed to access data from thousands of medical records. The algorithm currently used by an IRB to waive informed consent (see Appendix A [PDF, 330KB]) may be a starting place for a new type of expedited review.

Even if QI initiatives receive expedited review and a waiver of consent, having these decisions rendered by an IRB may be problematic for several reasons: (1) the many private organizations not funded by federal agencies are not required to use the IRB review process; (2) the IRB system today is already overburdened; (3) even with expedited review, the slow feedback cycle of IRBs is an impediment to the progress of QI; and (4) the IRB process was not designed to focus on privacy issues, and therefore IRB reviewers are not necessarily trained in this area. In fact, IRB members report that they typically rely on other organizations’ policies to safeguard patient privacy and confidentiality.

Training investigators to consistently evaluate the research aspects of their own initiatives and to request institutional review appropriately is one alternative to IRB review of all QI initiatives. Another alternative is review by an oversight committee. The Oversight Body for the Ethical Force Program, convened by the Institute for Ethics at the AMA, has developed a detailed proposal for appropriate use of personally identifiable health information, including the creation within each organization of an oversight committee called a data disclosure board (DDB).⁴³ The purpose of the DDB is to:

Among the activities identified for DDB oversight are QI projects, marketing, disease management programs, and accreditation activities. The E-Force Program also outlines a process for determining appropriate use of identifiable information (see Appendix B [PDF, 330KB]).

Casarett et al ⁹ offer a recommendation similar to that of the E-Force Program, suggesting that each institution that engages in frequent QI initiatives establish its own committee for reviewing QI projects.

Standardization

With both the Common Rule and HIPAA to consider, as well as state statutes that may take precedence over the federal regulations, it would clearly be advantageous to achieve national standardization in differentiating between QI and research. Otherwise, just as is currently apparent in the determinations rendered by IRBs, inconsistencies will abound among individual determinations in the public and private sectors and across states.

One notable effort toward standardization is a project recently proposed by The Hastings Center and partially funded by the Agency for Healthcare Research and Quality (AHRQ). The project will convene a multidisciplinary task force of nationally recognized leaders in QI, healthcare management, health services research, organizational ethics, law, and public policy. In a series of conferences over four years, the task force will study the underlying goals and methods of QI, delineate the ethical values served by QI and the ethical dilemmas generated by QI projects, and seek to develop a set of ethics standards by which QI studies can be assessed. If the task force determines that standards can be developed, the project will produce and disseminate a consensus document summarizing the recommendations of the task force. The project will also develop other publications designed to promote serviceable interpretation of, and improved consensus on, the relevant policies.

At this writing, the AMA is investigating opportunities to contribute technical expertise to the Hastings Center project and to secure additional outside funding. The AMA will also seek to ensure a means for the E-Force Program to provide input to The Hastings Center’s development of ethical standards.

Summary and key questions

Systematic evaluation of QI studies threatens to overwhelm the research review system and hinder the implementation of important QI initiatives nationally. The "generalizability" of the knowledge produced is an inadequate criterion for classifying QI projects as research; other criteria have been proposed but lack broad support and are inconsistently applied. Review or oversight bodies other than IRBs may be needed to determine when QI initiatives require additional review or patient consent.

Additionally, several fundamental, unanswered questions arise from this discussion:

• What is the appropriate framework or strategy for evaluating the risk to patients from QI initiatives?

• Who should develop that framework and who should implement it?

• Can the framework be standardized to apply to both the public and private sectors and across state lines?

• What are the appropriate channels to influence state and federal legislation?

Recommendations

The following statements, recommended by the Council on Scientific Affairs, were adopted by the AMA House of Delegates as AMA directives at the 2002 AMA Interim Meeting:

1. The AMA will seek to ensure an active role in the series of meetings and the deliberate process proposed by The Hastings Center to develop a framework for addressing the quality improvement/research issues. (Directive)
2. The AMA will continue to research this topic, particularly by reviewing forthcoming publications, corresponding with experts in the field, and completing a more detailed review of the framework developed by the Oversight Body for the Ethical Force Program, to prepare a supplemental report as relevant information becomes available. (Directive)

Also see the AMA's Advancing quality improvement in patient care Web site

CSAPH home page
Reports by topic

---