Assessing the quality of human research protection programs: the experience at the Department of Veterans Affairs.
Article Type: Report
Subject: Research ethics (Laws, regulations and rules)
Research ethics (Political aspects)
Human experimentation in medicine (Laws, regulations and rules)
Human experimentation in medicine (Political aspects)
Self-experimentation in medicine (Laws, regulations and rules)
Self-experimentation in medicine (Political aspects)
Institutional review boards (Research ethics) (Training)
Authors: Tsan, Min-Fu
Smith, Karen
Gao, Baochong
Pub Date: 07/01/2010
Publication: Name: IRB: Ethics & Human Research Publisher: Hastings Center Audience: Academic Format: Magazine/Journal Subject: Health Copyright: COPYRIGHT 2010 Hastings Center ISSN: 0193-7758
Issue: Date: July-August, 2010 Source Volume: 32 Source Issue: 4
Topic: Event Code: 930 Government regulation; 940 Government regulation (cont); 980 Legal issues & crime; 280 Personnel administration; 350 Product standards, safety, & recalls Advertising Code: 94 Legal/Government Regulation Computer Subject: Government regulation
Organization: Government Agency: United States. Department of Veterans Affairs
Geographic: Geographic Scope: United States Geographic Code: 1USA United States
Accession Number: 239462723
Full Text: In the late 1990s and early 2000s, federally supported research programs at a number of major academic institutions were temporarily suspended in response to persistent noncompliance with federal regulations governing research with humans. These suspensions included responses to incidents involving the death of two research participants. (1) In response to increased public scrutiny of human subjects research resulting from the suspensions, considerable efforts have been made to improve the oversight system pertaining to research with humans, with the ultimate goal of protecting people who participate in research studies from research-related harms. These efforts include--but are not limited to--stronger federal oversight of research, voluntary accreditation of institutional human research protection programs, increased institutional support for such programs, improved training for investigators and institutional review board (IRB) members, improved monitoring and reporting of adverse events, and greater involvement of research participants and the public in reform efforts. (2)

Although the investment to improve the system for protecting research participants has been considerable, there are scant data showing that reforms have made human research any safer. Moreover, there has been growing concern that an overemphasis on compliance with federal regulations governing research with humans may draw attention away from the ethical quality of IRB review of research and divert resources from ongoing protection of research participants. (3) For instance, Taylor points out that it is necessary to develop a valid, reliable, and robust measure of whether IRBs are achieving "the ethical goals of research oversight--protecting the welfare of subjects, promoting respect, and attending to justice in human research." (4) Such a tool would be helpful in assessing the quality, not just the compliance, of the IRB review process. Nonetheless, high-quality IRB reviews alone, while necessary, may not be sufficient to ensure research subject protections because IRB review of research is only one component of a human research protection program. (5)

Although it is difficult to directly measure the extent to which human subjects are protected from research-related harms, we believe it is possible to identify indicators that can be used to assess the quality of an institution's human research protection program, which includes attention to factors that are likely to or may lead to research-related harms. We report here on the quality indicators we developed to assess the human research protection programs in the Department of Veterans Affairs.

Quality Indicators for Human Research Protection Programs

In addition to the federal regulations governing research with humans, researchers in the VA system must comply with requirements established by the VA. For example, in the VA Health Care System, the IRB is a subcommittee of the Research and Development Committee (R&DC). Research involving human subjects cannot be initiated until it has been approved by both the IRB (unless it is exempt from IRB review) and the R&DC. (6) The VA has a research compliance office, and all VA facilities that conduct research are required to have an accredited human research protection program.

As the goal of an institution's human research protection program is to enhance protections for those who participate in research, we developed quality indicators that emphasize assessing the outcome of the human research protection program, rather than solely IRB review or compliance with research regulations. The quality indicators were developed through a process that included 1) one of the authors (MFT) identifying potential indicators; 2) individuals within and outside the VA with expertise in human subjects protections reviewing the proposed indicators; and 3) a five-member work group appointed by the Office of Research Oversight (ORO) reviewing and revising the proposed indicators. After six months of deliberation, the work group reached a consensus on which quality indicators to accept, and the ORO Executive Committee subsequently accepted the indicators to use in assessing the VA's human research protection programs.

Figure 1 lists the 16 quality indicators the ORO Executive Committee approved. The indicators cover a broad range of areas that may have significant impact on human subject protections, including accreditation of human research protection programs; initial and continuing review and approval of research by the IRB and the R&DC; participant informed consent; authorization pursuant to the Privacy Rule of the Health Insurance Portability and Accountability Act (HIPAA Privacy Rule) to use and disclose participants' protected health information; investigator compliance, qualification, and training; serious adverse events; research involving vulnerable subjects; international research; training of IRB and R&DC members; and educational materials for research participants. This instrument will allow us to compare the quality of human research protection programs across research facilities in the VA and will capture information that can be used to guide administrators in making management decisions where program improvements are most needed.

We designed the quality indicators to be used to assess the human research protection programs at VA research facilities annually, or at least once every other year. With the recent requirement that a fulltime Research Compliance Officer (RCO) at each VA research facility conduct annual audits of all informed consent documents and regulatory audits of all research protocols once every three years, (7) it is feasible that an assessment of the VA's human research protection programs using the quality indicators could be conducted annually.

We are cognizant of the fact that there is no evidence to indicate that the quality indicators are a valid and reliable measure of the quality of an institution's human research protection program. We also do not know whether the quality indicators correlate directly to the protection of research subjects. In addition, we are unable to put a quantitative, numerical value on each indicator. Nonetheless, we believe the quality indicators provide a useful first step toward developing a robust and valid assessment of human research protection programs. As we use these indicators at VA institutions, we expect them to be redefined and modified. We hope other institutions will find these indicators useful as they develop instruments to assess their own human research protection programs.

Acknowledgments

We thank J. Thomas Puglisi, PhD, Chief Officer, Office of Research Oversight, Department of Veterans Affairs, for his support and critical review of the manuscript.

References

(1.) Statement of Kenneth W. Kizer, Under Secretary for Health, Department of Veterans Affairs, on Oversight of Research in the Veterans Health Administration before the Subcommittee on Health of the Committee on Veterans' Affairs, U.S. House of Representatives, April 21, 1999; Steinbrook R. Protecting research subjects--the crisis at Johns Hopkins. NEJM 2002;346:716-720.

(2.) Steinbrook R. Improving protection for research subjects. NEJM 2002;346:1425-1430.

(3.) See ref. 2, Steinbrook 2002.

(4.) Taylor HA. Moving beyond compliance: Measuring ethical quality to enhance the oversight of human subjects research. IRB: Ethics & Human Research. 2007;29(5):9-14; Koski G. Beyond compliance ... Is it too much to ask? IRB: Ethics & Human Research 2003; 25(5):5-6.

(5.) Institute of Medicine. Preserving Public Trust: Accreditation and Human Research Participant Protection Programs. Washington, DC: National Academies Press, 2001.

(6.) Requirements for the Protection of Human Subjects in Research. VHA Handbook 1200.05, Department of Veterans Affairs (July 31, 2008), http://www1.va.gov/vhapublications/; Research and Development Committee. VHA Handbook 1200.01, Department of Veterans Affairs (June 16, 2009), http://www1.va.gov/vhapublications/.

(7.) Research Compliance Officers and the Auditing of VHA Human Subject Research to Determine Compliance with Applicable Laws, Regulations, and Policies. VHA Directive 2008-064, Department of Veterans Affairs (October 16, 2008). Available at http:// www1.va.gov/vhapublications/.

Min-Fu Tsan, Karen Smith, and Baochong Gao, "Assessing the Quality of Human Research Protection Programs: The Experience at the Department of Veterans Affairs," IRB: Ethics & Human Research 32, no. 4: 16-19.

* Min-Fu Tsan, MD, PhD, is Director, Mid-Atlantic Regional Office, Office of Research Oversight, Department of Veterans Affairs, Washington, DC; Karen Smith, PhD, is Director, Midwestern Regional Office, Office of Research Oversight, Department of Veterans Affairs, Chicago, IL; and Baochong Gao, PhD, is Deputy Director, Mid-Atlantic Regional Office, Office of Research Oversight, Department of Veterans Affairs, Washington, DC.
Figure 1. Quality Indicators for Assessing Human Research
Protection Programs in the Department of Veterans Affairs

1. Accreditation status of human research protection program

a) Full accreditation

b) Qualified accreditation

c) Accreditation withheld

d) Accreditation pending

e) Other (please specify)

2. Institutional review board (IRB) and Research and Development
Committee (R&DC) initial approval of human research protections

a) Number of protocols initiated without: IRB approval; R&DC
approval; IRB and R&DC approval

b) Number of protocols initiated prior to: IRB approval; R&DC
approval; IRB and R&DC approval

3. Informed consent requirement

a) Total number of subjects enrolled for whom informed consent is
required

b) Number of subjects enrolled without obtaining informed consent

c) Number of subjects enrolled prior to obtaining required informed
consent

d) Number of subjects with informed consent documents without
subject or legally authorized representative's signature

4. Authorization requirement of the Health Insurance Portability
and Accountability Act Privacy Rule (HIPAA Privacy Rule)

a) Total number of subjects enrolled for whom HIPAA authorization
is required

b) Number of subjects enrolled without obtaining required HIPAA
authorization

c) Number of subjects enrolled prior to obtaining required HIPAA
authorization

5. For-cause suspension of human research

a) Number of protocols suspended (by IRB or R&DC) due to serious
noncompliance

b) Number of protocols suspended (by IRB or R&DC) due to serious
adverse events (both anticipated and unanticipated)

c) Number of investigators suspended (by IRB or R&DC) due to
noncompliance

d) During this period, has the facility's human research protection
program ever been suspended (no/yes)? If yes, please explain the
reason(s) for the suspension.

6. Research-related serious adverse events (AEs)

a) Number of AEs determined by IRB to be serious, unexpected, and
related to research

b) Number of health volunteers with research-related serious AEs
resulting in death or hospitalization

7. Continuing review requirement

a) Total number of protocols requiring continuing review

b) Number of protocols lapsed in required continuing reviews by IRB

c) Number of protocols with lapsed IRB continuing review approval
for which research activities occurred (except continuation of
research intervention or interaction deemed to be in the best
interest of already-enrolled subjects by IRB)

8. Internal protocol audits

a) Number of protocols audited by Research Compliance Officer (or
equivalent): total number of audits; for-cause audits; routine
audits

9. Subject enrollment according to inclusion and exclusion criteria

a) Number of subjects enrolled who did not meet inclusion criteria

b) Number of subjects enrolled who met exclusion criteria

10. Scope of practices and privileges

a) Number of investigators and research staff working outside of
their scope of practices and privileges

11. Research involving vulnerable populations

a) Number of protocols involving: fetus; in vitro fertilization;
pregnant women; prisoners; children; mentally disabled persons;
subjects with impaired decision-making capacity

12. International research protocols

a) Total number of international research protocols

b) Number of international research protocols without obtaining
required prior approval of Chief Research and Development Officer
(CRADO)

13. Investigators sanctioned by Food and Drug Administration (FDA)
due to serious noncompliance

a) Number of investigators sanctioned by FDA for: restriction;
disqualification; debarment

14. Investigator human research protection program (including
information security) education/training requirements

a) Total number of investigators including research coordinators,
etc., requiring human research protection program
education/training

b) Number of investigators including research coordinators, etc.,
who have not completed required initial human research protection
program education/training

c) Number of investigators lapsed in required annual human research
protection program education/training

15. IRB and R&DC chairs and members of human research protection
program (including information security) education/training
requirements

a) Total number of IRB chairs and members

b) Number of IRB chairs and members who have not completed required
initial human research protection program education/training

c) Number of IRB chairs and members lapsed in required annual human
research protection program education/training

d) Total number of R&DC chairs and members

e) Number of R&DC chairs and members who have not completed
required initial human research protection program
education/training

f) Number of R&DC chair and members lapsed in required annual human
research protection program education/training

16. Subject education

a) Total number of subjects enrolled

b) Number of subjects who were given a copy of the information
brochure "Volunteering in Research--Here Are Some Things You Need
to Know"
Gale Copyright: Copyright 2010 Gale, Cengage Learning. All rights reserved.