PROMIS: a new tool for the clinician scientist.
Article Type: Editorial
Subject: Scientists (Surveys)
Treatment outcome (Surveys)
Author: Lawson, Douglas M.
Pub Date: 01/01/2011
Publication: Name: Journal of the Canadian Chiropractic Association Publisher: Canadian Chiropractic Association Audience: Academic Format: Magazine/Journal Subject: Health Copyright: COPYRIGHT 2011 Canadian Chiropractic Association ISSN: 0008-3194
Issue: Date: Jan, 2011 Source Volume: 55 Source Issue: 1
Product: Product Code: 8520110 Scientists NAICS Code: 54171 Research and Development in the Physical, Engineering, and Life Sciences
Geographic: Geographic Scope: Canada Geographic Code: 1CANA Canada
Accession Number: 259589908
Full Text: The Patient-Reported Objective Measurement Information System (PROMIS) holds great potential for supporting the clinical scientist. That potential includes: better surveys, better outcome measures, easier obtained outcome measures, better comparators, easier collaboration, and greater support for the clinician who wishes to engage in research in his/her clinical setting.

In the past, conducting research from within a clinical practice has been fraught with many challenges. Unless the clinician was located close to an educational institution, little support has been available. If patients were to be surveyed, which survey should be used? Which of the available surveys (patient reported measures) has been validated for the population of interest to the clinician? If the clinician was to make up his/her own survey, which rating scales should be used and how was the survey to be validated? Was the survey reliable? Were the survey outcomes valuable? If the survey was delivered by paper, how were the data to be entered electronically, and who would help analyze the data? How did the clinician's sample compare to a normal population, or to a sample of similar health challenges? If a single clinician's office held insufficient sample size for clinical research, how could collaboration with other offices be coordinated? Once the research was completed, who else could benefit from access to the data? How could the data be shared? There were many barriers to the clinician who wished to contribute to the scientific evidence within his/her field.

Starting in 2004, the National Institutes of Health (NIH) created a process to deal with the critical research challenges of clinical research. (1) The Patient-Reported Outcomes Measurement Information System (PROMIS; www.nihpromis.org) was one of the first initiatives. This initiative brought the focus of many experts in research and psychometrics to the problems of clinical research and patient reported outcomes. Patient reported outcomes include quality of life surveys, pain questionnaires, functional surveys, and satisfaction with care surveys. Advanced psychometric techniques were used to validate available survey instruments and to create better instruments for the clinician. The PROMIS research sites gathered data on both the general population and diseased populations in 2006 and 2007: normal subjects approximately 7,500, cancer approximately 1,000, heart disease approximately 500, rheumatoid arthritis approximately 500, osteoarthritis approximately 500, etc. (2) The general population was constructed to ensure adequate representation with respect to: sex, age, ethnicity and education (for a US population). Currently, the Assessment Center website (www.assessmentcenter.net) states that there has been adult testing in over 20,000 individuals in the US and child and youth testing in over 4,000 individuals in the US.

Both traditional (classical test theory) and modern (item response theory) analytic methods were applied to the data. The analysis allowed the validation of surveys, development and selection of appropriate rating scales and of individual items. For example in the area I am currently working (sleep disorders), the literature search revealed over 100 surveys with almost 3,000 items. (3) As well as the analysis, all surveys were subjected to qualitative focus groups research. As a result of the considerable investment of time, financial resources, and expert analysis, the PROMIS web site now lists 17 surveys. Table 1 lists the surveys available as of July 2010 (from the Assessment Center Instrument Library).

Table 1 lists the psychometric domain that each survey measures, the number of items in the bank, and the number of items on the short form (which can be printed and delivered by paper), for both adult and pediatric populations. The Assessment Center Instrument Library is the research arm of PROMIS. At the Assessment Center, researchers can register and engage in the research process by creating surveys on the Assessment Center web site. Any of the surveys/instruments listed above can be added to a researcher's study. Demographic information as well as unique questions can be added to the survey. Welcome pages, individual logos, and on-line consent forms can be added to a study. The Assessment Center allows a clinician scientist to create a study, administer the study, and download the data from the study for analysis. The researcher can compare the results of his/her study to the general population data from PROMIS, or to disease specific populations.

As PROMIS has applied modern psychometric analysis to the data, survey participants do not have to respond to the entire items in any bank in order for stable estimates. As few as 4-items might be needed to estimate the participant's measure. This means that multiple measures can be gathered without fatiguing the participants. PROMIS has a demonstration of the computer adaptive testing (CAT) of surveys on its web page, and it is worthwhile to visit that site and visit the Assessment Center and the CAT demonstration (www.nihpromis.org).

For those wishing to utilize the short-forms of the surveys and to distribute them as paper surveys within the clinical setting, paper versions have been created. The short-forms are available to registered researchers who agree to a quite reasonable copyright. The short-forms include a scoring format so that each clinician can score a participant's form and arrive at a stable measure for that participant that compares him/her to a normal population. In order to be able to score the short forms in the clinical setting, participants must respond to all questions in the form. The short forms are a great opportunity for those who might wish to compare their daily patients to a larger population. New patients could, for example, fill out an 8- to 10-item form on pain, disability, or fatigue and the researcher would have some evidence as to how they compare to a normal population.

For both the short-form and CAT forms of the surveys, participant's measures are reported as a transformed scale (or T-score). The mean of the transformed scale is set at 50 and the standard deviation is set at 10. Due to the large size of the population data gathered by PROMIS, a normal table can be utilized to determine how the participant's measure fits within a normal curve. Any introductory statistical text will provide a table for conversion as will Dr. Rollin Brand's web page (www.stat.ubc.ca/~rollin/stats/ref/tables.html). For example, a participant measure of 60 would indicate that he/she was one standard deviation above the mean and that about 84% of the general population would have a lower measure. A participant measure of 70 would indicate two standard deviations above the mean, and that about 98% of the general population would have a lower measure. (4)

For the purpose of collaboration, the Assessment Center of PROMIS allows for teams of researchers to be involved. Each team member can be provided specific access to the site (e.g., from full access to data, to only being able to view the surveys being used). Researchers from anywhere in the world can be provided access to the site and methods for each researcher's participants to access the site. In this way, participants from multiple sites can easily be analyzed separately or in combination to increase a study's sample size.

As with every research study, the study leader still is responsible for agreements with PROMIS, appropriate consent forms and proper ethics board approval. The PROMIS site has the means for participants to consent online, and for privacy information to be held confidential.

On the plus side, PROMIS has provided researchers with an amazing study administration tool (did I mention free of charge, and available throughout the world). The surveys have been validated for a general population and specific disease groups. The results are provided in a manner that the clinician's sample can be compared to the general population. Many of the barriers have been addressed and overcome by the previous research, analysis, validation and publication. For those wishing to develop their own surveys, the articles published by the PROMIS researchers provide a road-map of how this should be done. Here's the catch: once a year the researcher must provide demographic information to the PROMIS administrators. If the demographic and data prove to be of interest, they might be added to the PROMIS databases. Again, the researcher had control and this is an excellent opportunity to add information from a small research project to a larger study. It is quite possible that your study doesn't die when completed, but continues on as part of a larger research project.

If there is a negative, it is that the data are centered on a US population. This might be less of a problem for Canada than for other countries. Another issue is that a researcher's sample might be at the extreme end of the normal population dynamic (think breast cancer survivors dragon boat racing teams) and the PROMIS survey might not be applicable for such an extreme group.

PROMIS appears to hold the potential for providing the support needed to remove the barriers to most individuals wishing to practice as clinician scientists. I hope the reader will take the time to review the excellent work that has been done by PROMIS to date, and to consider the application of the Assessment Centers instruments in clinical research.

References

(1) Gershon R, Rothrock NE, Hanrahan RT, Jansky LJ, Harniss M, Riley W. The development of a clinical outcomes survey research application: Assessment Center. Qual Life Res. 2010 Mar 21; 19(5):677-85.

(2) Reeve B, Hays RD, Bjorner JB, Cook KF, Crane PK, Teresi JA, et al. Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Med Care. 2007 May; 45(5 Suppl 1):S22-31.

(3) Buysse DJ, Yu L, Moul DE, Germain A, Stover A, Dodds NE, et al. Development and validation of patient-reported outcome measures for sleep disturbance and sleep-related impairments. Sleep. 2010 Jun; 33(6):781-92.

(4) Gilbert N. Normal Distributions and Standard Scores. In: Statistics. Philadelphia, PA 19105: W.B. Saunders Company; ISBR 0-7216-4127-X.

Dr. Douglas M. Lawson DC, MSc *

* Calgary, Alberta Email: lawsondrdm@gmail.com
Table 1

                                               Adult

                                     # Items in     # Items in
Domain                                  Bank       Short Form(s)

Emotional Distress--Anger                29              8
Emotional Distress--Anxiety              29         4, 6, 7, 8
Emotional Distress--Depression           28           4, 6, 8
Fatique                                  95         4, 6, 7, 8
Pain--Behavior                           39              7
Pain--Interference                       41           4, 6, 8
Sleep Disturbance                        27           4, 6, 8
Sleep Related Impairment                 16              8
Physical Function                       124       4, 6, 8, 10, 20
  --Mobility
  --Upper Extremity
Asthma
Satisfaction with Participation in       12              7
  Discretionary Social Activities
Satisfaction with Participation in       14         4, 6, 7, 8
  Social Roles
Peer Relationships
Global Health                                           10
Profile (4-, 6-, or 8-items short                   29, 43, 57
  forms of Anxiety, Depression,
  Fatique, Pain-Interference,
  Sleep Disturbance, Physical
  Function, and Satisfaction with
  Participation in Social Roles)

                                             Pediatric

                                     # Items in     # Items in
Domain                                  Bank        Short Form

Emotional Distress--Anger                                6
Emotional Distress--Anxiety                              8
Emotional Distress--Depression                           8
Fatique                                                 10
Pain--Behavior
Pain--Interference                                       8
Sleep Disturbance
Sleep Related Impairment
Physical Function
  --Mobility                             23              8
  --Upper Extremity                      29              8
Asthma                                   17              8
Satisfaction with Participation in
  Discretionary Social Activities
Satisfaction with Participation in
  Social Roles
Peer Relationships                       15              8
Global Health
Profile (4-, 6-, or 8-items short
  forms of Anxiety, Depression,
  Fatique, Pain-Interference,
  Sleep Disturbance, Physical
  Function, and Satisfaction with
  Participation in Social Roles)
Gale Copyright: Copyright 2011 Gale, Cengage Learning. All rights reserved.