Monitor - January 1997

A quarterly publication for members of the Medical Outcomes Trust
January 1997

Depression is a serious and often overlooked health problem. The lifetime risk for major depressive disorder is estimated at 7 to 12 percent for men and 20 to 25 percent for women. Estimates also suggest that only one-third of people suffering from depression seek help. Yet current research and major clinical guideline projects indicate that treatment is straightforward and effective. Most major depressive disorder can be resolved within six months with proper treatment. Some people believe that the effective diagnosis and treatment of major depressive disorder is one of the US health care system’s most significant challenges.

Systematic approaches for detecting depression and evaluating treatment effectiveness hold substantial promise for improving care for depression.

The initiatives described in this issue reflect the major coordinated efforts to use outcomes research and management systems for understanding the optimal treatment for depression and assuring that quality treatment is provided. A new family of studies in the US, funded by the Agency for Health Care Policy and Research (AHCPR) and by the National Institute of Mental Health (NIMH), will explore how to improve the cost-effectiveness of depression care and will evaluate the AHCPR clinical practice guidelines for Depression in Primary Care. Through these research efforts, much needed information will become available about the prevalence of depression in primary care, the effectiveness of different treatment approaches, the functional improvement gained by cost of treatment, and other key features of treatment and delivery models that can lead to improvement in the quality of depression care.

Commentary
Contributed by G. Richard Smith, MD, Professor of Psychiatry at the University of Arkansas for Medical Sciences, and Director, Center for Outcomes Research and Effectiveness. Dr. Smith is one of the developers of the Depression Outcomes Module.

Central to the issue of care for depression is a paradox. There is substantial evidence that depression is treatable and that there is a tremendous burden of disability associated with depression. The processes and outcomes of depression care are measurable. Yet depression is often not recognized and not treated even though it is known to be helpful and easy to do so.

There are many hypotheses about why this paradox occurs, ranging from the stigma associated with mental health problems to inadequate training of physicians and other health care providers to detect and treat mental health problems during their formal education. But outcomes management can help counteract this problem. Outcomes management systems can be used to improve the basic level of care that is provided for patients with major depression and other depressive disorders. Outcomes management and accountability systems are not designed to help experts treat rare cases, but are aimed at developing systems of care that will help physicians and other health care providers recognize and treat appropriately the majority of cases.

In general, there are three key components of the treatment of depression that require evaluation in outcomes management systems. The first is determining whether or not the diagnosis has been made appropriately. Major depression should not be used as a catch-all for any mental disorder. The second component is selecting the appropriate treatment. Generally, for outpatient treatment, there are two types of treatment available, plus a combination of both treatments. One is pharmacological treatment and the other is psychotherapy. The AHCPR guidelines present the treatment options in a format that allows for consistency and reduces management complexity.

The third component is monitoring the outcome of treatment. Simply providing the prescription for the medication is not enough. The patient has to be seen in follow-up visits. The clinician must be sure that the patient is taking the prescribed medication and must take an aggressive approach to resolve the depressive episode. Frequently a change in medicine or a change in the form of treatment needs to occur. If the clinician takes a fairly assertive position to see the patient frequently and to look for improvement in his or her health status, then it’s easy to treat those patients.

Often clinicians within both primary and specialty care throw up their hands and say “I don’t know how to treat it.” Through the guideline projects, both AHCPR’s and the American Psychiatric Association’s, a straightforward process to take care of most patients with depressive disorders has been presented. If the clinician cannot care for the patient, the patient should be referred to somebody who can take care of that disorder. Health care providers should not let depression go untreated.

The new drugs (SSRI antidepressants) available for treating depression have fewer side-effects and are easier to use than the older drugs. Clinicians can aggressively prescribe these drugs for people with major depression, including those patients who may be suffering from other chronic diseases, like diabetes or asthma. Depression can be treated simultaneously with most other chronic diseases, as soon as it is recognized. There may be diagnostically confusing issues with depression and another chronic diseases–for instance, loss of appetite could be a symptom of diabetes or depression. But because the problem of undertreated depression is so severe and the side-effects of the medications few, clinicians should not hesitate to treat depressive symptoms.

Outcomes monitoring, or accountability, systems have an important role to play for the providers, for the payors, and for the consumers. These systems can contribute substantially to our understanding of how patients with different characteristics benefit from different treatments. When used to assure that an effective treatment process has occurred, outcomes management will contribute directly to improving the health and well-being of patients who receive care within that system.

Epidemiologic studies estimate that depression affects one in eight people over a lifetime. Since the majority of people affected seek help from their primary care providers, there is tremendous pressure on physicians in primary care to detect, diagnose, and treat depression. Some of the greatest challenges for managed care providers are estimating the prevalence of depression in the plan’s enrolled population, identifying the impact of untreated depression on health utilization within the plan, rapidly and effectively detecting depression, and evaluating effectiveness of treatment.

Arne Beck, PhD, Research and Development Director for the Kaiser-Permanente plan in Colorado, is spearheading a group of projects aimed at addressing these challenges and improving the treatment of depression within the plan’s enrolled population. The constellation of projects and studies include: a utilization impact study of undiagnosed vs. diagnosed depression, a depression prevalence study, pilot test of an automated diagnostic tool, a study of adding psychologists to the primary care team, validation of an automated diagnostic tool, and development of a performance indicator for the National Committee for Quality Assurance (NCQA) on depression treatment.
At various stages of development, the projects and studies described below are expected to yield valuable information and improved outcomes for those suffering from depression and other common mental health problems. The automated diagnostic tool (the QPD panel) is described below.

• Feasibility study of automated diagnostic tool and analysis of health care utilization
On the 1995 national depression screening day, 88 outpatients on a routine visit to one outpatient clinic completed the QPD panel (see Sidebar), the Zung depression scale, and an evaluation survey of the ease of use of the QPD panel. Clinically significant scores were referred to the primary care provider. A mental health provider was on-site to go over the results with those who completed the questionnaire, to provide minimal intervention and arrange for referrals. The results of this screening were evaluated to determine patient acceptance of the QPD, to assess convergent validity of the QPD depression scale through its correlation with the Zung depression scale, and to examine outpatient primary care and mental health services prior to and following administration of the QPD. There was high patient acceptance of the QPD and high correlation of the QPD depression scale with the Zung scale. Patients screened and diagnosed for a psychiatric disorder with the QPD showed high prior outpatient utilization patterns, although two-thirds had no mental health visits during the previous 12-month period. Utilization data following screening indicated that mental health visits increased to at least one visit for 50% of those diagnosed with the QPD.

• A prevalence study, collected from a sample of all patients visiting the plan, using the SF-12 Health Survey as a screen and, if triggered, the Shedler Quick PsychoDiagnostic (QPD) panel.
This project is collecting information from a random sample of patients waiting to see a clinician in primary care clinics. The handheld computer is loaded with the SF-12 Health Survey and the QPD panel. The computer presents the SF-12 Health Survey first and, if responses to four mental health items from the SF-12 Health Surveys indicate a low mental health score, the computer automatically presents the QPD
panel questions. The study includes about 300 patients from two sites. Preliminary results indicate that the rate of diagnosis for major depression within a primary care site is consistent with the range from other studies.

• Pilot test of the automated diagnostic tool in conjunction with physician/psychologist collaboration study
A pilot project is underway to test the use of the QPD panel for routine use in primary care. In the pilot, the clinician identifies patients for the test and the patient completes the questionnaire in the waiting room, usually immediately prior to the visit. The result of the questionnaire, displayed as a lab report, is provided to the clinician for review during the visit. Both patient and physician response has been positive. Physicians participating in the pilot requested additional information that has now been added to the report: if the patient is positive for the diagnosis, the symptoms associated with the condition are listed.

Also included in the pilot is a study of the effectiveness of psychologist/physician collaborations in the early detection and treatment of depression and other psychiatric disorders. If the diagnosis is positive for a psychiatric disorder, the primary care physician will refer the patient to the psychologist. The psychologist does further assessment and offers short term cognitive behavioral therapy. If the patient wants medication, the primary care physician will write the prescription. Two licensed clinical psychologists have been reassigned to a primary care clinic for participation in the study. The study is being conducted at one primary care site. Two hundred patients will be enrolled in the study; 31 have been enrolled to date. The study will evaluate patients’ functional status, clinical improvement, satisfaction with care, and side effects at one, three, and six months.

• Validation study of the automated diagnostic tool
A validation study of the QPD panel is in progress. For this study, a qualified psychologist administers the Structured Clinical Interview for DSM-IV (SCID) to a sample of patients who have completed the QPD panel. Patients enrolled in the study are those presenting for initial treatment at a mental health clinic within the Kaiser Permanente Colorado system. The SCID interviewer is blind to the QPD panel results. Two hundred and fifty validation interviews are planned; 150 have been completed to date. Preliminary results indicate validity is extremely high; false positives have been rare.

• Development and testing of a depression performance indicator for the NCQA chronic care initiative.
As part of the NCQA’s chronic care performance measurement project funded by the Robert Wood Johnson Foundation, the Kaiser Permanente Colorado plan is testing a plan-level indicator for treatment of depression. The indicator will draw on pharmacy data to calculate the adequacy of pharmacological treatment for patients with major depressive disorder. Capturing the ICD-9 code for major depressive disorder at the time of diagnosis (within past six months) or entry into the system, the measure will link pharmacy information, including refills, to determine if the dose of medication prescribed was adequate and the recommended course of medication was completed. This measure has been developed in conjunction with Harvard Pilgrim Health Care for proposed use in a new set of performance measures for chronic care.

The automated diagnostic tool
Shedler Quick PsychoDiagnostic (QPD) Panel

The tool in use at Kaiser-Permanente Colorado was developed by a clinical psychologist to address the problem of underdetection and undertreatment of depression in the primary care setting. Its original version for detecting depression was expanded, during its six years of development, to screen for and diagnose six additional psychiatric conditions common in primary care: dysthymic disorder, generalized anxiety disorder, panic disorder, obsessive compulsive disorder, bulimia nervosa, and alcohol/substance abuse.

At the Kaiser-Permanente Colorado site, the questionnaires are administered through small, handheld computers. Patients answer the questions in the waiting room and return the computers to the receptionist who places them on a docking station where, through infrared technology, the questionnaires are scored and the results printed onsite. The scored report is reviewed during the office visit.

The computerized scoring system produces a “lab report” for the physician to review during the office visit. The “psychiatric lab report” gives a severity score for each of the seven conditions, displaying the out-of-range scores that signal clinically significant findings for the disorder. Whenever a severity score is out of range, a note appears on the report indicating whether the patient meets DSM-IV criteria for a psychiatric disorder. Additionally, the report provides a listing of the specific symptoms that contribute to the diagnosis. The combination of information about each score—out-of-range values and those meeting diagnostic criteria—allows clinicians to identify symptoms that might have substantial impact on health and functioning but do not meet diagnostic criteria, such as symptoms associated with subsyndromal depression.

The developer of the QPD panel, Jonathan Shedler, PhD, is a research and consulting psychologist with academic appointments at Harvard Medical School and Adelphi University. During his research work at Adelphi, Dr. Shedler began the work that led to the development of the QPD panel. Over several years, he conducted focus groups with physicians to learn why they didn’t diagnose depression and how they could use existing tools to assist in screening and detection. He learned that the existing screening tools, although effective, were impractical for use in the primary care setting. Physicians were adamant that they would not use anything that required additional time from them or their office staff: this caveat ruled out any instrument that needed hand scoring or training for interpretation of scores. During the focus groups, he also learned that what physicians wanted was a reliable diagnostic tool, similar to a lab test, for the full range of psychiatric conditions likely to be seen in primary care and they wanted it to be generated from patient self-reports.

During the development process, Dr. Shedler field-tested the questions and collected response information pertaining to its use by both patients and physicians. The instrument contains up to 200 questions, but the patient answers only those questions relevant, chosen according to the branching logic of the instrument. For a single diagnosis of major depressive disorder, the patient would answer 59 questions, in about four minutes. Questions are presented in a non-intrusive manner, beginning with questions that appear to be about physical symptoms, such as “My hands or feet often feel cold or clammy.” In fact, the question is relevant to assessing anxiety. Questions that seem to be about physical symptoms come first, because these questions are consistent with what a patient expects to be asked in a doctor’s office. The test leads gradually to questions that are more psychological in content. Both patient and physician response to the instrument has been extremely positive.

The last of several validation studies are being completed now and validity findings will be published soon. Dr. Shedler reports that validation data to date is very strong. He has found that the depression severity scores from the QPD Panel correlate highly with established depression scales, including the CES-D, Beck Depression Inventory, Zung Depression Scale, and the Hamilton Depression Scale (correlations with all scales are in the area of .80). Also, using the SCID interview as a reference standard, he reports that the sensitivity and specificity for the DSM-IV diagnosis of major depression are .92 and .85 respectively. Publications from several validation studies are planned.

The instrument is currently available from Shedler and Associates for several different computerized platforms. For more information, contact Jonathan Shedler, PhD, 42601 East Highway 82, Aspen, CO 81611. Tel: 970-925-4686; Fax: 970-925-9271; e-mail: Shedler@rof.net.

Excerpt from Sample Report

Physician: Dr. Joel Fleischman		Date: 2/8/95
Patient: Smith, John		Sex: M
Ref No: 117588193		Age: 27

**-------------------- Diagnostic Report --------------------**
Test	Results		Reference Age
	within range	out of range
Depression		21	0-10
Anxiety	8		0-10
Panic Disorder	2		0-8
OCD	0		0-3
Bulimia	0		0-4
Alcohol/SA	3		0-6

Note: Patient appears to meet DSM-IV Criteria for Major Depressive Episode. Patient expresses suicidal ideation

Initiatives
Summaries of organizational efforts to evaluate
outcomes of depression care

The Outcomes Roundtable is a multidisciplinary and multistakeholder group developing standards within the mental health arena for evaluating outcomes. The Feasibility Project will provide experience to help identify outcomes assessment data collection methods that are effective with client populations who have severe and persistent mental disorders.

The Feasibility Project collects information from individuals with depression or schizophrenia and involves seventeen sites, where data collection began in the late summer of 1996. Participating sites have selected at least 40 individuals with either of the two disorders to comprise samples for major depressive disorder and schizophrenia. Information collected includes: information on current treatment and rehabilitative services, assistance with work, housing and related needs, symptoms, functional status, quality of life, and satisfaction with care outcomes. The SF-12 Health Survey is being used for functional status; the other surveys have been constructed in response to the needs identified by members of the Roundtable. Questionnaires are administered at the beginning of data collection and at four month follow-up. With the individual’s consent, questionnaires for supporting information are also sent to a family member and the principal provider. Initial data collection is scheduled for completion in January 1997, follow-up by May 1997, with a report expected soon after.

For more information:
Ann Skinner, MSW, Johns Hopkins University, 624 North Broadway, Baltimore, MD 21205. Tel: 410-614-4022; FAX: 410-955-0470; E-mail: askinner@phnet.sph.jhu.edu.

For information about all Outcomes Roundtable activities and publications, contact Jackie Keller at the National Alliance for the Mentally Ill, at 703-516-7969.

Center for Outcomes Research and Effectiveness Medicaid Administrative Project
Located at the University of Arkansas for Medical Sciences, the Center for Outcomes Research and Effectiveness (CORE) conducts applied research and is directed by G. Richard Smith, MD, Professor and Vice Chairman of the Department of Psychiatry. Researchers from CORE developed the Depression Outcomes Module. The Medicaid Administrative Project is a study in progress, directed by Brenda Booth, PhD, Associate Director of the Centers for Mental Health Care Research at the University of Arkansas. Designed to provide data to the Arkansas Division of Mental Health Services (DMHS), the Community Mental Health Centers (CMHC), and the Arkansas Medicaid Office, the project follows the progress of individuals receiving crisis care from the DMHS or CMHCs for major depression (and schizophrenia). Using the Depression Outcomes Module (DOM), researchers evaluate individuals with major depression at baseline and at four, eight and 12 months. Following the 12-month follow-up, the researchers also conduct a record review. The DOM has been customized for this population and questions about alcohol use have been added. Early reports suggest that drinking more than a minimal amount of alcohol may predict hospitalization.

For more information:
Suzanne G. McCarthy, Applied Projects Coordinator, Center for Outcomes Research and Effectiveness, Freeway Medical Tower, 5800 West 10th Street, Suite 605, Little Rock, Arkansas 72204. Tel: 501-660-7550; FAX: 501-660-7543.

Performance Measurement
Foundation for Accountability (FACCT) System Performance Measures: Major Depressive Disorder

FACCT has released a set of measures to evaluate outcomes of care for patients diagnosed with Major Depressive Disorder (MDD). The quality measurement set for MDD measures 1) how many patients feel sufficiently connected to their providers and are able to get the care they need over time; 2) how satisfied patients are with the respect and attention given them by their providers; 3) how satisfied patients are with the skill and responsiveness of their providers and the results of their treatment; 4) how many patients’ conditions improve significantly within six months; 5) to what extent patients are able to continue their daily work activities; and 6) how well patients cope, socially and emotionally, after treatment.

Patients surveys to be completed are: the D-Ark (select components from the Depression Outcomes Module for symptom severity, lost work time and disability days, etc.), the SF-36 Health Survey (for functional health status and quality of life), and the Behavioral Healthcare Rating of Satisfaction (BHRS) for satisfaction with care. An additional measure to identify patients who are not receiving adequate follow-up is still under development.

For more information:
Christina Bethell, PhD, FACCT; Tel: 503-223-2228; Fax: 503-223-4336.

Large Scale Studies
Agency for Health Care Policy and Research (AHCPR) Patient Outcome Research Team (PORT) Improving Cost-Effectiveness of Prepaid Depression Care

This PORT-II is a large demonstration project designed to address issues for depressed patients in primary care including patients known to be depressed and those not previously so recognized. This five-year project will evaluate the cost effectiveness of alternative practice strategies and of specific treatments for depression in prepaid group practices in several areas of the US. Patients will be randomly assigned to their usual care or to one of two interventions (improving psychotropic medication management or improving counseling for depression). Within the clinics, patients with current major depression or chronic depression will be identified and followed for two years. This study emphasizes policy-relevant outcomes by examining the trade-offs among direct and indirect (social) costs, functioning outcomes, and cost effectiveness and cost utility, or value of care in terms of serious functioning limitations removed per dollar spent. Kenneth B. Wells, MD, MPH, is the principal investigator for the project, which began in July 1994 and will extend until June 1999.

For more information:
Charlotte A. Mullican, AHCPR Project Officer, at 301-594-1485.

National Institute of Mental Health Effectiveness of AHCPR Depression Guidelines

A cooperative agreement, now in its third year of funding, is studying the impact of practice guidelines for major depression in primary care, alternative models of disseminating practice guidelines, and the effects of guideline-concordant care on patient health-related outcomes. The three coordinating research sites are University of Arkansas, Johns Hopkins University, and RAND/UCLA. Each project is using an experimental design to test quality improvement or guideline dissemination strategies. All three sites and another major study, the AHCPR PORT-II (see above), are collecting comparable data with comparable longitudinal data points.

Evaluation in Health Plans
Harvard Pilgrim Health Care (HPHC) Depression Detection and Management
in Primary Care

In response to the Medical Directors Group’s priority, HPHC implemented a pilot program to detect and treat depression at one of the plan’s sites in October 1996. The pilot includes administration of the dichotomous (yes/no) three-item depression screener (see Resources) to patients on well visits, prior to examination by the clinician. If the patient answers “yes” to any of the questions, the screen is positive. For patients who screen positive, the clinician discusses the responses and interviews the patient using a two-page assessment form containing questions to help diagnose depression according to DSM-IV criteria. Six courses of follow-up are available to the clinician, with approval by the patients, whether or not the diagnosis is positive to the DSM-IV criteria. Extensive educational support is available for clinicians and patients; clinicians also have access to a psychiatric consultant as needed.

Preliminary results have found that of the 256 patients screened while on a well visit, 121 (47%) screened positive for depression. Of those who screened positive, 32 were subsequently diagnosed as positive for depression via the assessment form: 26% of all who screened positive and 13% of all well visits screened. The pilot program will continue until December 1996.

For more information:
Kerry Coughlin, Clinical Quality Management, Harvard Pilgrim Health Care, 10 Brookline Place West, Brookline, MA 02146-7229. Tel: 617-730-7809; Fax: 617-731-8249.

In May 1995, PhyCor, Inc., launched a Patient-Centered Outcomes Initiative within the physician-practice management company that operates multi-specialty clinics and managed independent practice associations (IPAs).

A year later, 21 clinics with over 2,200 enrolled patients have contributed data to the pooled data set. Very early in the data collection, analysts observed a much higher rate of depression in the enrolled asthmatic and diabetic patients than the expected rate of 12% for a normal population, based on responses to the three-question depression screener (see Resources) included in the health status survey. To pursue the diagnosis of depression in those who screened positive, PhyCor worked with the Center for Outcomes Research and Effectiveness (CORE) and Ofstead & Associates to begin a pilot study at three clinics. The study found that one third (17 of 52) patients who screened positive were actually diagnosed with Major Depressive Disorder or Dysthymia or both.

PhyCor, Inc., has its headquarters in Nashville, Tennessee and operates 44 clinics with approximately 2,650 physicians in 23 states and manages IPAs with over 8,000 physicians in 15 markets.

For more information:
Ron Loeppke, MD, Vice President for Health Affairs and Corporate Medical Director, PhyCor, Inc., 30 Burton Hills Boulevard, Nashville, TN 37215.

Depression/Health Status Survey Blue Cross Blue Shield (BCBS) of Massachusetts

In response to a purchaser/provider consortium initiative to measure functional outcomes in the mental health population, BCBS of Massachusetts developed a tool for use with mental health service users within their plan: Customer Reported Outcomes & Satisfaction Scales (CROSS). The tool contains about 80 questions, including demographic and comorbidity information, within three dimensions of functioning (physical, social and affective) and a range of other areas that measure general health and satisfaction with functioning. BCBS of Massachusetts has used the tool in a cross-sectional study of their within-plan Medicaid population to evaluate the health status and levels of depression as well as the utilization of both medical and mental health services. 1803 completed responses were obtained from a mailing to 6500 Medicaid-eligible enrollees. Data collected from the survey will be used for future analyses to support policy and planning.

For more information:
John Hayes Mason, PhD, Director of Health Services Evaluation, Blue Cross Blue Shield of Massachusetts, Tel: 617-832-4909; Fax: 617-832-4915.

Resources
Measures for Screening for Depression and Evaluating
Outcomes of Treatment

Major sources for the measures described below were: Outcomes Assessment in Clinical Practice, by Li Sederer and Barbara Dickey, Baltimore, MD: Williams and Wilkins, 1996 and Measuring Disease by Ann Bowling, Bristol, PA: Open University Press, 1995.

An eight-item self-report questionnaire was developed at RAND in the late 1980s to screen for major depression and Dysthymia. The screener begins with three dichotomous questions and continues with questions that contain four-point response ranges. The tool uses a scoring algorithm to determine a cut-off.

Burnam MA, Wells KB, Leake B, Landsverk J. Development of a brief screening instrument for detecting depressive disorders. Medical Care 1988, 26: 775-89.

The first three questions of the RAND screener (which also appear on the Diagnostic Interview Schedule (DIS) ) were analyzed for their ability to predict depression in a population already diagnosed with depression. Positive scores can be calculated by hand. The presence of a Yes response to any question indicates a positive screen. Recent informal results of confirmed diagnoses following a positive screening are described on page 6 (PhyCor and Harvard Pilgrim Health Plan).

Rost K, Burnam MA, Smith GR. Development of Screeners for Depressive Disorders and Substance Disorder History. Medical Care 1993, 31(3):189-200.

Berwick et al described the performance of five items from the RAND battery as a screener for general mental health disorders, including depression. These five items comprise the mental health scale of the SF-36 Health Survey. Scores below 52 on the mental health scale indicate risk for mental disorder. (This score represents the transformed value of the cut-off point in the Berwick study for SF-36 scale scoring.)

Berwick DM, Murphy JA, Goldman PA, Ware JE, Barsky AJ, and Weinstein MC. Performance of a five-item mental health screening test. Medical Care 1991; 26 (2):169-176.

Weinstein MC, Berwick DM, Goldman PA, Murphy JM, and Barsky AJ. A Comparison of Three Psychiatric Screening Tests Using Receiver Operating Characteristic (ROC) Analysis. Medical Care, June 1989 27(6); 593-607.

This is a 20-item self-report depression scale developed to assess the frequency and severity of depression in the general population. Six symptom areas are addressed: depressed mood, feelings of guilt/worthlessness, helplessness/hopelessness, psychomotor retardation, loss of appetite, and sleep disturbance. The scale produces a single score, ranging from 0 to 60 (higher score indicates higher symptomatology). Scores of 16 and above are used as the cut-off for a positive screen.

Eaton WW, Kessler LG. Rates of symptoms of depression in a national sample. Am J of Epid 1981, 114:528-38.

Radloff LS, Locke BZ. The community mental health assessment survey and the CES-D scale. In: Weissman MM, Myers JK, Ross CE (eds), Community Surveys of Psychiatric Disorders. New Brunswick, New Jersey: Rutgers University Press, 1986.

The CES-D is available from the Epidemiology and Psychopathology Research Branch, Division of Epidemiology and Services Research, Department of Health and Human Services, National Institutes of Health, Bethesda, MD 20892.

A children’s version of the CES-D is available from Dr. MM Weissman, Department of Psychiatry, College of Physicians and Surgeons of Columbia University, 722 West 168th Street, Box 14, New York, NY 10032.

The BSI is a 53-item subset of the 90-item Symptom Checklist-90-Revised (SCL-90-R). The test measures nine primary symptom dimensions (including depression) and three global indices that measure severity, distress, and number of symptoms reported. The tests can be used either as a screening tool or as a measurement of patient progress during and after treatment.

Derogatis LR, Spencer PM. The Brief Symptom Inventory (BSI): Administration, Scoring and Procedures Manual I. Baltimore, MD: Johns Hopkins University Press, 1982.

The BSI is exclusively published and distributed for a fee by NCS Assessments, 5605 Green Circle Drive, Minnetonka, MN 55343.

The Beck Depression Inventory is a 21-item scale, administered by self-report, that measures symptoms such as mood, suicidal ideas, irritability, social withdrawal, work difficulty, insomnia and others. Respondents rate items with a 4-point scale; the total score range is from 0-63. Two subscales may also be scored: cognitive-affective and somatic-performance.

Beck AT, Steer RA, Garbin MG. Psychometric properties of the Beck Depression Inventory: twenty-five years of evaluation. Clin Psycol Rev 1988; 8:77-100.

The BDI is available for a fee from The Psychological Corporation, 666 Academic Court, San Antonio, TX 78204-2498.

The HDS is an observer rating scale, usually employing two trained observers, comprised of 21 items covering depressed mood, feelings of guilt, suicidal ideation, insomnia, and others. The instrument is used only for establishing the severity of depression following a diagnosis.

Hamilton M. Development of a rating scale for primary depressive illness. Br J of Soc Clin Psych 1967:6;278-296.

The HDS is available from Professor M. Hamilton, Department of Psychiatry, University of Leeds, Woodhouse Lane, Leeds, LS2 9JT, UK.

The DOM is a set of four different forms to allow a comprehensive evaluation of the types of care received for major depression, the outcomes of that care, and the patient characteristics that influence outcomes and type of care. The module consists of two patient forms (administered by either self-report or interviewer self-report), a clinician form, and a medical record review form. A Patient Baseline Assessment, with 80 items, measures diagnosis at baseline using DSM-IV criteria, general functioning with the SF-36 Health Survey, symptom severity, number of bed days, psychiatric history, and sociodemographic characteristics. The Patient Follow-Up Assessment, administered 4 months after baseline and until the condition is resolved, is similar to the Baseline questionnaire. Both forms take about 25 minutes to complete. A Clinician Baseline Assessment consists of 20 items and takes about 5 minutes to complete. The final form, the Medical Record Review, consists of 11 items and is completed every 4 months during treatment by a clerical worker.

Rost K, Smith GR, Burnam, MA, Burns BJ. Measuring the Outcomes of Care for Mental Health Program: The Case of Depressive Disorders. Medical Care May 1992: 30(5) Supp, MS 266-273.

The DOM is available from G. Richard Smith, MD, Center for Outcomes Research and Effectiveness, Freeway Medical Tower, 5800 West 10th Street, Suite 605, Little Rock, AR 72204. Tel: 501-660-7550; Fax: 501-660-7543.

The DOM is also distributed as the Depression TyPE from the Health Outcomes Institute, 2001 Killebrew Drive, Suite 122, Bloomington, MN 55425. Tel: 612-858-9188; FAX: 612-858-9189.

The Basis-32 is a 32-item self-report, or interview-administered, questionnaire that permits assessment of self-reported difficulty in symptoms and functioning for patients or clients receiving mental health services. Degree of difficulty is assessed in five major areas: relation to self/others, daily living/role functioning skills, depression/anxiety, impulsive/addictive behavior (including substance abuse) and psychosis. The tool is designed for use in research studies assessing outcome of psychiatric care from the patient’s or client’s perspective.

Eisen SV, Dill DL, Grob MC. Reliability and validity of a brief patient-report instrument for psychiatric hospital care. Hosp Comm Psych 1992; 42(11):1120-1126.

Permission and documentation to use the instrument can be obtained by contacting Susan V. Eisen, PhD, Department of Mental Health Services Research, McLean Hospital, 115 Mill Street, Belmont, MA 02178. FAX: 617-855-2948.

Generic Functioning and Quality of Life
The following instruments distributed by the Trust have been used to measure functional status and quality of life in evaluations of individuals suffering from depression. Recent studies are reported below.

Fisher DC, Lake KD, Reutzel TJ, Emery RW. Changes in health-related quality of life in heart transplant recipients. J Heart Lung Transplant. 1995 Mar-Apr;14(2):373-381. SIP used in conjunction with Beck Depression Inventory 4, 8, 12, 24, 36, 48, and 60 months after surgery.

Williams JW, Kerber CA, Mulrow CD, Medina A, Aguilar C. Depressive Disorders in Primary Care: Prevalence, Functional Disability, and Identification. J Gen Int Med Jan 1995: 10; 7-11. Reported on the functional impact of both major depression and subsyndromal depression.

Beusterien KM, Steinwald B, Ware JE. Usefulness of the SF-36 Health Survey in Measuring Health Outcomes in the Depressed Elderly. J Ger Psych Neur 1996; 9:1-9. Examines usefulness of the SF-36 Mental Health Scale and the Mental Component Summary Scale to estimate the burden of depression and in monitoring changes in functional health and well-being over time among the depressed elderly.

Heiligenstein JH, Ware JE Jr, Beusterien KM, Roback PJ, Andrejasich C, Tollefson GD. Acute effects of fluoxetine versus placebo on functional health and well-being in late-life depression. Int Psychogeriatr. 1995;7 Suppl:125-37. Differences in changes of scores between groups from baseline to six weeks were significant (favoring the fluoxetine group) for scales of mental health, role-emotional, physical functioning, and bodily pain.

Walker V,Streiner DL, Novosel S, Rocchi A, Levine MA, Dean DM. J Health-related quality of life in patients with major depression who are treated with moclobemide. Clin Psychopharmacol. 1995 Aug;15(4 Suppl 2):60S-67S. The GHQ and seven domains of the SF-36 detected a statistically significant linear trend (improvement) over time.

Souetre E, Martin P, Lozet H, Monteban H. Quality of life in depressed patients: comparison of fluoxetine and major tricyclic antidepressants. Int Clin Psychopharmacol. 1996 Mar;11(1):45-52. Fluoxetine treatment may be associated with higher levels of social functioning and health perception than usual TCA treatment.

The following articles were identified through a Medline search on Depression
(major topics only) and Quality of Life, Outcome Assessment (health care), Health Status, Risk Assessment or Severity of Illness. The abstracts have been abbreviated and arranged in reverse chronological order within five categories: Outcome Assessment in Practice Settings, Pharmaceutical Evaluations, Quality of Life/Functional Status Measurement, Risk Factors or Predictors.

Geriatric patients with depression. Improving outcomes using a multidisciplinary clinical path model.

Bultema JK, Mailliard L, Getzfrid MK, Lerner RD, Colone M.. J Nurs Adm. 1996 Jan;26(1):31-8.

The authors describe a psychiatric clinical pathway for geriatric patients with depression and identify common multidisciplinary interventions and a pattern of outcomes over the course of treatment for these patients.

Simon GE, Lin EH, Katon W, Saunders K, VonKorff M, Walker E, Bush T, Robinson P. J Gen Intern Med. 1995 Dec;10(12):663-70.

Efforts to increase the intensity of depression treatment in primary care should focus on the subgroup of patients who fail to respond to initial treatment.

The process and outcomes of care for major depression in rural family practice settings.

Rost K, Williams C, Wherry J, Smith GR Jr. J Rural Health. 1995 Spring;11(2):114-21.

The Depression Outcomes Module appears to be a reliable and valid instrument for monitoring the outcomes of care for major depression in family practice settings.

The best strategy for making care for depression more cost-effective is through quality improvement, not through changing specialty mix.

Quality of life in depressed patients: comparison of fluoxetine and major tricyclic antidepressants.

Souetre E, Martin P, Lozet H, Monteban H. Int Clin Psychopharmacol. 1996 Mar;11(1):45-52.

Fluoxetine treatment may be associated with higher levels of social functioning and health perception than usual TCA treatment.

Acute effects of fluoxetine versus placebo on functional health and well-being in late-life depression.

Heiligenstein JH, Ware JE Jr, Beusterien KM, Roback PJ, Andrejasich C, Tollefson GD. Int Psychogeriatr. 1995;7 Suppl:125-37.

Improvements observed in the fluoxetine group were both clinically and socially significant.

Health-related quality of life in patients with major depression who are treated with moclobemide.

Walker V, Streiner DL, Novosel S, Rocchi A, Levine MA, Dean DM. J Clin Psychopharmacol. 1995 Aug;15(4 Suppl 2):60S-67S.

The GHQ and seven domains of the SF-36 detected a statistically significant linear trend (improvement) over time.

The selective serotonin reuptake inhibitors offer a preferred choice of treatment in view of their improved safety and tolerability profile.

The association of physical health and depressive symptoms in the older population: age and sex differences.

Beekman AT, Kriegsman DM, Deeg DJ, van Tilburg W. Soc Psychiatry Psychiatr Epidemiol. 1995 Jan;30(1):32-8

Physical health and depression are closely related in the elderly. Depression is sufficiently different from physical health to be distinguished from it, and sufficiently related to be relevant for further study.

Treating depression in medical conditions may improve quality of life [news].

Describes a depression-specific measure of quality of life, the Quality of Life in Depression Scale, has been developed for use in clinical trials.

Scales of depression, ill-being and the quality of life—is there any difference? An assay in taxonomy.

An alternative taxonomy for the classification of depression, ill-being and quality of life is discussed.

Changes in daily life experience associated with clinical improvement in depression.

Barge-Schaapveld DQ, Nicolson NA, van der Hoop RG, De Vries MW. J Affect Disord. 1995 May 17;34(2):139-54.

Experience sampling method provides quantitative evidence of changes in real life time use and subjective experience accompanying clinical improvement.

Fisher DC, Lake KD, Reutzel TJ, Emery RW. J Heart Lung Transplant. 1995 Mar-Apr;14(2):373-381.

Patients did not generally experience problems with depression within the first 4 months after heart transplant.

Depression in scleroderma is a debilitating comorbid condition that should be recognized and treated in its own right.

Symptoms of depression, acute myocardial infarction, and total mortality in a community sample.

High levels of depressive symptomatology are associated with increased risks of MI and mortality. This risk factor is best viewed as a continuous variable that represents a chronic psychological characteristic rather than a discrete and episodic psychiatric condition.

Patterns of depressed mood in midlife women; observations from the Seattle Midlife Women’s Health Study.

Menopausal status did not differentiate women with patterns of depressed mood from those without depressed mood.

Psychological predictors of subsequent medical care among patients hospitalized with cardiac disease.

Levine JB, Covino NA, Slack WV, Safran C, Safran DB, Boro JE, Davis RB, Buchanan GM, Gervino EV. J Cardpulm Rehabil. 1996 Mar-Apr;16(2):109-16.

Psychological depression appears to be an important predictor of rehospitalization among persons who have been admitted with coronary artery disease.

Sadovnick AD, Remick RA. Allen J, Swartz E, Yee IM, et al. Neurology. 1996 Mar;46(3):628-32.

Although there appears to be a very high rate of depression among MS patients, the data for their first-degree relatives do not support a clear genetic basis for this depression.

SHEP Cooperative Research Group (Systoloc Hypertension in the elderly). Wassertheil-Smoller S, Applegate WB, Berge K, Chang CJ, Davis BR, Grimm R Jr, Kostis J, Pressel S, Schron E. Arch Intern Med. 1996 Mar 11;156(5):553-61.

Among elderly persons an increase in depressive symptoms over time may be a marker for subsequent major disease events.

Depressed mood and the incidence of Alzheimer’s disease in the elderly living in the community.

Devanand DP, Sano M, Tang MX, Taylor S, Gurland BJ, Wilder D, Stern Y, Mayeux R. Arch Gen Psychiatry. 1996 Feb;53(2):175-82.

Depressed mood moderately increased the risk of developing dementia, primarily Alzheimer’s disease.

Does physical improvement reduce depressive symptoms in HIV-infected medical inpatients?

The etiology and management of depressive symptoms in HIV-infected medical inpatients may differ depending on the initial severity of depressive symptoms.

Association of depression with reduced heart rate variability in coronary artery disease.

Carney RM, Saunders RD, Freedland KE, Stein P, Rich MW, Jaffe AS.. Am J Cardiol. 1995 Sep 15;76(8):562-4.

Decreased heart rate (HR) variability may help explain the increased risk for cardiac mortality and morbidity in depressed CAD patients.

Self-reported depressive symptoms in association with medication exposures among medical inpatients: a cross-sectional study.

Depressive symptoms among medical inpatients have a biopsychosocial etiology. Corticosteroid exposure may be a biological risk factor for depressive symptoms in this population.

When depression is seen for the first time late in life, genetic and psychosocial factors appear to be less important than structural changes in the brain due to atherosclerosis.

Based on an interview with Dana Gelb Safran, ScD, Senior Policy Analyst,
The Health Institute, New England Medical Center, Boston, MA

comprehensiveness, coordination, and accountability. In a revised definition published in 1994, the IOM has made explicit that primary care requires a “sustained partnership” and a whole-person orientation to care. Work begun in 1993 by Dana Gelb Safran, ScD, and her colleagues at The Health Institute has operationalized this definition for primary care performance assessment and produced the Primary Care Assessment Survey (PCAS). With funding from an Agency for Health Care Policy and Research (AHCPR) grant, Dr. Safran is using the PCAS to study differences in primary care performance across different models of health care delivery. The study has provided substantial data (n=7000) to assess the measurement properties of the PCAS.

This article reports on the development of the conceptual model underlying the PCAS and preliminary results of field tests and initial projects using the survey to evaluate performance in primary care.

Primary care has traditionally resisted a precise, linear definition. Over the past few years Dr. Safran has contributed to the evolution of a definition that addresses the multi-dimensional nature of primary care and is viable for use in research and policy. In 1994, she and The Health Institute convened a Primary Care Consensus Conference, which yielded a consensus statement regarding the definition of primary care. The results of that conference were summarized in a background paper requested by the IOM Committee on the Future of Primary Care and were reflected in that committee’s report.

The PCAS operationalizes the IOM definition of primary care, measuring each of the attributes identified: access, continuity, comprehensiveness, integration, clinical quality, interpersonal treatment and trust. It is designed specifically to assess the patient’s relationship with a clinician or set of clinicians. “To measure primary care in a manner consistent with the IOM definition, we ask patients to focus on the clinician or set of clinicians whom they regard to be their regular providers and to evaluate the care that they’ve received from those providers over the course of time,” Dr. Safran explained. Comprised of 51 items forming 11 scales, the survey is written at a 5th grade reading level and takes approximately seven minutes to complete. (See Figure 1.) The items per scale range from one to eight. Because the reliability and validity of patient evaluations of technical aspects of care are least well-documented in the research literature, there is only one item that asks patients to evaluate technical aspects of their care: thoroughness of physical exam. But beyond the technical aspects of care, there are many aspects of care that are important to evaluate.

“One of the significant contributions that we’ve made in our conceptual framework,” explained Dr. Safran, “is that we have differentiated between interpersonal treatment and communication. Those concepts have typically been grouped together as the interpersonal quality of care and set apart from the clinical quality of care.” In the PCAS model, communication is considered an essential element of the clinical quality of care because doctor and patient can’t appropriately assess symptoms, or move forward to a diagnosis or treatment plan without clear communication back and forth. The interpersonal quality—how nice the clinician is, how warm, caring, concerned and respectful—is measured separately from communication since the interpersonal qualities are important but not necessarily part of the clinical quality of care.”

The 11 primary care scales each range from 0 to 100 points, with higher scores indicating more favorable performance. From Year 1 results of the AHCPR-funded study, the PCAS scales have been shown to discriminate, in significant ways, primary care performance under different models of health care delivery and to correlate significantly to important outcomes of care.

The attributes measured by the PCAS are all essential to primary care. However, some of the attributes are shared with other care settings. Dr. Safran noted that most of the dimensions could be meaningfully applied and interpreted in any clinician-patient relationship that has a longitudinal component. She told us, “Most of the concepts we call ‘distinguishing features’ would not be relevant to assessing the interaction between a consultant (one-time referral) and a patient. But they could become relevant if an ongoing relationship is established with a specialist.”

“Continuity is one of the domains that’s considered a distinguishing attribute of primary care but not typically required of specialty care. However, for a patient who has developed cancer, one would expect a sustained relationship and visit-to-visit continuity with the oncologist.

“Another example is the whole-person orientation, a component of the comprehensiveness domain. Clinicians have identified the whole-person focus as a most important and unique feature of primary care. One could argue that the whole person approach could benefit the patient-clinician interaction under any circumstance, but it’s a necessary feature of primary care and not considered a necessary feature of consultative care.”

Dr. Safran and her colleagues have begun to link attributes of care to outcomes. “There’s been a fair amount of work linking individual attributes of care to certain outcomes, like adherence or satisfaction, but rarely have there been studies that allow you to get a sense of the relative importance of attributes to these outcomes. We might know that communication is important for adherence, likewise continuity and interpersonal treatment. But how do you know which is the most important?” Dr. Safran said.

In a study of Massachusetts state workers, approximately 50 supplementary items were added to the survey to let researchers measure selected outcomes and study the relationship between the 11 primary scales and the outcomes. Data were collected on three outcomes: patients’ adherence to their doctor’s advice, overall satisfaction, patient’s self-reported 4-year health transitions. The analyses of these data are a first step in answering the question of the relative importance of the primary care variables and the actual relationships.*

Some quite provocative relationships were observed in that study. “For adherence,” Dr. Safran reflected, “we found two attributes of primary care were the most important correlates of patient adherence: patients’ trust in their physicians and their sense that the physician had a comprehensive contextual knowledge of them, a whole-person focus to care.

“With respect to patient satisfaction as an outcome, trust explained 35% of the variance in satisfaction. For self-reported health transitions, there were five attributes of care that were about equally important correlates of health improvements: trust, comprehensive knowledge, integration, thoroughness of physical exams, and communication. These results are sufficiently interesting to suggest that moving forward with some longitudinal work to identify whether there are in fact causal mechanisms behind these relationships is a crucial next step. Then we can in full confidence say to a practice, if you want to improve adherence or if you want to improve satisfaction, the most important thing is trust, or the most important thing to assure is that clinicians gain a whole-person understanding of their primary care patients.”

A key design feature of the PCAS relates to the distinction between rating style items and report style items. Rating style items have an evaluative response set (such as “Excellent” to “Very Poor”), as opposed to report style items that ask specific quantitative or yes/no responses (such as “How many minutes did you wait?” or “How many times has something like this happened to you?”). The combined use of ratings and reports is allowing the investigators to study whether patients’ evaluations of their care reflect an objective reality, their personal values and expectations, or some of both. In the PCAS, there are eight questions for which both report style and rating style items are paired.

Although still in the very beginning stages of analyzing data, Dr. Safran already observes that patient reports (such as ‘How long did you wait?’) predict a large majority of the variance in how patients evaluate care—more than, for example, demographics or health status. Dr. Safran explained, “You can’t guess, based on somebody’s age or sex, how a patient will evaluate care. And that has been quite consistently found by others in the literature. We found the same thing. Evaluations are very closely linked to the reports of what actually happened. Although those reports don’t explain everything, it’s a small part compared to reality. That’s why I don’t think of our assessment as ‘patient satisfaction.’ To me, and I think to a lot of other people, labeling the field as patient satisfaction implies that what a patient tells you is merely a perception. Our data strongly suggest that the majority of what patients indicate through their evaluations is reality, not just personal expectations.”

*A longitudinal study design would be required to test this. However, because the Massachusetts study design was cross-sectional, the investigators cannot be certain about whether a causal relationship exists between the attributes of primary care and outcomes.

Based on an interview with Neil K. Aaronson, PhD, Head, Division of Psychosocial Research and Epidemiology, The Netherlands Cancer Institute, Amsterdam

in daily clinical oncology practice at the Nether-lands Cancer Institute. Ten of the 15 medical oncologists at the Antoni van Leeuwenhoek Huis Department of Internal Medicine in Amsterdam are participating in an intervention study to facilitate doctor-patient communication in outpatient palliative care settings. The primary intervention of this four-year study funded by the Dutch Cancer Society is patient completion of the EORTC QLQ-C30 immediately prior to the doctor visit and review of the computerized score in graph form by both patient and doctor during the outpatient visit.

The Antoni van Leeuwenhoek Huis is a specialized cancer treatment hospital, affiliated with the research institute, where a large number of patients are seen on an outpatient basis. This intervention study, directed by Neil A. Aaronson, PhD, of the Netherlands Cancer Institute, was conceived as a response to a phenomenon he and his colleagues refer to as the "doorknob phenomenon." Dr. Aaronson explained, "A lot of patients tend to go through their outpatient control visit with their oncologist focused on the results of the physical exam, blood tests and other objective measures. Then they get dressed, get ready to leave and, with one hand on the doorknob, say ‘By the way, doctor...’ Then will come the complaints about fatigue or other types of problems which they hesitate to discuss with their doctors. The strategy seems to be to hold off until it’s almost a certainty that there’s little room to talk about concerns, but to raise them anyway. This phenomenon was a signal to us that it might make sense to try to accelerate the process up front."

Dr. Aaronson chose the EORTC QLQ-C30 because many of the physicians have experience with this questionnaire from their participation in clinical trials. The questionnaire is in wide use throughout the research institute and the affiliated hospital. Another feature that contributed to its selection as the intervention tool is its emphasis on symptoms. Dr. Aaronson has found that clinicians can more readily understand what symptom scale scores mean, whereas it is not as easy for clinicians to interpret some of the functioning scales.

The EORTC QLQ-C30 questionnaire consists of 30 items, organized into five functional scales, three symptom scales, a global quality of life scale, and single questions on additional symptoms. Each of the scales and questions is scored to produce a score ranging from 0 to 100. A high score denotes better functioning for the global quality of life scale and the functional scales: physical, role, cognitive, emotional, and social. For the symptom scales, higher scores mean more symptomatology. The symptom scales consist of fatigue, nausea, and pain, and additional single-item scales including dyspnea, sleep, appetite, constipation and diarrhea.

For this intervention study, the patient completes the questionnaire in the waiting room prior to the visit with the doctor. Scores for the current visit, and any previous visit, are printed in graph form in time for the patient and doctor to view the result during the visit. The cumulative information yields a profile over time of changes in functioning and symptom levels. For this study, an additional question has been added to the EORTC QLQ-C30 that asks the patient to identify the three or four most bothersome limitations or symptoms. The answers are displayed in text under the appopriate graph. (A sample graph is included in Figure 1.)

The current study, which will evaluate data from communication analysis, satisfaction and health status surveys, and medical record review, is a refined and more rigorous version of a small pilot study that tested the feasibility of introducing health status or quality of life measures into the daily routine of surgeons, radiotherapists, and medical oncologists. The pilot study included five clinicians and 20 patients. The primary intervention was identical to the intervention described above, but the evaluative components differed. In the pilot, there was no control group. The research team compared length of visit with the intervention to existing length of visit data without the intervention, analyzed the types of issues raised during the physician/patient encounter (including who raised each issue), and collected satisfaction information from both the physicians and the patients. The pilot study results revealed that quality of life assessments were easy to incorporate in an outpatient clinic setting: the assessments required no additional time and doctors and patients reacted positively to their use. There was also a noticeable trend toward doctors taking more responsibility for raising issues with patients, although no causal link could be attributed to the intervention.

In the current intervention study, the participating clinicians have been limited to the medical oncologists, spanning all specialties. The medical oncologist is the central figure in the palliative care of the patient, fulfilling the treating and coordinating role. If the patient is referred to another specialist, the patient eventually comes back to the oncologist. The patients targeted for this intervention are those who are beginning a course of palliative chemotherapy. "We wanted to focus on an area where there would be a fair amount of change in the health status of the patient during the study period," Dr. Aaronson explained. "With palliative chemotherapy, this is in fact the case. On the one hand you’re trying to reduce the tumor load and alleviate the symptoms. On the other hand, the treatments often have side effects. If you’re trying to assure that the physician is aware of how the patient is reacting to treatment, whether the treatment is in fact impacting on the disease in a way that is also functionally relevant, then that’s an ideal time to test this kind of intervention."

During the course of palliative outpatient treatment, the key is how the patient is reacting to treatment. In part that reaction is determined objectively by measuring tumor response and doing lab tests to find out if the patient is anemic or developing infections. Another part is determined by assuring that the treatment toxicity is within acceptable bounds. "With the information we’re providing on function and symptom experience, we’re simply expanding the range of information that the doctor is getting," Dr. Aaronson explained. "Pain is a perfect example. You can’t measure that objectively, so the doctors are depending on asking questions anyway and we’re just structuring that. For some of the more observable types of symptoms, it’s getting the patient’s perspective, which might be different from the physician’s own judgment or reliance on proxy indicators."

Although the results of the pilot study indicated that both physicians and patients reacted positively to the use of quality of life information, the research team has not ruled out the possibility that the intervention as designed might have some side effects that are unwelcome. One possibility is that physician/patient communication could actually decrease as a result of having the quality of life graph at the outpatient visit. "What we’re hoping is that the graph is a stimulus to discuss those issues," Dr. Aaronson said, "but it could also potentially shut people down. For example, what if a patient walks into the doctor’s office with a form in his hand and the doctor does not discuss some of these areas of functioning or symptoms where problems are indicated? If the patient knows the doctor has a copy of the chart, but the doctor does not address the problems, it could send a signal to the patient that the problems are not important. Because the patient just responded to key symptom and functioning concerns by completing the questionnaire, the patient might not raise the concerns verbally, which could potentially lead to less communication. We don’t think that that will happen often, but that could happen." It could also be that because the quality of life summary is included in the medical chart, the doctor in fact writes less, rather than more, making fewer notations because he or she thinks that the graphic contains adequate information about, for instance, whether the pain has gotten better or worse.

Dr. Aaronson has worked with the clinicians to educate them about the strengths and weaknesses of the quality of life tool. "One of the things I think is important in the use of any quality-of-life instrument in clinical practice settings is that everyone has to be aware of their limitations," he said. "As a part of the intervention, the physicians are clearly instructed that they should not base any decisions on the summaries that they are receiving from the questionnaires. The summaries should be used as an opening to explore issues that may come up. In other words, if the pain score of a patient changes from time one to time two, that doesn’t mean that the doctor should be changing the prescription. The doctor should be following up with the patient and probing, ‘Tell me about your pain. It looks as if the pain is getting worse.’"

The study in progress has a rigorous evaluative component in order to explore the range of outcomes possible from the intervention, from proximal to distal. The most proximal is the exchange of information. The researchers expect that by making information available that it will be more readily discussed. As a result, the physician may be better appraised of the functional status and quality of life of his or her patients and issues that are of concern to the patients may be discussed. Discussion of issues meaningful to patients may lead to higher satisfaction levels and potentially lead to action, in terms of treatment approaches. These are the types of outcomes that the research design is intended to evaluate.

The research design is a randomized control study, with cross-over. For each of the 10 participating medical oncologists, 10 to 15 patients initiating a course of chemotherapy will be assigned to the intervention group with an equal number assigned to the control group. The assignment will be made in cross-over design, with physicians randomized initially to the intervention or control group, then crossed over to the other group. The patient cohort follows the physician’s assignment. When a patient in the intervention group starts a course of palliative chemotherapy, he or she will complete the EORTC QLQ-C30 at baseline and at three consecutive follow-up visits. The evaluation consists of a comprehensive range of measures, including:

The analysis of the patient/doctor communication will be accomplished by audiotaping the visit and using the Roter Interaction Analysis System (RIAS) method to code and classify the communication. This method rates "utterances" on a number of dimensions, including content and nature. For example, is the communication a functional issue? Is a symptom being discussed? The communication could also be classified as advice-giving, emotional support-giving, or question-asking.

Physicians and patients will also complete the COOP/WONCA charts (generic health status questionnaires) at baseline and the final visit. The results will be used to compare the physician’s assessment of the patient’s health status with the patient’s self-report. At the baseline and final visits, the SF-36 Health Survey will be used to evaluate change in functional status and quality of life. The patient satisfaction measure in use is an adaptation of the hospital’s standard instrument, containing about 20 questions on dimensions such as communication, continuity of care, interpersonal manner, and technical aspects of care. The study began enrolling patients in June of 1996 and has over 87 patients currently enrolled.

An Interview with Eugene C. Nelson, DSc, MPH Professor, Community and Family Medicine, Dartmouth Medical School, and Director of Quality, Education, Measurement and Research, Lahey Hitchcock Clinic

lexicon of quality improvement for health care. This concept is one of the four directions of the clinical value compass that Dr. Nelson and his colleagues use to guide the design, implementation, and analysis of improvement projects. The clinical value compass consists of four "outcomes" directions: clinical outcomes, functional health status, satisfaction against need, and costs.1,2

The Trust asked Dr. Nelson to define the concept of satisfaction against need and illustrate the role it plays in quality improvement.

Satisfaction against need is a key component of the clinical quality compass. How do you define this term?

There are two ways. The first relates to the patient’s view of how much he or she was helped, that is, the perceived health benefit. This is perhaps the most important outcome of care from the patient’s perspective. Theoretically, a patient could have received technically great care, but it just didn’t help. Or the interaction with a clinician might have been interpersonally great, but the patient’s ability to perform in work or other daily life activities did not improve. Within the course of treatment, every patient has an actual health transition, biologically and functionally based, and a perception about the transition that occurred, the perceived health benefit.

There’s a distinction between satisfaction with services delivered and perceptions about the health outcome of that service. Satisfaction against need represents a measure of the patient’s perceived health benefit. The types of questions that address this concept include: "How much were you helped? How would you compare your health status now with before you sought care? Looking back on the care you received, how would you evaluate the health benefit you received?"

The second aspect of "satisfaction against need" relates to meeting needs that may or may not be expected by the patient. These needs include areas such as: (a) need for hope, control and safety, (b) need for respect and self-esteem, and (c) need for useful information. An example is how well clinicians communicate with patients around what patients perceive as "must know information." A patient who has suffered an acute myocardial infarction and is ready to be discharged has several levels of information needs, such as: "What are the danger signs of the heart attack? When is it safe for me to resume my normal work activities? How will I avoid having another heart attack?" The extent to which the clinical communication process meets these various information needs will either fuel or alleviate the anxiety that the person has about surviving or dying and may improve the ability to assume more control over future events.

A patient can assess the goodness of care, against both what he expects and what he needs, at every step of an episode of care, both in theory and actuality. Health transitions are individualistic, yet by coming up a level and looking at one’s last hundred patients, a clinician can find themes that emerge repeatedly or commonly. These represent the commons of expectation and the commons of need. The commons emerge out of hearing people’s stories and observing them and trying to understand them better than they might understand themselves. A key to providing great medical care would be finding ways to match what we do for patients with what they both need and expect—in the right way, at the right time, and in a manner that produces a high level of satisfaction.

Let me illustrate with a case study that involved a targeted measurement over time of satisfaction against need.

At our medical center a cardiac team, with a solid history of successful and impressive clinical improvements, asked this question: "What if we were to try to target our improvement work, in addition to working on reducing mortality and taking out unnecessary costs, to improve the care from the patient’s point of view? What would that look like?

The team spent about 15 months exploring the answers to these questions. The first thing they did was to settle on an aim: to improve cardiac care in ways patients would appreciate, that is, through the patient’s eyes. Their next step was to understand the process from the patient’s point of view. They constructed a step-by-step flowchart and identified 15 steps that most patients would take from initial recognition of the problem through to post discharge.

Then the team questioned, "How do we learn about the patient’s expectations and needs?" They convened a total of four focus groups of patients and family members: two for valve patients and two for open heart surgery patients. During the focus groups, participants were invited to tell their stories with respect to the surgeries and their episodes of care. Then patients were invited to talk about each stepping stone on their journeys based on the steps identified on the flowchart. At the conclusion of the sessions, patients also responded to the question, "Knowing what you now know, what would be the two most important things we could do for people like you?"

The team "mapped" the patient responses from the focus groups onto the flowchart and went on to create a structured, close-ended questionnaire that was administered to 50 consecutive open heart and 50 consecutive percutaneous transluminal coronary angioplasty (PTCA) patients. The questionnaire was built directly from the focus group qualitative results, with very specific questions targeted to subjects known to be important to their patients. The questionnaire set out to measure the patient’s satisfaction against need at every step within the episode of care.

The questionnaire responses provided a snapshot of the strengths and weaknesses, delights and disappointments at each step of the process: these were identified as the quality characteristics. When the team had two layers of information–qualitative and quantitative–they shared this information with the larger clinical team that provides cardiac care. At a dinner attended by about 150 staff, the team of 15 presented their findings and encouraged their colleagues to use the information to improve care. There followed a blossoming of improvement projects that were patient driven. One major project involved creating a critical path for the discharge and post-discharge process. A small project involved changing the kind of sheath used in the patient’s leg post surgery. Clinicians had been unaware that the stiff sheaths hurt and caused back pain more often than the flexible ones. So they changed to flexible sheaths–a small change, but with substantial impact on the pain experienced by some patients.

A year later the team resurveyed 100 consecutive open heart and valve patients, using most of the same questionnaire (about 80% of the same questions). The new imprint of the "goodness of care" in the care delivery process was observable empirically: the mean level of satisfaction in all the steps of the process was higher. For each step of the process, the delight zone received a substantially higher proportion of responses, the disappointment zone, substantially lower.

How do you see the relationship between generic satisfaction of care measurement and specialized efforts like the cardiac care project just described?

As a matter of business within the Dartmouth Lahey Hitchcock delivery system, every week we draw small random samples of inpatients and outpatients, community residents and other groups, and interview them over the phone to find out how they evaluate the care they have received.

This information is available in small samples for specific units. The cardiac group mentioned above could have used it but the sample would have been too small to hone in as the questionnaires they designed did.

The information collected to measure system performance, say at the hospital or departmental level, is a high level monitor. Whenever you wish to go down to the front lines, you have some choices. You can oversample, using the core questions, and then add on questions that are specific to your clinical population. For two large patient populations within our service area—bowel surgery and obstetrical care—we oversample and ask special questions. The other option is a ground-up approach.

Local, ground-up improvement efforts can benefit from customized measures. A carpal tunnel improvement team in southern New Hampshire developed their own measures when a particular surgeon became interested in outcomes research to better understand how his patients were doing and to use outcomes measurement to provide better care. The surgeons in the region who did hand surgery formed a study group and analyzed what they were currently doing to provide care to carpal tunnel patients, starting at the point when they determined a patient was a good surgical candidate and following the patients out three months post surgery. The team designed into the care process a questionnaire which is completed by patients pre-surgery, immediately post-surgery, and at three months. This tracks clinical status from the patient’s view, functional status, satisfaction, and indirect social costs. The questions that were included to measure satisfaction focused on, among other things, perceived symptom relief and satisfaction with time to return to work. The surgeons also standardized their coding for severity, co-morbidity, and hand range of motion. The clinical team learned how to master the measurement of care, in real time, in their own work setting.

Satisfaction against need is often concerned with the cross between functional and clinical outcomes of care. If the problem is hand functioning, a good question is: how satisfied are you with your ability to do what you need to do with your hand? The level of satisfaction is measured in the context of the biological and functional need to have the hand work. When looking at the value of a particular health care intervention, measuring satisfaction against a specific need will yield more precise information than will a more generic measure.

1. Nelson EC, Mohr JJ, Batalden PB, Plume SK. Improving Health Care, Part 1: The Clinical Value Compass. The Joint Commission Journal on Quality Improvement. April 1996: 22(4); 243-258.

2. Nelson EC, Mohr JJ, Batalden PB, Plume SK. Improving Health Care, Part 2: A Clinical Improvement Worksheet and Users’ Manual. The Joint Commission Journal on Quality Improvement. August 1996: 22(8); 531-548.

Based on interviews with
Mark Kosinski, MA, Health Institute, New England Medical Center
Deborah Johnson, PhD, The Mellen Institute, Cleveland Clinic Foundation
Marcia Stevic, PhD, RN, Health Services Advisory Group

guidance on evaluating the clinical relevance of scores from the developer’s published material and from examples of use in other clinical settings. This article uses the eight scales of the SF-36 Health Survey* to illustrate a variety of approaches to interpreting heath status scores.

The SF-36 Health Survey Manual and Interpretation Guide (the Manual), published by Ware et al¹, provides the developer’s suggestions for interpreting the eight scales of the SF-36 Health Survey to understand a population’s ability to work, use of health care services, quality of life, changes in health status, and functional challenges. These topics, as well as others published in the Manual, are explored through interpretation strategies that can be characterized as either content-based, criterion-based, or norm-based.

Content-Based. The content-based methods analyze the rate of endorsement for specific questions within a particular scale. Table 8.2
(p 8:6) of the Manual provides a useful quick reference for interpreting the content of each of the eight scales. For example, it explains that for the General Health scale the lowest score indicates that the respondent evaluates his or her personal health as poor and believes it likely to get worse. Whereas for the highest General Health score, the respondent evaluates his or her health as excellent. For the total sample of the US general population, the mean score for General Health is 72.0, with a standard deviation of 20.3.

Criterion-Based. The criterion-based methods link information not contained in the questionnaire (such as job loss, days in hospital) with the scores from a particular scale. The analyses in Chapter 9 of the Manual provide a reference for linking meaningful variables in outcomes studies. For example, in Table 9.10 (p 9:16), the Mental Health scale scores are presented in conjunction with the data from the Medical Outcomes Study about mental health resource use. For scores within the top third, middle third and bottom third of the scale, the percent with any use of mental health services, the percent who saw a mental health specialist, and total mental health expenditures per year are presented.

Norm-Based. In Chapter 10 of the Manual and Interpretation Guide, the normative data for the US general population by age group, by gender, and by dichotomous limitations indicators are presented. These norms were estimated from responses to the National Survey of Functional Health Status (NSFHS), a 1990 cross-sectional survey that included a total base sample of 2,909 households. Additionally, norms have been estimated for patients within the Medical Outcomes Study panel. Comparisons with the norm data can be useful in estimating the extent of the difference between a study group and the general population in a point-of-time assessment and in assessing a group for the first time in a longitudinal study.

The following illustrations reflect two common uses of health status information in clinical settings: a designed research study driven by a hypothesis and an outcomes management approach relying on indicators of patient status. In each case, the approach to interpretation most closely follows Dr. Ware’s definition of "criterion-based" because the health status scores are linked to external data for their interpretation. Although the published US norm data are not specifically referred to in these illustrations, the clinicians involved are very familiar with the US population ranges and routinely consider the relationship between the norms and the ranges for their own populations.

Illustration #1
Provided by Deborah Miller, PhD, Cleveland Clinic Foundation

Dr. Miller uses the SF-36 Health Survey for a variety of purposes within her research at the Mellen Center for Multiple Sclerosis Treatment and Research, a center for excellence at the Cleveland Clinic Foundation. Two of her studies represent distinct uses for the health status survey information: one uses the SF-36 as an outcome measure of quality of life to evaluate use of a new treatment, the other uses the health status score as a factor to predict health resource utilization.

When the FDA released Betaseron, a form of beta-Interferon given every other day by self-injection, it was the first disease-modifying drug available for multiple sclerosis (MS). Depression had been noted to be an infrequent but significant side effect, but there was no other information on psychological outcomes connected with the drug’s use. Dr. Miller and her colleagues designed an 18-month prospective study for all patients initiating treatment with Betaseron to assess the drug’s impact on emotional status and quality of life and the relationship of these variables to side effects.

The research team hypothesized that patients who were more depressed prior to the onset of Betaseron treatment would experience more side effects and lower quality of life than other patients. To test this hypothesis, they used the Beck Depression Inventory (BDI) to measure depression and chose the SF-36 Health Survey as a measure for quality of life. Questionnaires were administered at baseline, after one month of treatment, and after six months of treatment. Side effects were measured with a checklist developed for the study after the first and second injections, and after the one-month and six-month injections.

The team identified three SF-36 Health Survey Scales to represent their quality of life measure: general health, mental health and social functioning. BDI scores correlated with all three scales at each point in time, thus confirming the hypothesis that more depressed individuals were likely to report lower quality of life in the areas of general health, mental health and social functioning.

The study included 58 MS patients, 15 of whom had baseline BDI scores within the depressed range. The researchers found that the quality of life scores for the population remained stable over the course of treatment and that depression and quality of life did not worsen for those with pre-existing depression.

In another study, Dr. Miller and a colleague included the SF-36 Health Survey as one of several potential predictors of resource utilization by patients initiating care at an outpatient MS clinic. The prospective cohort study was conducted over a period of four and a half months and enrolled 189 consecutive patients with a definite or probable diagnosis of MS presenting at the clinic. Patients completed the SF-36 Health Survey and the treating neurologist completed sociodemographic information and a history and current status of the disease (clinical information). The researchers combined the health status, sociodemographic, and clinical information with direct and indirect cost information from the hospital’s financial information system for the two-month period following the evaluation.

Complete data were available on 127 (67.1%) patients. Although no significant differences were observed in univariate comparisons of costs in patients with differing levels of clinical or sociodemographic variables, significant differences were found for two of the SF-36 Health Survey scales. In the univariate comparisons, patients with both General Health and Role Emotional scale scores above and below their respective means were observed to have significant differences in both direct and indirect cost components after controlling for multiple comparisons (p<0.003). In multivariable models, lower Role Emotional scores remained significantly associated with higher costs of care (p<0.01, R2 for each model=0.05).

Dr. Miller and her colleague concluded that it was feasible to anticipate health resource needs in patients presenting for care of chronic diseases, using readily available clinical, sociodemographic, and quality of life indices.

Illustration #2
Provided by Marcia Stevic, PhD, RN, Health Services Advisory Group

Dr. Stevic has had considerable experience interpreting health status scores in clinical settings. Her interpretation approach focuses on looking for relationships between the SF-36 scales and relevant clinical measures as indicators to guide patient education, care planning and intervention. "One of the things that I have found over time is that, for the various diseases or conditions—from diabetes to total hip replacements, two of the eight scales are usually highly correlated with the clinical findings pertinent to those conditions," Dr. Stevic said. For diabetes, she has found that the most useful scales are Vitality and General Health. A very low score on Vitality is often linked with very poor glycemic control. "If a person has a Vitality score of 50 and glycemic control of 10, that person is going to need serious health care intervention soon."

Since the SF-36 scores are not designed for use on an individual basis, Dr. Stevic follows the approach used by other clinicians adapting the instruments for individual use: she uses the results to structure a conversation with the patient. For diabetic patients, Dr. Stevic uses the Vitality and General Health scales to form questions to assess patients’ expectations of their future health status. For example, at baseline Dr. Stevic will ask: "Where do you expect your energy level to be at six months? Over time do you expect your health to get better, worse?" She measures health status again at six months and compares the current health status to the patient’s expectations.

Dr. Stevic uses the current vs. expected results as a measure of success of patient education or to drive interventions based on patient intervention for the future. She also uses the Vitality scores to discuss issues of compliance with diabetic patients. "Discussing low scores on Vitality with a patient can also help identify control problems that people might not have expected if they take their insulin once a day."

Dr. Stevic encourages clinicians to examine the relationship between individual scale performance and a few key clinical indicators to find a useful interpretation strategy drawn from scales meaningful for each patient group.

1. Ware JE, Snow KK, Kosinski M, Gandek BG. SF-36 Heath Survey Manual and Interpretation Guide. Boston, MA: The Health Institute, New England Medical Center, 1993.

Thank you to January issue contributors
In addition to thanking Audrey Bernam and the other Editorial Advisors, the Trust also extends its thanks to the following people for their contributions to the coverage of Depression: Catherine Borbas, Mark Kosinski, Paul Nutting, Cori Ofstead, David Radosevich, Katherine Rost and Harry Wetzler.