Medical Outcomes Trust - Member Service

Programs Underway at the Joint Commission
on the Accreditation of Healthcare Organizations

Ten years ago, the Joint Commission on the Accreditation of Healthcare Organizations (JCAHO) introduced its Agenda For Change, an initiative to revise the standards in the traditional accreditation process. This three-part program included revisions of the actual standards themselves, changes to the compliance protocols, and the introduction of a system by which health care organizations would be measured on the basis of actual performance. The latter component of the program was predicated on the concept that a single system of performance measurement was a viable strategy and that health care organizations would be prepared to handle this type of measurement. As Jerod M. Loeb, PhD, Director of Research and Evaluation and Chief Scientific Officer at the JCAHO explained for this article, this concept has taken ten years to become a reality.

In the ten years since the Indicator Measurement System (IMSystem) – JCAHO’s voluntary performance measurement system for hospitals – was developed, the field of systematic performance-based measurement has experienced an explosion of technology development and the menu of system options has expanded significantly. This evolution, along with the concerns raised by accredited institutions for greater flexibility, a choice of systems and ways to keep costs under control, has led to a policy shift by the JCAHO that will permit increased choice and allow greater potential for relevance to each organization’s identified needs in performance and quality improvement.

Performance Measurement
To support this policy shift, two initiatives are underway at the JCAHO. First, on August 19, 1996 a letter was sent to the 5,000 hospitals and 1,700 long term care institutions accredited by the JCAHO listing those performance measurement systems that are willing to “open the black box” or release their proprietary information for review by the JCAHO’s Council on Performance Measurement (COPM). The 19-member COPM will evaluate these systems (including the JCAHO’s IMSystem) using attributes developed by the Council for this purpose (see sidebar) and develop formal recommendations for the JCAHO Board early this month. The result will be a list of JCAHO approved systems of performance measurement.

The list of approved systems will then be released to health care organizations. According to Dr. Loeb, it is expected that mandates for accreditation will begin in December 1997. All hospitals and long term care institutions will be required to select an approved performance measurement system and select a small group of indicators (not less than two) that represent about 20% of the institution’s patient population. In the years following, these requirements will be applied to other types of organizations that are accredited by the JCAHO. It is also expected that the indicator requirements will expand over time – moving from selecting two indicators to selecting four or five indicators that represent 40% of the patient population, and so forth. Measures of functional status and patient satisfaction will be required in the future. Data submission will not be required until the first quarter of 1999. Loeb stresses that the underlying objectives are to take incremental steps, maintain flexibility, and to continue to acknowledge the scientific underpinnings of measurement. “It is important to note that we are looking at complimentarity between standards compliance and use of outcomes – moving from making accredidation decisions based solely on standards compliance
toward a more complex process that integrates outcomes and standards,” states Loeb.

National Library of Healthcare Indicators
In addition, the JCAHO is developing a National Library of Healthcare Indicators and its first compendium will be released later this year. According to Loeb, the JCAHO defines a health care indicator as “a quantitative measure of performance that is used to evaluate and improve outcomes as well as the performance of various functions or processes of care.” In response to a national request for indicators for health plans and health care networks, over 900 indicators were reviewed by JCAHO content experts for their appropriateness and utility for quality improvement. From this process, 225 indicators were selected and are described in the catalog for health plans and networks. Additions to the library are under development for areas such as ambulatory care, behavioral health, and home health care.

The intention of this initiative is to simplify the process of selection for the health care organization and one day to incorporate all indicators that are included in the performance measurement systems approved by the JCAHO. As Loeb points out, “a single performance measurement system may include over 2,000 indicators and the current catalog contains just over 200 indicators.” He expects that in five years concordance between the two initiatives will exist.
The COPM’s role will also expand to include the review of individual indicators. This committee will be an ongoing component of the JCAHO strategy to advance the appropriate use of performance measurement systems.
Questions about these initiatives may be directed to Dr. Loeb at (630) 792-5971 or e-mail jloeb@jcaho.org.

Attributes for Evaluation of
Performance Measurement Systems

» Performance Measures and Data Elements
The system contains performance measures focusing on processes and/or outcomes related to patient care and/or organizational performance (1996); the system collects and stores (or has access to) data at the level of individuals (future); and procedures and/or algorithms are documented (future).

» Database
The system has a documented objective that describes its intent or purpose, and one or more automated databases are present that: are operational; are able to supply ongoing, longitudinal data; can link specific health care organizations with their respective data; and can support intra-organizational and intra-system comparisons of performance measure data (1996).

» Data Quality
The measurement system regularly audits the accuracy and completeness of performance measurement data received from participating health care organizations (1996); methods have been implemented for incorporating the results of data quality monitoring into continuous improvements in data quality (future); mechanisms are available for the education/training of relevant staff at participating organizations (1996); and data quality is monitored continuously at the individual participant level (future).

» Risk Adjustment/Stratification for Patient Factors
Rationale is provided to account for those measures that are risk adjusted/stratified and for those measures that are not risk adjusted/stratified (1996); and measures with denominators composed of individual level data are risk adjusted/stratified with a unique model tailored to the specific measures (future).

» Performance Measure-Related Feedback
Actively participating health care organizations are provided access to comparative (inter-organization) performance measure-related feedback by the measurement system at least once per year (1996); and education and data interpretation resources are made available by the measurement system to participating health care organizations (1996).

» Relevance for Accreditation
The measurement system demonstrates face validity of its measures, including expert review and evidence of clinical and/or organizational relevance (1996); and measures are reviewed by the system to ensure accurate and consistent identification of events they were designed to identify across multiple settings (1996). The system is large enough to provide meaningful intra-system comparative analysis (future); and the system assists the Joint Commission with conducting evaluative studies designed to demonstrate utility and relevance to accreditation (future).

The following article is excerpted from the LHS manual, published by the Department of Health Care of the Elderly, University of Nottingham, Nottingham, UK. The principal developers of the London Handicap Scale are Doctors Rowan H. Harwood and Shah Ebrahim.

Intention and Scope
The London Handicap Scale is an instrument to measure an aspect of health status in groups of patients with chronic, multiple, or progressive diseases.

The term “handicap” is used in a technical sense, slightly different from its everyday meaning. Handicap is defined as the disadvantage experienced by an individual resulting from ill-health, compared with what is normal for someone of the same age, sex and background. It takes into account the physical and psychological effects of diseases or combinations of diseases, and relevant non-health factors such as the physical and social environment. It also takes into account the effects of health service interventions acting at various levels: curing diseases, relieving or palliating symptoms, reducing disabilities and modifying the environment.

The instrument generates a profile of handicaps on six different dimensions, and an overall handicap severity score. The dimensions are mobility, physical independence, occupation, social integration, orientation and economic self-sufficiency. The definitions for each dimension are generally self-explanatory, but need some qualification. Mobility is the ability to get from one place to another, using whatever help, aids or means of transport that are normally available. The disadvantage associated with being reliant for help on aids or another person for help is covered under physical independence handicap. Occupation is doing what one wants or needs to do with one’s time, including work, household and leisure activities. Social integration describes the ability to maintain relationships with other people in the face of ill-health. Orientation is the ability to perceive and understand one’s immediate environment, including thinking, perception and communication. Economic self-sufficiency includes both the effects of ill-health on the ability to earn a living and the use of resources to overcome disadvantages associated with ill-health.

Each dimension has six levels, arranged in order of increasing disadvantage. A respondent chooses which level most closely fits his or her situation. Together this information forms a descriptive profile of the individual. This profile can be converted into an overall severity score using a table of scale weights. The score gives an estimate of the relative desirability or “utility” of the particular combination of disadvantages. The weights were calculated from studies of the opinions of representative populations. Strictly they should only be applied when comparing groups of patients, and scores for any one individual will only be an approximation to that individual’s handicap. In most applications this distinction makes little difference, since the intention will be to describe the overall experience of groups of patients (such as users of a service, arms of a comparative trial, residents of a particular area or patients with a particular diagnosis). It must be emphasized, however, that the scale is not primarily intended for assessing or monitoring individual patients, although the descriptive profile may be found useful.

The London Handicap Scale was designed with the needs of the randomized controlled trial in mind, and for comparisons between different interventions for health policy purposes. However, it will also be of use in cross-sectional surveys, and longitudinal or cohort studies, either as an explanatory variable or as an outcome variable.

The scale is suitable for adults of all ages. The particular issues of chronic disease, multiple pathologies, and the importance of environmental and social factors arise most frequently with elderly people, for whom the scale should be particularly appropriate. It was not designed for use with children.

Application
The instrument comprises two parts: a classification questionnaire and a table of scale weights. The questionnaire was designed for self-completion. The intended target group, however, will contain an important minority who are cognitively impaired, have poor vision or communication problems, making self-completion difficult. In these cases the best available source of information should be sought. This means the patient or subject him- or herself should provide the information if this is possible, since the patient’s perceptions may differ from those of other people. If self-completion is not possible, a close relative or carer may help the subject to complete the questionnaire. If the subject is not able to contribute to the process at all, a carer who has close and regular contact and who knows the subject well is the most likely to provide accurate information. If such a carer is not available for some reason, a closely involved professional will provide information that is better than nothing for most purposes. Where many subjects are likely to require assistance, this represents a potential source of bias, and should be quantified using additional questions such as: “Was the questionnaire completed on behalf of the study participant?” and “Did anyone help you complete the questionnaire? If so, who?”.

The scale has been used in some interview settings. Giving the participant a visual cue card (a printed list of the different levels) for each dimension allows the process of self-completion to be replicated most closely.

Once the classification questionnaire is completed, each individual can be described by six numbers (each representing a level on a dimension). In this form, data can be entered onto a spreadsheet or other computer data entry software package. The appropriate score for each level of each dimension is then applied, and entered into the formula:

where Um, Uoc, Upi, Usi, Uor and Uess are the scores of the appropriate level of each dimension given in the scale weight table and 50.5 is a constant. This gives a score between 0 and 100, with 100 representing no disadvantage and 0 the maximum possible disadvantage. (Earlier versions of the scale used a 0-1 range to be consistent with the conventional representation of “utilities”. This has simply been multiplied by 100 for ease of presentation).

The Seattle Angina Questionnaire (SAQ) is a 19-item instrument developed to quantify five clinically relevant domains of coronary artery disease. Nine items define the limitations in a patient’s physical capacity due to symptoms of angina pectoris. Two items assess the frequency of anginal symptoms and an additional item defines whether a change in the precipitants of angina has occurred over the past month. Four items describe patients’ satisfaction with the treatment of their coronary artery disease and three items explore how much patients’ coronary artery disease has limited their quality of life.

Application
The SAQ has been used as an outcome measure in several ongoing clinical investigations. The Quality of Life in Angina Research Trial is a randomized study of once-daily vs. multi-dosing anti-anginal medications that has successfully employed serial administration of the SAQ to define the quality of life differences between these two strategies for treating patients with chronic stable angina. Similarly, the OPUS trial is comparing optimal angioplasty with intracoronary stenting to define the quality of life differences between these two strategies of percutaneous coronary revascularization.

A quality improvement initiative based in Washington state is using the SAQ to gain improved insight into the functional status outcomes of coronary bypass surgery. The CABG Outcomes Assessment Team is a physician-led effort in which a pilot project, representing almost all cardiothoracic surgery groups in Washington (n=12) have prospectively enrolled up to 130 patients each into a study that will quantify the baseline, 6 month and 1 year functional status of their patients. In addition to describing the process of post surgical recovery, the SAQ will be used to define differences in the patient selection and outcomes of each group in the state. This information will then be presented blindly to the entire surgical community and allow a process of reflection and discussion to better define how to select and manage patients with advanced coronary artery disease.

Another such project involves a “natural history” experiment in a Veterans Affairs general internal medicine clinic. In this project a sample of 100 patients with stable coronary artery disease were enrolled in an angina clinic that was run by a nurse practitioner with telephone support from a cardiologist. The SAQ was able to document substantial improvements in symptom relief and quality of life in the patients enrolled in the angina clinic compared with the hundreds of patients in the general internal medicine clinic who did not receive additional management assistance from the nurse practitioner.

Since self-administered questionnaires can accurately describe important clinical information, they can be scored and tracked over time to plot the course of a patients’ condition. In coronary artery disease, the progression of symptoms can often be insidious since patients modify their activities to minimize the frequency and severity of their symptoms. In general internal medicine clinics, when physicians need to attend to a wide variety of comorbid conditions, both the doctor and the patient may fail to recognize that the symptoms of coronary artery disease have worsened. A description of the clinical status of patients’ coronary artery disease, as provided by the SAQ, may prove to be a valuable aid to generalist physicians who manage patients with coronary disease. An eight-center Veterans Affairs trial has recently begun to test this hypothesis in 16 general internal medicine units.

References for the SAQ
Spertus JA, Winder JA, Dewhurst TA, et al. Development and evaluation of the Seattle Angina Questionnaire: a new functional status measure for coronary artery disease.
J Am Coll Cardiol. 1995;25:333-341
Spertus JA, Winder JA, Dewhurst TA, Deyo RA, Fihn SD. Monitoring the quality of life in patients with coronary artery disease. Am J Cardiol. 1994;74:1240-1244