| Health System Improvement
CONQUEST 2.0: A tool to evaluate and select outcome measures |
||||||||||||||||||||
|
Editors Note: This article was written by Ann G. Lawthers, Sc.D., Lecturer, Center for Quality of Care Research and Education at the Harvard School of Public Health
A significantly enhanced form of CONQUEST, version 2.0, is scheduled for release by AHCPR in the fall of 1998. The enhanced version includes a new and much improved user interface, new measures, new conditions, as well as detailed specifications for constructing over 700 of the databases measures. This article describes CONQUEST, traces its development and shows how the database can be used by individuals interested in measuring patient health outcomes. What is CONQUEST? Put simply, CONQUEST is a database of clinical performance measures and clinical conditions combined with a user-friendly interface (see Exhibit 1). The interface permits users to find measures suited to their specific measurement purpose. Examples of CONQUEST functions include searching for measures related to a specific clinical condition (such as diabetes or asthma), a specific age group (e.g. children between the ages of 4 and 15), selected settings (e.g., inpatient or ambulatory), or type of outcome (e.g., clinical, functional or risk state). Or, a user may find detailed specifications for the construction of a measure. A database of clinical performance measures Clinical performance measures are one type of quality measure. The other two types of quality measures are access and satisfaction measures. The CONQUEST database includes only clinical performance measures. The measure sets included in CONQUEST are listed in Exhibit 2. A clinical performance measure assesses whether a health care provider delivers clinical services that are appropriate for each patients condition; provides them safely, competently, and in an appropriate time frame; and achieves desired outcomes in terms of those aspects of patient health and patient experience that can be affected by clinical services. Among the clinical performance measures in CONQUEST, 271 or 23 percent are measures of patient outcomes. Another eleven of these measures capture both a process and an outcome in single measure. For example, abnormal temperature within 24 hours of discharge is a mixed measure. The abnormal temperature is an outcome while the decision to discharge the patient is a process of care. The CONQUEST Measure Database includes information about the following aspects of a measure: · Measure Content: description of numerator and denominator, clinical rationale for the measure, age groups and care settings to which the measure applies, care needs captured by measure (e.g. primary or secondary prevention, diagnosis, treatment, management of disease). · Data Required to Construct a Measure: data sources, sampling required inclusion/exclusion criteria, time window, type of numerator criteria. The Specifications Database includes detailed information about sampling, data collection time windows, specific data elements required, as well as the algorithms for combining the data elements to create the measure. · Information to Aid Evaluation of the Measure: reliability and validity testing (description of specific procedures used included in the Specifications Database), extent of testing, extent of use, standards of quality (benchmarks) used, and allowances for patient factors (with description of the procedures used included in the Specifications Database). Of the outcome measures in CONQUEST 2.0, 64% capture functional outcomes, 32% clinical outcomes, and the remaining 4% health risk state or mixed outcomes. The various data sources for outcome measures in CONQUEST 2.0 include the medical record, administrative and patient survey data. Most of the outcome measures in the database rely on medical records as the source of data for the measure. Interestingly, almost 40% of the measures require more than one data source for their construction. Commonly an administrative source is used to find the case and the medical record to ascertain the outcome. A database of clinical conditions CONQUEST also includes data about clinical conditions (see Exhibit 3 for a list of the conditions), such as: · Form and time course of condition: usual forms or variants of the condition, as well as the time course of the variant (e.g. acute, acute recurrent, chronic stable, chronic progressive, chronic cyclic). Knowing the time course of a condition is particularly important for designing appropriate outcome measurements. · Prevalence and ages affected: prevalence of the condition by population subgroups as well as the ages usually affected by the condition. · Condition comorbidities and risk factors: Comorbidities are conditions that exist at the same time as the primary condition. Risk factors are chemical, genetic, psychological, or physiological states that predispose an individual to the development of a disease. Knowledge of co-mobidities and risk factors helps users design appropriate risk adjustment procedures for a measure. · Potentially preventative adverse outcomes of the condition: Natural sequelae of a condition that may be modified or prevented with appropriate treatment. · Burden of illness: The health care costs and utilization associated with the condition. · Services used in the prevention, detection and management of a condition: As recommended by clinical practice guidelines or scientific literature. Development of CONQUEST The failed health reform agenda of 1994 provided the impetus for the development of the first CONQUEST database. The Agency for Health Care Policy and Research (AHCPR) recognized the need for a means of summarizing and classifying the rapidly expanding number of clinical performance measures in a concise yet accessible fashion. The objectives of the first CONQUEST project were to collect and analyze examples of the range of clinical performance measures, summarize information about the properties of measures, as well as assess the feasibility of deriving measures from existing databases. The outcome of the project was a database of clinical performance measures. The second CONQUEST project, begun in 1995 and which resulted in CONQUEST 1.0, created the user interface and, more importantly, verified the accuracy of the information contained in the database. To verify the information in the Measure Database, each measure developer was contacted and interviewed at length. Data in the Condition Database derived from AHCPR guidelines was verified by conducting interviews with the chairs of the guideline panels. The most recent CONQUEST project, completed in January 1998, focused on improving the user interface and expanding the scope of measures and conditions contained in the database. CONQUEST 2.0 is scheduled to be released by AHCPR in the fall of 1998. At no time during the development of any of the CONQUEST releases, was the objective to create a comprehensive database of measures. Rather the intent has always been to provide examples of a broad range of types of measures. Measures in the database included those from well known sets such as the Joint Commissions IMSystem set and the HEDIS 3.0 set. The FACCT measures were still in development at the time version 2.0 was prepared and thus are not included in the database. Using CONQUEST to compare and evaluate measures of patient outcome Using CONQUESTs various search functions, a user interested in outcome measurement can easily compile a list of potential measures. As an example, lets assume that you are the Director of Quality Measurement for MegaHealth. MegaHealth is a managed care organization which provides primary health care services for about 200,000 patients. In addition, MegaHealth contracts with hospitals for inpatient services. As Director of Quality Measurement, you want to compare the quality of care delivered by the hospitals to your members. Specifically, you wish to compare the hospitals on the rate post-surgical complications and use the information in your contracting negotiations. You begin by conducting a User-Specified Search. The criteria you select are data type - outcome and condition - surgical or medical treatment complications. This search yields 56 measures, of which 49 relate to post-surgical complications. The next step is to evaluate whether the measure is suitable for your purpose. Which measure is best for you? Never underestimate the importance of defining your measurement purpose
Evaluation criteria are much stricter for measures which will be used for accountability. These measures must be specified in detail and have high reliability to ensure that data will be collected in a uniform and consistent way. Measures used for accountability should also have validity: content, construct and criterion. Regardless of the purpose of measurement, measures should relate to important clinical conditions, be interpretable, clinically relevant, and actionable. Although different groups may propose slightly different criteria for evaluating measures, most have common threads. As an example, Exhibit 4 shows how CONQUEST relates to the Medical Outcomes Trust Instrument Review Criteria and the recently adopted Joint Commission Attributes of Core Performance Measures. Using CONQUEST to evaluate measures In our example, the purpose of measurement is accountability as the results will be used to inform contracting decisions. Thus the criteria to evaluate the measures must consider reliability, validity, and rigor of development and specifications. For this example, you decide to use the Medical Outcomes Trust criteria (Exhibit 4). CONQUEST has several built-in reports which simplify comparative analyses. These special reports compare measures by selected attribute. The reports relevant to your example include reliability testing, validity testing, scientific validity, risk adjustment, and measure purpose. Using these reports, you decide to eliminate from consideration those measures with no reliability or validity testing, those developed by consensus only, those without risk adjustment, and those developed only for clinical decision-making. This leaves 4 measures from the Joint Commissions IMSystem set and 26 from the Complications Screening Program (CSP) set as potentially suitable for your purpose. Next you print the measure specifications reports for several measures. When you compare the measures on the data collection burden you note that the CSP measures rely only on administrative data where the Joint Commission measures require medical record data collection. You decide to proceed by telephoning the contact individual listed in the database to obtain further information about the CSP measures. Summary CONQUEST 2.0, scheduled for release this fall, provides significant improvements over previous versions of the database. The user interface is easier to use and the number and scope of reports has increased. New measures and conditions are included. CONQUEST provides key information about measures and conditions to individuals interested in conducting clinical performance measurement. |
||||||||||||||||||||