While most agree that outcomes research is essential to understanding the populations served and determining the effectiveness of new interventions and programs, according to Alvin Tarlov, MD, President of the Medical Outcomes Trust, when it comes to outcomes research , nationally, we are much further along in concept adoption than actual implementation. In an effort to encourage organizations nationally to progress from concept adoption to implementation , this article will describe the key ingredients of a successful integrated outcomes research program.

An integrated outcomes research program for the purpose of this article is defined as one that is woven into the fabric of the organization, that involves participants other than primary researchers, that conducts studies to answer important questions for the organization and that uses study results to drive clinical and administrative decision- making. The key ingredients to a successful integrated outcomes research program are 1) organizational alignment 2) an incremental plan for building infrastructure and 3) dedicated resources.

Organizational Alignment

Organizational alignment in simple terms means that: 1) the mission of the organization is clearly stated 2) a strategic plan is developed to realize the mission 3) everyone in the organization is working toward a common end 4) there are processes in place to evaluate outcomes as they relate to the mission, plan and goals and 5) that information from the outcome evaluations are fed back into the process to continually refine the strategic plan (see figure 1). The mission statements of health care delivery systems typically focus on improving population health. The mission statement of Hartford Hospital for example is "to promote, restore and maintain the health of the population we serve." From an outcomes research perspective there are patient populations, programs and services that require study in order to determine if the organization is achieving its mission.

Strategic plans are usually developed at both the organization level and at the individual service level. Strategic plans typically include the values of the organization and the core activities that will be executed to achieve the mission of the organization. Senior administrators that place a high value on quality, research and education are more likely to support the development of outcomes research programs than those who do not. If values are more narrowly focused on something like increasing market share, than an individual or group of individuals advocating for development of an outcomes research program will need to be prepared to demonstrate how outcomes research can support such values.

Strategic planning needs to occur at both the macro and micro levels. Individual service level strategic plans should link to the overall strategic plan and the goals and objectives of leadership and individual employees should link to the strategic plan. Programs and services need to include outcome evaluations as part of their plans so that the results of the outcome evaluations can be used to continually refine the strategic plan. Performance management systems (especially those that tie compensation to performance) that require employees at all levels of the organization to meet goals derived based on the values and activities outlined in the strategic plan can result in accomplishing much more without requiring a tremendous expansion of resources. In summary, alignment is all about directing energies and resources away from less fruitful activities and toward programs needed for the work of today, such as outcomes research.

An Incremental Plan for Building Infrastructure

Four years ago, the following four areas of activity were outlined at Hartford Hospital to build a solid foundation for outcomes research: 1) eliciting support of experienced researchers 2) refinement of performance measurement systems 3) education of, and exploration of research topics with, clinical team leaders and 4) advancing information systems so that they readily capture, store and provide access to research ready data (see Figure 2).

Experienced investigators conducting clinical studies are good individuals to start with as they typically: already value research, understand scientific principles and desire to remain at the cutting edge in their respective fields of study. This group of individuals could easily expand on the work they were doing and consider including measures such as health related quality of life and cost analysis to their studies. Resistance from such a group could however, arise if they felt their present work was in some way being criticized, if they are of the mindset that any study finding outside of a randomized double-blind experiment is not worth considering or if resources previously allocated for their work were being diverted to outcomes research activities. The best approach is to identify a few early adopters and demonstrate some small successes.

The second area of activity that can advance outcomes research is refinement of performance indictors for an organization. At Hartford Hospital this was accomplished in collaboration with the department of Quality Management. Indicators such as length of stay, readmission rates, patient satisfaction, clinical outcomes (captured as part of clinical pathway variance analyses) and health status need to be captured consistently and accurately if they are to be incorporated into outcome studies. Much time was spent on clearly defining each indicator and defining efficient processes for obtaining and reporting the data. Outcome Reports were created for high volume patient populations to include the indicators described above. These reports were helpful in preparing clinicians and managers to look at data and consider a variety of variables in assessing their services.

Preparing clinical team leaders to identify issues and populations for study was the third area of activity. Introductory level education regarding outcomes research concepts was provided to clinicians and mangers. Data from the Integrated Outcome Reports were used as a platform to discuss their high risk, high volume populations. Particularly at first, it was important that outcomes research was presented as an extension of the organization's quality plan rather than a stand alone activity or an activity that was done by the "scientists across the street". From these early discussions, areas for study were identified and clinicians were assigned to work with outcomes research staff. As part of the annual goal setting process that supports the strategic plan, clinical leadership was expected to identify a high risk, high volume population to study and for whom to subsequently improve care delivery. Meetings were scheduled with the clinical leadership (i.e., Nursing Directors and Medical Chiefs of service) annually to build on the introductory education and discuss new goals.

The fourth area of activity involved the advancement of existing information systems to readily capture data for outcome studies. This continues to be an ongoing challenge at Hartford Hospital despite other program successes. Administrative data bases were historically created for billing purposes not for research purposes. Therefore, the information derived from these databases have limitations. In addition, electronic medical records are not in widespread use because of the logistical issues they pose given the lack of: networked terminals at each point of care, sophisticated and affordable remote access devices, a full staff of clinicians that are computer literate and patients who are comfortable with providers using computers during routine care. Finally, the clinical information that is available electronically may be stored in systems that do not allow retrospective retrieval of aggregate patient data thus , severely limiting the utility of the data for study purposes. While the trials and tribulations of information systems today fail to provide all of the desired research solutions, small advances can be made such as creating standard extract tools and automatic feeds from administrative databases to study databases so that the labor intensity of manual data entry can be reduced at least for standard demographic and utilization information.

The activities described above were begun simultaneously at Hartford Hospital and varied in time intensity over the four years. For instance introductory education and refinement of the performance indicator system was the main focus of year one while development and implementation of study protocols and advancement of the information systems were the focus of the latter years.

Dedicated Resources

The program commenced four years ago with a Director for Outcomes Research Management formally reporting to the Vice President for Research and accountable to the Chief Operating Officer. Both relationships were extremely important to grow a scientifically sound and organizationally integrated program. The first year clerical and statistician support was shared with the overall research program. Information service needs were provided on an ad hoc basis by other departments. Study coordinators and research assistants were hired gradually as grant awards were received, mainly in the early stages of development from the Hartford Hospital internal small grant competitions.

In addition to the director position, the program presently consists of a part time medical consultant, a full time administrative associate, a full time statistician, a part time senior scientist, a full time data systems analyst and programmer and a part time community health assessment consultant. Four full-time and one part time project coordinators are funded by specific research grants. Outcome studies are up and running in virtually all major clinical service areas (e.g., Asthma, Diabetes, Congestive Heart Failure, Stroke, Total Joint Replacements, Women’s Health, Mental Health, Trauma, Transplant, Breast Cancer and Prostate Cancer). For these projects, support was provided by outcomes research staff in study design, grant writing, hiring of study personnel, data management, statistical analysis and interpretation and presentation of study findings. The Director also has a contract with the City of Hartford Health Department to co-manage the city’s community health assessment function and a partnership agreement with the University of Connecticut MPH Program for student internships. The program at this point, has evolved to the establishment of an Institute for Outcomes Research and Evaluation and has been identified as a priority in the 1998-2000 strategic plan.

Implementation of an outcomes research program at minimum requires a full time leadership person who is credible to clinicians and administration. In addition, access to at least part-time clerical, statistician and data systems specialists are essential.

In summary, outcomes research contributes not only to the scientific literature but also creates a culture that values excellence in clinical practice and improving health from a patient’s perspective. Outcomes research by design brings structure and evidence based practices to clinical programs. Results from the studies provide the basis for continuous quality improvement efforts and may also be used for contracting and marketing purposes. Finally, commitment to an outcomes research program on the part of senior administration, reflected in alignment with strategic objectives and dedicated resources, demonstrates a commitment to supporting clinical staff and referring physicans in their efforts to provide high quality, cost effective care and improve population based health.