Measuring effectiveness of mental health care
Look at patient-specific, system-oriented measures
By Patrice Spath, ART, BA
Consultant in Health Care Quality and Resource Management
Forest Grove, OR
To effectively manage the care provided to mental health patients, case managers must be familiar with a wide array of services: emergency assessment, observation, crisis intervention, inpatient hospitalization, partial hospitalization, outpatient treatment, intensive outpatient treatment, and home health. Efficient case management intervention is crucial for patients to move swiftly yet appropriately through this range of services. To evaluate the quality and effectiveness of case management programs in mental health services, health care professionals must determine whether case management interventions accomplish their goals.
Unlike most medical cases, the mental health case manager must deal with a challenging set of client variables, including manipulative personality traits, substance abuse, maladaptive family dynamics, secondary gain, etc. In the face of all these factors, the case manager must continually ask how much of the care can safely be provided in a less acute setting, and help the treatment team determine exactly what needs to be done in the inpatient setting.
To complicate the picture, the case manager must consider "soft" data when making placement decisions. Unlike other areas of health care, mental health caregivers lack the objective data of a blood pressure, temperature, or X-ray to determine whether patients are likely to succeed in a less intensive environment. Therefore, it is necessary to arrange close follow-up as patients move through the continuum of care.
Measures of case management performance should be a combination of patient-specific and system-oriented measures. For example, patient-specific measures evaluate how well the case manager is meeting the patient’s needs. Performance measures can be used to judge whether:
• the services given were appropriate to the needs of a particular patient;
• the patient had access to the necessary services throughout the continuum of care;
• the patient’s caregivers were provided sufficient support;
• the case manager served as an effective bridge between institutional and community-based care systems.
System-oriented measures evaluate how well case management services are achieving aggregate program goals. Performance measures of system-oriented issues can be used to judge whether:
• a broad array of noninstitutional services is being used;
• the delivery of patient care services is efficient;
• health services delivery is coordinated throughout the continuum of care;
• individuals most at risk of inpatient admission are effectively managed to reduce inappropriate use of institutional services.
The measures chosen should relate to the goals defined for the facility’s case management program. For example, if the primary goal of case management is to reduce unnecessary use of resources, patient-specific and system-oriented measures might include:
• percent of patients meeting inpatient admission criteria (i.e., Milliman and Robertson’s Optimal Recovery Guidelines, InterQual’s ISD-AC Intensity/Severity/Discharge Screens for Acute Care, or placement criteria of the American Society of Addiction Medicine);
• percent of patients meeting continued inpatient stay criteria;
• percent of patients placed in the least intensive/restrictive environment appropriate to their therapeutic needs;
• percent of inpatient cases reviewed with medical director for medical necessity;
• percent of cases for which written criteria are used to determine necessity for level of care;
• percent of cases denied payment for treatment by the health plan;
• percent of all cases denied payment for treatment by the health plan that are appealed;
• percent of health plan payment denials that are later overturned in the appeals process.
Case managers responsible for patients in a capitated health plan have access to a broader range of utilization data. In this instance, utilization-related performance measures may include:
• cost per treatment episode by diagnosis, type of patient, care levels, etc.;
• acute inpatient days per 1,000 covered lives;
• intensive outpatient/partial hospitalization days per 1,000 covered lives;
• residential days per 1,000 covered lives;
• outpatient visits per 1,000 covered lives;
• acute inpatient average length of stay;
• intensive outpatient/partial hospitalization average length of stay;
• residential average length of stay;
• outpatient average number of sessions.
If appropriate assessment and care planning are primary goals, measures may include:
• percent of discharge planning assessments completed within "X" hours of admission (organization to define time frame);
• percent of patients meeting criteria for case management intervention who are evaluated by a case manager;
• percent of case-managed patients who are visited by case manager within "X" hours of inpatient admission (organization to define time frame);
• percent of high-risk clients who indicate they do not need or desire case management;
• percent of discharge/case management plans that were appropriate to the client’s needs and preferences;
• percent of clients and client family members who report that they were provided with case management outcome expectations;
• percent of patients meeting criteria for medical consultation who receive medical consultation;
• percent of cases in which alternative discharge plans are provided within a specified time if patient does not improve;
• percent of patients reporting that they actively participated in decisions about the treatment plan.
If patient satisfaction is a primary goal of case management, the following measures may be used:
• percent of patients reportedly satisfied with the organization’s case management services;
• percent of responses to patient complaints about case management services within a specified time period;
• percent of patients who report that case managers can be reached easily;
• percent of patients who report that case managers are sensitive to their ethnicity, language, culture, and age;
• percent of patients who report that they did not feel coerced into treatment options or services.
Patients must receive comprehensive care
To maintain quality while optimizing resource utilization, the entire treatment team must be actively involved in providing effective services. Ancillary support services such as social work and case management must be available on a daily basis to help the patient navigate the maze of care. The attending physician, nurses, and other direct caregivers must help ensure the patient receives comprehensive quality care in the most appropriate setting. In this team environment it is difficult to focus performance measurement efforts solely on the contributions of the case manager. For this reason, many facilities collect and report performance measures in the aggregate, without singling out any one discipline as responsible for the measurement results.
Facilities wishing to create case management-specific performance reports must try to select measures that represent factors over which the case manager has primary control. For example, hospital readmissions may be affected by many factors over which the case manager has no control. Thus, readmission rates don’t directly correlate with case managers’ performance.
To minimize the effect of variables beyond the case manager’s control, determine for each of the case management functions the outcomes likely to be affected if the case manager is not working effectively. The matrix on p. 165 can be used to answer this question. The major functions of case management are listed on the left side of the matrix. Brainstorm the adverse outcomes that might occur if case managers are not doing a good job in these areas. Then, count how many times these adverse outcomes occur. For example, if case managers are doing a poor job at identifying cases that require case management, a possible undesirable outcome is that high-risk patients will not be seen. The performance measure for this adverse outcome would be: "Percent of patients meeting case management triggers who are not seen by a case manager." Use the results of your brainstorming activities to select performance measures that, as much as possible, evaluate the unique contribution made by the mental health case manager.
[Editor’s note: For more information, consult the following sources:
National Leadership Council. Performance Indicators in Behavioral Healthcare: Measures of Access, Appropriateness, Quality, Outcomes, and Prevention. Tiburon, CA: Institute for Behavioral Healthcare; 1996. Telephone: (415) 851-6469.
American Managed Behavioral Healthcare Association. Performance Measures for Managed Behavioral Healthcare Program. Washington, DC; 1995. Telephone: (202) 434-4565.
Spath, Patrice. Performance Improvement in Psychiatric and Substance Abuse Treatment Services. Forest Grove, OR: Brown-Spath & Associates; 1997. Telephone: (503) 357-9185.]