Metrics can pile up until they become overwhelming to clinicians and administrators alike, with their usefulness lost in the process. The solution may be to strategically reduce the number of routinely reportable metrics to only those most appropriate and valuable.
Indiana University Health in Indianapolis has found that reducing the number of reportable metrics not only alleviates a lot of workload, but also improves the quality of care. Preventable harm incidents and inpatient infection rates fell significantly after IU Health reduced the number of quality metrics tracked by front-line staff and physicians from 199 to just 10.
Total harm events throughout the 15-hospital system fell from 120 per month to fewer than 80 over a two-year period. Infections were reduced by half.
Cutting the list down so sharply allowed personnel to focus on the measures that matter the most, says Jonathan Gottlieb, MD, executive vice president and chief medical officer of IU Health. Clinicians had felt overburdened with collecting data and reporting metrics, he says. Gottlieb realized when he joined the IU Health system three years ago that it had become overburdened with metrics.
“There was a formal expectation of combining hundreds of metrics into an index and then tracking that index. It became obvious to us that though we do need to follow dozens and dozens of metrics, we also need to focus on the most important ones,” he says. “Otherwise, you just get swamped.”
Rather than the administration dictating where the focus should be, IU Health enlisted front-line staff to identify the quality metrics they thought would be most important. They were asked to try to limit their choices to the top 10 or 12 quality metrics that determined quality of care and to disregard how they affected reimbursement under any pay-for-performance reimbursement program. The goal was to ensure quality of care rather than to improve revenue, Gottlieb explains.
Their choices were presented to the IU Health quality council, made up of about 70 members from throughout the system.
“The ones they picked were very consistent, thank goodness, with most of the pay-for-performance metrics,” Gottlieb says. “With minimal editing, the council came up with a list of the top 10 metrics they thought would be important to ensuring quality of care, no matter what.”
Teams Tackle Individual Metrics
IU Health asked physician and nursing leaders throughout the system to each adopt one of the metrics and put together a multidisciplinary team that could spend a few months studying the evidence and their own experience to identify gaps in how IU Health was performing on the metric and where they thought it should be.
The teams studied specific issues such as central line infections, looking for potential gaps in everything from determining the need for a central line and the equipment used to procedures for inserting and maintaining the catheter. The central line infection team ended up training almost 200 physicians with a standardized four-hour course on how to insert the catheter using evidence-based techniques. Incoming residents and fellows must undergo the same training.
Several hundred nurses underwent training on how to maintain the catheter and IU Health standardized the type of catheter used throughout the system. The electronic medical record also was changed to include a daily prompt for physicians either to enter a valid reason for keeping the catheter or to remove it.
Those team assessments were provided to the quality council, which presented C-suite leaders with a plan for reducing the number of metrics routinely reported and improving performance on the most important ones.
“This became a rallying point for the leadership. The hospital presidents, chief financial officers, and operating officers all jumped on board and said this was a helpful plan and they could see what metrics we should collectively focus on,” Gottlieb says. “They were happy to see that there was a lot of consistency with the pay-for-performance metrics, but mostly this seemed to be the right thing to do for our patients.”
These are the 10 systemwide metrics currently tracked by IU Health:
- Clostridium difficile infection;
- central line-associated bloodstream infection;
- catheter-associated urinary tract infection;
- medication error with harm;
- falls with injury;
- hospital-acquired pressure injury;
- serious reportable events;
- potentially preventable mortality;
- surgical site infection — colon surgery;
- surgical site infection — hysterectomy.
The two surgical site infection metrics will be dropped in 2018 because their numbers have been reduced so much that there is little room for additional improvement. They will be replaced with two other metrics, which have yet to be chosen.
The common theme in the top 10 list is preventable harm, Gottlieb notes. Five are hospital-acquired infections, and the rest are similarly preventable.
Other Metrics Still Used
IU Health did not simply dispose of 189 metrics it had been tracking. The system still tracks 199 metrics, but the difference now is that not every clinician or care team is involved with all of them. The top 10 metrics are monitored and reported by all care teams, but the other metrics are still used when they are relevant to a particular patient, care team, unit, or hospital.
“Our hospitals that do cardiac surgery participate in a lot of benchmarking around cardiology, but everyone else doesn’t have those metrics because they are not relevant and would take focus from what does matter to their patients,” Gottlieb says.
“But when someone has cardiac surgery, we also don’t want them to get a central line infection. So that’s the kind of thing that is in the top 10 because it can apply to such a wide range of patients and settings.”
IU Health reports the performance on each of the 10 metrics for the system as a whole and for every hospital with a “harm dashboard.”
The new system addressed the metrics creep that can affect any hospital or health system, Gottlieb says. No one starts out thinking staff should be responsible for 199 metrics, but hospital leadership keep adding a few here and there over time because each one is valid in its own right. After a while, clinicians and quality professionals are bogged down with a huge number of metrics and no one wants to take responsibility for saying which ones should be scrapped.
“It was tough for people to figure out what to do with their limited time and resources,” Gottlieb says. “We had put all these metrics out there and made them responsible for them, but everything seemed to have the same level of importance. It was a required metric and you were going to be held accountable for it, but there were so many that people got frustrated and that undermined the effort to get them invested in improvement.”
There were many stakeholders involved in all of those metrics, and no one wanted “their” metric to be abandoned or reduced in prominence. Resistance from those stakeholders was expected, and IU Health overcame it by appealing to their sense of professionalism, Gottlieb says.
“We acknowledged that we all come to work to help patients and not to harm them, yet we know that preventable harm does occur in our health system. We asked them to support us in focusing on preventing that harm to patients rather than narrowly looking at the so-called quality metrics they found important,” he says. “Everyone agreed protecting patients surpassed any other concerns, so it tapped into their compassion and professionalism. It was a good place to start.”
It also was important to emphasize that IU Health was not abandoning the metrics that did not make the top 10. The health system sent the message that the change was all about prioritizing metrics rather than eliminating some.
Dashboard Reflects Patients
IU Health also changed how the metrics are figured into the harm dashboard that illustrates performance on the prioritized metrics. Previously, the overwhelming amount of data had to be massaged so much that the resulting index had little meaning to the people it was supposed to motivate for improvement, Gottlieb says.
“We decided to make the metric the number of people who were impacted,” he says. “Suddenly, they could see this as the number of patients — actual people affected by this preventable harm, rather than some arbitrary index derived from a lot of meaningless numbers.”
The health system initially set a target of reducing preventable harm by 13%, then pushed that to 16% for the next two years.
From 2015 to 2016, IU Health saw a reduction of 26%, which meant 200 fewer patients experienced a preventable injury.
IU Health tied the incentive compensation of every leader at the health system to local and system aggregate performance of the 10 metrics, Gottlieb explains.
“Suddenly, the CFO of the system was a little more interested in how we were doing in preventing urinary tract infections, not just for himself but for everyone who worked for him,” he says.
“Every month, people couldn’t wait to get that scorecard in their email to see how we were doing. It sent the message that we were all on the same page. We weren’t being told to do one thing, but being rewarded for something else.”
- Jonathan Gottlieb, MD, Executive Vice President and Chief Medical Officer, Indiana University Health, Indianapolis. Email: [email protected].
