Special Feature: Does This New Monitoring Gizmo Work?
Special Feature
Does This New Monitoring Gizmo Work?
By David J. Pierson, MD, FACP, FCCP
One day after an operation to remove a large mediastinal tumor, a young man is critically ill in the ICU. The surgery was prolonged and difficult, and massive blood and crystalloid infusions were required. The patient’s fluid balance is many liters positive, and he is massively edematous. His chest radiograph shows diffuse bilateral lung opacities, and he requires mechanical ventilation with 100% oxygen and positive end-expiratory pressure (PEEP) to maintain an arterial saturation in the 90s. He is on continuous infusions of norepinephrine to support his blood pressure and vecuronium to maintain therapeutic paralysis. However, although orders have been written for morphine and lorazepam as needed for sedation, he has not received any of either medication for 18 hours because the BIS Monitor® attached to the patient indicates that he is adequately sedated.
Monitoring Sedation by the Numbers
The Bispectral Index® consists of electronically manipulated electroencephalographic (EEG) data from a sensor applied to a patient’s forehead and displayed on the BIS Monitor® (Aspect Medical Systems, Natick, Mass) as a number from 0 to 100.1 A BIS score of 0 is said to correspond to a flat-line EEG, while 100 means that the patient is fully awake; from studies on normal volunteers, a BIS score of 60 indicates deep sedation.2
BIS monitoring was introduced in the mid-1990s to eliminate awareness during anesthesia and to facilitate rapid postanesthesia recovery. Its spread from the operating room and postanesthesia recovery unit into the ICU illustrates an often-repeated sequence in the history of critical care: a device or technique developed for a particular setting and tested under one set of conditions is then put into clinical use in managing patients with different problems and under different circumstances. This essay addresses this phenomenon and attempts to provide a framework by which the clinician can judge the appropriateness of using new devices and techniques, particularly those for monitoring critically ill patients, using the BIS Monitor® as a case in point.
Definition of a Critical Care Gizmo
A gizmo, according to the dictionary, is "a gadget, especially a mechanical or electrical device considered to be more complicated than necessary."3 In the context of this discussion, the definition of "gizmo" may be expanded to include a device or technique in monitoring that is new, used in a new way, or used in a different patient population or clinical setting from that originally intended.
Basically, any device or technique involving technology that has not been clinically validated in the context of its use could be thought of as a gizmo. The entire history of critical care has occurred in only a few decades, and lots of mainstays of today’s clinical practice have started out this way: continuous electrocardiographic monitoring, arterial blood pressure monitoring, and pulse oximetry come quickly to mind. However, one could also name many devices and techniques that have come and gone, having been found unnecessary, too complicated, or even harmful after being used on large numbers of patients.
Why Innovation in ICU Monitoring is Needed
Monitoring is central to critical care. It has several purposes, as summarized in Table 1.4
| Table 1: Purposes of ICU Monitoring | |
| • | Assess adequacy of vital organ function |
| • | Track course of patient’s illness |
| • | Track effects of therapeutic interventions |
| • | Determine need for specific interventions |
| • | Assess performance of life-support devices and their monitors |
| • | Assess patient discomfort and the effects of efforts to relieve patient distress |
| • | Detect complications and track their severity |
| • | Detect readiness for reduction or withdrawal of interventions |
In attempting to fulfill these purposes and to improve patient care, it is necessary to overcome a number of distinct obstacles. The latter include problems with inaccuracy and imprecision in existing devices, and interindividual variations in reading, interpreting, and transcribing the data they produce. There is the problem of artifact—that is, the signal or result is produced by some mechanism other than the intended one, as with the effect of ambient light on pulse oximeter probes.
Monitoring is also plagued by factitious data (data the monitor was intended to produce but which does not have the intended implication), as with increased airway pressures generated when a patient coughs. Monitors produce false alarms, accurate measurements signifying real physiologic changes, which are due to physiologic variation rather than to an adverse change in the patient’s condition. Some monitors are difficult to use because they are too complex for routine application, despite accurately measuring what they were intended to measure.
All of these things, plus the inherent imperfections and unreliability of mass-produced sensors and other devices, mean that there is presently a substantial gap between concept and reality in ICU monitoring. New monitoring devices or techniques could help in patient management in several ways, but also tend to introduce new problems if they do not deliver as promised (see Table 2).
| Table 2: Potential Advantages and Disadvantages of a New Monitoring Device or Technique | ||
| Advantages | ||
| • | Capability of assessing something that could not be monitored before | |
| • | Capability of assessing something that can already be monitored, but: | |
| More accurately | ||
| Less expensively | ||
| With less risk or discomfort to the patient | ||
| More meaningfully | ||
| Disadvantages | ||
| • | Added complexity, cost, or morbidity without gaining practical advantage | |
| • | Generation of measurements that are not accurate in this setting | |
| • | Generation of measurements that do not mean the same thing in this setting | |
From Idea to Clinical Practice: Drugs vs. Gizmos
The initial question about whether a given new monitoring gizmo works should really be 4 questions:
- Does it operate as designed?
- Does it measure what it is intended to measure?
- Does the measurement mean what we think it means?
- Does using the measurement make a difference that is clinically relevant?
While I doubt that anyone would argue with the appropriateness of these questions, answering them requires different amounts of effort and expense. Some types of evaluation are straightforward and not too expensive to do, while others are so onerous as to prevent some potentially worthwhile innovations from being clinically introduced. This is illustrated by comparing the processes by which different kinds of intervention are introduced to clinical practice.
Most clinicians are aware of the process by which a new drug is approved for clinical use. After being identified, chemically characterized, and manufactured in enough quantity to test, the new agent must first be shown not to be unacceptably toxic, usually in studies involving both animals and healthy human volunteers. Once approved for testing on actual patients, it must undergo a series of studies to show that it does have the effect for which it was intended. Small-scale preliminary studies are followed by large-scale (and very expensive) clinical trials to demonstrate clinical efficacy. Only after this extensive several-year (and multimillion-dollar) process is a new drug approved by the Food and Drug Administration (FDA) for clinical marketing and use in ordinary patients.
The process by which a new medical device such as a mechanical ventilator or monitor is approved for clinical use is typically quite different. An all-new kind of device for use on patients must go through rigorous testing (ie, both resource-consuming and prolonged), analogous in some ways to what happens with new drugs under the classification of "Investigational New Device" (IND). In many cases however—especially with ventilators and many monitoring devices—all that is necessary is to demonstrate what is called "substantial equivalence"—the 510(k) process.5
If the FDA can be convinced that the proposed new device is just a variation on the basic design of existing (approved) devices—that is, substantially equivalent to them—the approval process is far easier. This is because of the 1976 amendment to the Food, Drug, & Cosmetic Act,5 which seeks to ensure that new devices do not have to overcome greater regulatory hurdles compared to similar devices that were on the market prior to that time. Companies typically market new devices to the consumer as totally new and different from their competition, but the federal approval process is based on demonstration that they are nearly the same as already-approved devices. Once approved, the new device can be sold to hospitals and used on patients.
The introduction of a new procedure or way of using an existing device (eg, a new surgical procedure or the new lung-protective ventilator strategy for managing the acute respiratory distress syndrome) is different from both of the above. Typically the government is not involved at all. Someone comes up with a new way of doing things, the word spreads, and patients by the dozens, hundreds, and even thousands may be subjected to the new technique. Along the way the promulgator of the innovation and others typically report their experience at scientific meetings and publish clinical series, and if the technique is sufficiently promising or popular, large-scale clinical trials may eventually be performed.
Scientific validation, however, is not part of any federal approval or regulatory process in the case of new techniques and innovative uses of existing technology. However, it should be. This is emphasized by Rubenfeld in a discussion of appropriate study design for assessing new devices and procedures in monitoring: "Intensive care monitoring does not merit clinical use simply on the basis of its making sense; its risks, benefits, and costs, like those of other medical interventions, need to be carefully and empirically demonstrated."6
Levels of Testing for ICU Devices and Techniques
Two questions need to be answered before a new monitor is placed in clinical use. Can we do it? And, is it useful? In the approval and introduction sequence just described, wide clinical use sometimes occurs as soon as the first question is answered. A new gizmo is devised, marketed to ICU clinicians, and tried on patients. Only some time later do studies get done to show whether patient care is really made more effective or safer (or less expensive) by the new device.
In its Consensus Conference on Innovation in Mechanical Ventilatory Support, the American Respiratory Care Foundation emphasized the need to match the risks and costs of innovation with the efforts undertaken to demonstrate effectiveness and safety.7 The scheme proposed for device evaluation by that consensus conference has recently been expanded by MacIntyre8 and is as appropriate for monitoring gizmos as it is for ventilators. Table 3 lists the levels of evidence proposed by the consensus conference, adapted to innovations in ICU monitoring, with examples of the kinds of data that are needed at each level.
| Table 3: Evaluation of a New Monitoring Device or Technique | |
| Level I evaluation | |
| Appropriate setting | |
| Minor risk to patient; only minor incremental cost increase | |
| Acceptable end points of evaluation: Engineering data | |
| Measurement accuracy | |
| Sensor sensitivity or responsiveness | |
| Noise | |
| Durability | |
| Level II evaluation | |
| Appropriate settings | |
| Minor incremental risk to patient; moderate cost increase | |
| Moderate risk to patient; minor or moderate cost increase | |
| Acceptable end points of evaluation: Physiologic (intermediate end point) data | |
| Gas exchange; ventilation, | |
| Airway pressures, | |
| Respiratory system mechanics, | |
| Respiratory muscle function, | |
| Air trapping and auto-PEEP | |
| Level III evaluation | |
| Appropriate settings | |
| Minor or moderate risk to patient; major cost increase | |
| Major incremental risk to patient (regardless of cost factors) | |
| Acceptable end points of evaluation: clinical outcomes data | |
| Survival, | |
| Days on ventilator; days alive off ventilator | |
| Length of ICU stay; hospital length of stay | |
| Incidence of important complications | |
| Adapted from: references (7) and (8). | |
Does This Particular Monitoring Gizmo Work?
We know from the example of end-tidal CO2 monitoring and transcutaneous arterial blood gas monitoring that systems validated in healthy people under carefully controlled circumstances do not always work out as well in the uncontrolled environment of the ICU and in unstable, critically ill patients. This brings us back to BIS monitoring. Several aspects of the patient scenario with which I began this essay are different from the setting for which the BIS Monitor® was originally introduced. What is the meaning of the processed EEG signal and its derived index in critically ill patients with different underlying diseases? How do hemodynamic instability, severe hypoxemia, PEEP, and multiple organ failure affect the measurement and its interpretation? What are the local effects of vasoconstrictor drugs or soft-tissue edema at the sensor site? How well do the sensor and other aspects of the system stand up to frequent patient turning and other manipulation in the sometimes frenetic critical care environment?
At its web site, the manufacturer promotes use of the BIS Monitor® in the ICU, stating that "BIS provides an objective tool for assessing level of sedation in critically ill patients. "[It] may be especially useful for assessing sedation in patients with neuromuscular blockade [and] mechanical ventilation. . ."9 To what extent has BIS monitoring been studied in the ICU setting, and what do the available data say about whether we should adopt this new gizmo in managing our patients?
A June 4, 2001, MEDLINE search on "Bispectral index" yielded 171 citations, but only 2 of them were full papers reporting clinical trials in ICU patients.10,11 Two additional studies were presented in poster form within the last 6 months, at the annual conventions of the Society of Critical Care Medicine12 and the American Thoracic Society.13
In the first published investigation of BIS monitoring in the ICU, De Deyne and colleagues in Belgium studied 18 deeply sedated surgical ICU patients, all of whom were unresponsive to bedside stimulation and deeply sedated according to the clinical Ramsay Sedation Score (RSS).10 The BIS score was below 60, indicating deep sedation, in 15 of the 18 patients, but the scores varied widely. De Deyne et al concluded that more extensive study was needed.
Simmons and associates at Maine Medical Center evaluated multiple assessments of sedation in 63 sedated, ventilated adult ICU patients.11 A single, trained observer blinded to the bispectral index prospectively evaluated the clinical level of sedation using the revised Sedation-Agitation Scale (SAS). Sedation levels varied widely, from very deep sedation (SAS 1, BIS 43) to mild agitation (SAS 5, BIS 100). The average BIS score correlated statistically with the average SAS score (r2 = 0.21; P < 0.001). However, the correlation between the two varied in medical, surgical, and trauma patients, and De Deyne et al concluded that further research was necessary to define the role of BIS monitoring in the ICU.
Nelson and associates from Honolulu recently presented results from a prospective study of 26 intubated, ventilated medical ICU patients.12 Nelson et al made simultaneous assessments of sedation using BIS, the Spectral Edge Frequency (SEF, a second computer-derived EEG parameter), the RSS, the Modified Observer Assessment of Alertness and Sedation Scale (MOASS), and the Glasgow Coma Scale (GSS). A blinded observer made all the clinical assessments. A total of 176 separate sets of assessments were made. Although statistically significant, the correlation between GCS and MOASS and BIS or SEF was not very strong, and the BIS or SEF scores could not be determined using the other tests. Disturbingly, some low BIS scores were obtained in patients who were awake, and high BIS scores in patients who were unarousable. In 17 readings, the BIS and SEF scores decreased when the patient was stimulated. Nelson et al concluded that, in their MICU patients, the bedside EEG monitor and BIS or SEF scores could not be used in lieu of clinical assessment of the level of sedation.
The most recent study was presented in May by de Wit and colleagues from Boston.13 They reported on simultaneous comparisons of BIS and SAS scores in 19 medical ICU patients, in whom 60 sets of measurements were made before and after stimulation. SAS scores ranged from 1 to 5, and BIS scores ranged from 32 to 98. Although a statistical correlation was obtained between the two assessments (r2 = 0.47 and 0.43 before and after stimulation, respectively), BIS varied widely for a given SAS. For SAS = 1, BIS scores ranged from 32-96; for 2, 36-98; for 3, 74-98; for 4, 75-98; and for 5, 89-97. de Wit et al concluded that, although SAS correlated with the level of awareness as assessed by BIS, at lower SAS scores the range of BIS scores was wide.
These last two studies have not yet appeared as full papers, and still need to be subjected to full peer review. Thus, the available database on BIS monitoring of critically ill patients in the ICU setting is pretty limited. In a 1999 editorial accompanying publication of the Simmons study previously cited, Shapiro stated, "More investigations are needed to determine the reliability of the BIS in the ICU and how we should use it to provide optimal sedation/analgesia."14 This remains the case.
Unfortunately, the definitive study establishing the role of BIS monitoring may not soon be forthcoming. In a recent power- and cost-analysis of what would be required to demonstrate whether BIS monitoring prevents awareness during general anesthesia, O’Connor and associates estimated that a study of as many as 800,000 patients might be required, depending on the frequency of the event in question.15 They calculated that the cost to prevent a single case of awareness during anesthesia could be as high as $400,000, although it could also be much less. O’Connor et al point out that there are reported cases of awareness during anesthesia despite BIS values indicating adequate sedation.15 Based on the available data from observations in the ICU, the frequency of this complication could be considerably higher there.
Conclusion
Innovations in ICU monitoring—new gizmos—typically reach clinical introduction and become part of routine patient care with less rigorous testing than occurs with new pharmacologic agents, particularly with respect to the clinical usefulness of the data generated. The management of critically ill patients has been greatly aided by some of these gizmos, although others have added only complexity and expense and a few have been clinically harmful. Because of the manner in which new monitoring technology and approaches reaches the bedside, it is up to the clinician to maintain objectivity and a degree of skepticism until the real value of a particular gizmo has been demonstrated.
Monitoring sedation in critically ill patients, particularly in the face of neuromuscular blockade, is an extremely important activity presently guided by cumbersome and imprecise tools. This is definitely an aspect of critical care that is in need of innovation. The BIS monitor has been embraced by many anesthesiologists as a convenient, quantitative aid in the prevention of awareness during anesthesia and to hasten postanesthesia awakening. The possibility that this new gizmo might also be helpful in guiding sedation in critically ill ICU patients is exciting and should stimulate wider investigation. For now, however, it is experimental in the ICU, and I would be hesitant to use it outside the investigational setting, particularly as the sole assessment of a patient’s level of sedation.
References
1. Sennholz G. Bispectral analysis technology and equipment. Minerva Anestesiol. 2000;66(5):386-388.
2. Glass PS, et al. Bispectral analysis measures sedation and memory effects of propofol, midazolam, isoflurane, and alfentanil in healthy volunteers. Anesthesiology. 1997;86(4):836-847.
3. Encarta World English Dictionary. New York, NY: St. Martin’s Press; 1999:755.
4. Pierson DJ. Goals and indications for monitoring. In: Tobin MJ, ed. Principles and Practice of Intensive Care Monitoring. New York, NY: McGraw-Hill; 1998:33-44.
5. Slutsky AS. FDA: The regulatory process. Respir Care. 1995;40(9):952-956.
6. Rubenfeld GD. Study design in the evaluation of intensive care monitoring. In: Tobin MJ, ed. Principles and Practice of Intensive Care Monitoring. New York, NY: McGraw-Hill; 1998:1425-1430.
7. American Respiratory Care Foundation. Consensus statement: Assessing innovations in mechanical ventilatory support. Respir Care. 1995;40(9):928-932.
8. MacIntyre NR. Innovation in mechanical ventilation: What are the drivers? Respir Care. 2001;46(3):267-272.
9. www.aspectms.com (accessed 6/1/01)
10. De Deyne C, et al. Use of continuous bispectral index EEG monitoring to assess depth of sedation in ICU patients. Intensive Care Med. 1998;24:1294-1298.
11. Simmons LE, et al. Assessing sedation during intensive care unit mechanical ventilation with the bispectral index and the sedation-agitation scale. Crit Care Med. 1999;27(8):1499-1504.
12. Nelson RS, et al. Comparison between bispectral EEG and Ramsay sedation scale in mechanically ventilated patients [abstract]. Crit Care Med. 2000;28(12):A44.
13. De Wit M, et al. Correlation between clinical sedation score and bispectral index [abstract]. Am J Respir Crit Care Med. 2001;A900.
14. Shapiro BA. Bispectral index: Better information for sedation in the intensive care unit? [editorial] Crit Care Med. 1999;27(8):1663-1664.
15. O’Connor MF, et al. BIS monitoring to prevent awareness during general anesthesia. Anesthesiology. 2001 Mar;94(3):520-522.
Subscribe Now for Access
You have reached your article limit for the month. We hope you found our articles both enjoyable and insightful. For information on new subscriptions, product trials, alternative billing arrangements or group and site discounts please call 800-688-2421. We look forward to having you as a long-term member of the Relias Media community.