AI Tries to Improve Surgeons' On-Time Performance

Related articles

Perhaps it is only me, but I suspect that it is true for the majority of my colleagues, scheduling our time in the operating room is a very personal matter. That should be no surprise, most artisans or craftsmen (craftspeople would be more gender-neutral, but it just sounds jarring) are very particular about the tools, team, and environment that they craft within. I always liked the top of the schedule, so I could do my cases and be on my way; no waiting, as some other surgeon, dwadled in the moment for no particular reason, or showed up late. So when it came to booking OR time, I handled that myself, wheeling and dealing as best I could to get the times and sequence that I wanted. 

Multiply that behavior by 30 or 40 people competing for only so many slots of time, and you can see why OR booking staff begin to twitch when the surgeon wants to speak with them. I will not bother you with the details of the methods used to arrive at an equitable schedule, but suffice to say, in most cases our previous operative times were the default value for the time we needed, unless we insisted that things were going to be different today. A new study lets machine learning take on scheduling. Can it improve the efficiency of a very important hospital profit center?

Inaccurate timing causes a lot of downstream headaches. Finish too soon, and the next surgeon or their patient may not be ready. Yes, we can bring the patients into a buffer area, we call it the holding area, but in today’s just in time world of patient satisfaction, no one wants to wait a few hours, just to be available. Finish late, and everyone behind you is delayed, you start running into shift changes which as the day turns to night means overtime or less staff. On-time performance is a good thing, for everyone. 

Prior performance does not guarantee subsequent performance

Yes, like those financial disclaimers, a surgeon’s prior performance may differ from the case being scheduled, so often the surgeons' subjective evaluation overrides the average determined by operating room logs. In this study, a machine learning algorithm was given the log for 31,000 cases and asked to predict operative times for the remaining 8,000. The outcome measure was how well each method predicted operative times within 10% of actuality. 

  • Kicking it old school, simply using the average time for the last 10 similar cases, 30% of predictions were within range, 42% over, and 27% under their averages.
  • When surgeons overrode the estimates, prediction accuracy improved by about 2%. [1]
  • Machine learning improved prediction accuracy to 50%. The factors exerting the most influence, the surgeon’s track record on times for that specific procedure - the old school measure; patient status as an inpatient or outpatient trailed far behind those factors. A variety of other patient characteristics contributed slight improvements in predictive values. 

The study suggests that time estimates may be a little harder to do than anticipated. As always the researchers report limitations, single-center, academic center, greater patient acuity. I would argue that an obvious omission is a greater limitation. Operating times were defined as wheels-in to wheels-out (have to love those airplane analogies), meaning from the time the patient entered the operating room till they left. But that time includes three distinct phases, putting the patient asleep, doing the surgery, and waking the patient up. 

Those first and last activities are done by the anesthesia team, and I will say, at least anecdotally, that induction and emergence can vary significantly by patient as well as the one doing the putting to sleep and waking up. Anesthesia teams are usually not assigned until the schedule is finalized so it cannot be fully taken into account, but it is an important variable. It would be interesting to see the impact of those times on the estimates, by both surgeon and machine.

There was a final concern, in the conflict of interest disclosures one of the study’s authors reports that he is a paid consultant and owns an equity position in Perimatics, but that it is “outside of the scope of this work.” With an idle moment on my hands I took the opportunity to look at that company. Just to be clarify the record, he is the company's cofounder, it is a spin-off of the University where the research was performed and according to at least one source,

“The startup, which develops a variety of technologies for hospitals, is taking aim at the operating room problem with a new AI technology that uses data on patients and surgeons to more accurately predict how long each surgery will take.”

Does that sound like it is “outside the scope” of this research to you? Just asking. 

 

[1] Most surgeon estimates lowered the time necessary. I would suggest that when trying to cram 5 hours of surgery into a 4-hour block, that is the most frequent approach – “lie” about the length of time for at least one case. 

 

Source: Improving Operating Room Efficiency: A Machine Learning Approach to Predict Case Time Duration Journal American College of Surgeons DOI: 10.1016/j.jamcollsurg.2019.05.029