Deep-learning-based real-time remaining surgery duration (rsd) estimation

Abstract

Embodiments described herein provide a surgical duration estimation system for continuously predicting real-time remaining surgical duration (RSD) of a live surgical session of a given surgical procedure based on a real-time endoscope video of the live surgical session. In one aspect, the process receives a current frame of the endoscope video at a current time of the live surgical session, wherein the current time is among a sequence of prediction time points for making continuous RSD predictions during the live surgical session. The process next randomly samples N−1 additional frames of the endoscope video corresponding to the elapsed portion of the live surgical session between the beginning of the endoscope video corresponding to the beginning of the live surgical session and the current frame corresponding to the current time. The process then combines the N−1 randomly sampled frames and the current frame in the temporal order to obtain a set of N frames. Next, the process feeds the set of N frames into a trained model for the given surgical procedure. The process subsequently outputs a current RSD prediction based on the set of N frames.