Verb Surgical Inc.

Varun Kejriwal Goel

Jocelyn Elaine Barker

Richard William Timm

Meysam Torabi

Taylor W Aronhalt

Pablo E Garcia Kilroy

Danyal Fer

Darrick Tyler Sturgeon

Amer Ghanem

Rami Abukhalil

Embodiments described herein include a process for detecting energy tool activations. The process can begin by receiving a surgical video of a surgical procedure involving energy tool activations. The process then applies a sequence of sampling windows to the surgical video to generate a sequence of windowed samples of the surgical video. Next, for each windowed sample in the sequence of windowed samples, the process applies a deep-learning model to a sequence of video frames within the windowed sample to generate an activation/non-activation inference and a confidence level associated with the activation/non-activation inference for the windowed sample. As a result, a sequence of activation/non-activation inferences and a sequence of associated confidence levels are generated. The process subsequently identifies a sequence of activation events in the surgical video based on the sequence of activation/non-activation inferences and the sequence of associated confidence levels.

Energy tool activation detection in surgical videos using deep learning

Embodiments described herein provide various techniques and systems for building machine-learning surgical tool presence/absence detection models for processing surgical videos and predicting whether a surgical tool is present or absent in each video frame of a surgical video. In one aspect, a process for ensuring patient safety during a laparoscopic or robotic surgery involving an energy tool is disclosed. The process can begin receiving a real-time control signal indicating an operating state of an energy tool during the surgery. Next, the process receives real-time endoscope video images of the surgery. The process simultaneously applies a machine-learning surgical tool presence/absence detection model to the real-time endoscope video images to generate real-time decisions on a location of the energy tool in the real-time endoscope video images. The process then checks the real-time control signal against the real-time decisions to identify an unsafe event and takes a proper action when an unsafe event is identified.

Real-time surgical tool presence/absence detection in surgical videos

An analysis system trains a machine learning model to detect stapling events from a video of a surgical procedure. The machine learning model detects times when stapling events occur as well as one or more characteristics of each stapling event such as length of staples, clamping time, or other characteristics. The machine learning model is trained on videos of surgical procedures identifying when stapling events occurred through a learning process. The machine learning model may be applied to an input video to detect a sequence of stapler events. Stapler event sequences may furthermore be analyzed and/or aggregated to generate various analytical data relating to the surgical procedures for applications such as inventor management, performance evaluation, or predicting patient outcomes.

Video-based analysis of stapling events during a surgical procedure using machine learning

Growing community of inventors

Oakland, CA, United States of America

Varun Kejriwal Goel