Verily Life Sciences LLC

Willow Innovations, Inc.

Exploramed Nc7, Inc.

Joëlle Barral

Xing Jin

Caitlin Q Donhowe

Martin Habbecke

Michal Levin

Asha Shrinivas Nayak

Sandra Waugh Ruggles

Mary Garrett

Venita Chandra

Beverly Tang

Erika Palmer

James Shuma

Jessica Hudak

Eden Rephaeli

Blake Hannaford

Vikram Singh Bajaj

Christine Denise Ching

Robin Thellend

Neil Inala

Timothy Chung

Michael Newton

Brian Carnes

Daniel Hiranandani

Systems and methods for eye tracking and content tagging in minimally invasive surgical videos are described herein. Minimally invasive surgical videos may be captured while performing robotic surgeries. Robotic surgical systems described herein include robotic arms with interchangeable surgical tools. An endoscope at the end of one of the robotic arms captures video of the surgical procedure. The video is displayed on a display of the surgical system and an eye tracking device captures data corresponding to a gaze direction of the user on the display. Content tags are automatically generated in the image data based on areas of focus of the user. Information within the content tag may be generated based on surgical procedure steps derived from the image data, a log of current surgical procedures, and image data of previous surgical procedures.

Automatic content tagging in videos of minimally invasive surgeries

Systems and methods for configuring and instructing movement of a surgical device using configuration data from previous surgeries are described herein. Robotic surgical systems described herein include robotic arms with interchangeable surgical tools. The kinematics of the robotic arms are compared against configuration data from previous procedures and differences between the kinematics and the configuration data are identified. The configuration data also includes simulations of robotic surgical systems throughout a procedure. Potential collisions or complications from present and future configurations are identified. The differences and instructions for resolving the differences and potential collisions are presented to a user for correction.

Surgical robotic system configuration

One example method for improving robotic surgical safety via video processing includes identifying, during a robotic surgical procedure, one or more surgical tools using one or more images of the surgical procedure captured by a camera, the robotic surgical procedure employing a robotic surgical device controlling the one or more identified surgical tools; predicting, for at least one of the one or more images, one or more loaded surgical tools that are controlled by the robotic surgical device and should be in a field of view of the camera; comparing the one or more identified surgical tools with the one or more loaded surgical tools; determining that at least one of the one or more loaded surgical tools does not match any of the one or more identified surgical tools; and causing the at least one loaded tool of the robotic surgical device to be disabled.

Robotic surgical safety via video processing

Systems and methods for depth detection and virtual modeling using surgical robots during a surgical procedure are described. The robotic surgical systems include robotic arms with interchangeable surgical tools. An endoscope at the end of one of the robotic arms includes a depth sensor for detecting a distance from the camera to patient anatomy. The depth data and image data are used to generate a feature model of the patient anatomy for reference by a surgeon. The feature model is displayed to the surgeon in an augmented reality view with preoperative and intraoperative images mapped thereon for reference and guidance of a surgical procedure.

Robotic surgery depth detection and modeling

Systems and methods for identifying surgery tools of a robotic surgery system in surgery video based on robot kinematics are described. Kinematic data describing a position of a first robotic surgery arm including a surgery tool is used to determine a nominal position of the first robotic surgery arm. A search area within image data obtained during a robotic surgery from an endoscope attached to a second robotic surgery arm is determined based on the kinematic data and a nominal position of the surgery tool. The surgery tool is identified within the search area of the image data using an object detection technique.

Surgery tool segmentation with robot kinematics

Examples of systems and methods for detecting delays during a surgical procedure are disclosed. For example, one disclosed method includes receiving, a computing device, video of a robotic surgical procedure; determining, by the computing device, an estimated time-to-complete ('TTC') the robotic surgical procedure based on the video; and in response to determining a deviation between the estimated TTC and an expected duration of the robotic surgical procedure exceeds a threshold, outputting an indication that the robotic surgical procedure is deviating from the expected duration.

Systems and methods for detecting delays during a surgical procedure

One example method for detecting phases of a surgical procedure via video processing includes accessing a video of the surgical procedure and dividing the video into one or more blocks, each of the blocks containing one or more video frames. For each of the blocks, the method includes applying a prediction model on the video frames of the respective block to obtain a phase prediction for each of the video frames. The prediction model is configured to predict, for an input video frame, one of the plurality of phases of the surgical procedure. The method further includes generating an aggregated phase prediction for the respective block by aggregating the phase predictions of the video frames, and modifying the video of the surgical procedure to include an indication of a predicted phase of the respective block based on the aggregated phase prediction.

Surgical workflow and activity detection based on surgical videos

One example method for robotic surgery using multi-user authentication without credentials includes receiving, by a robotic surgical device, a case code associated with a medical procedure; validating the case code; determining one or more users associated with the case code; and providing access to the one or more users to the robotic surgical device to enable a robotic surgical procedure.

Robotic surgery using multi-user authentication without credentials

A system for synchronizing one or more breast-pumping sessions of an individual (such as a mother) and milk consumption by a second individual (such as a baby or infant) is described. In particular, based on measurements of a volume of the collected milk as a function of time and received information specifying milk consumption by the second individual as a function of time, a control circuit may determine a need for milk. Then, the control circuit may provide feedback based on the determined need for milk that synchronizes the one or more breast-pumping sessions and the milk consumption. For example, the feedback may alert the individual to initiate a breast-pumping session and/or may provide a signal to a breast pump that initiates a breast-pumping session.

Synchronizing breast pumping with infant feeding

A system for providing networked communications includes a plurality of head-mountable devices, each in communication with a control system via a communication network. Each of the plurality of head-mountable devices includes a display, and may also include an image-capture device and/or a microphone. The control system receives, via the communication network, surgical data of a patient obtained from at least one source of surgical data. Wearers of the head-mountable devices may select aspects of the surgical data and the control system causes those selected aspects to be displayed on the respective wearable device. The control system may also generate alerts and cause the alerts to be displayed on the wearable devices. An alert may include a notification that additional surgical data is available for access by the wearer of the head-mountable device.

Growing community of inventors

Mountain View, CA, United States of America

Joëlle Barral