Intuitive Surgical Operations, Inc.

Heath Feather

Alexandre V Matveev

Randal P Goldberg

Paul W Mohr

Brian David Hoffman

Daniel H Gomez

Tabish Mustufa

Huan L Phan

Brandon D Itkowitz

Christopher R Burns

Joseph M Arsanious

Ian E McDowall

Jeffrey M DiCarlo

Zhen He

Siddarth Sen

Rohitkumar Godhani

Adam T Schmidt

A console for a medical includes a display, an eye tracking input module, a second input module, and one or more processors coupled to the display, the eye tracking input module, and the second input module. The one or more processors is configured to perform operations including receiving a first user input from the eye tracking input module; determining, based on the first user input, whether a gaze of a user is directed toward an activation region of the display; in response to determining that the gaze of the user is directed toward the activation region, displaying, via the display, a control; receiving a second user input from the second input module; and adjusting the control based on the second user input.

System and method for hybrid control using eye tracking

An exemplary processing system of a computer-assisted surgical system identifies an object in a surgical space, accesses a model of the object, identifies a pose of the object in the surgical space, aligns the model with the pose of the object to define depth data for the model in the surgical space, and generates a depth map in the surgical space for at least a portion of the object based on the depth data for the model. Corresponding systems and methods are also described.

Using model data to generate an enhanced depth map in a computer-assisted surgical system

An apparatus includes a controller coupled to a stereoscopic image viewer. The stereoscopic image viewer includes a viewing lens, an image capture unit, and a plurality of light sources. The image capture unit is configured to capture image frames. The controller receives the captured image frames. The controller is configured to switch light output from one of the plurality of light sources to another one of the plurality of light sources if a corneal reflection in a captured image frame intersects a fixed reflection in the captured image frame.

Dynamic illumination for eye-tracking

A system comprises a display configured to present a graphical user interface including a viewing area and one or more user interface elements. The system also comprises a gaze tracking device configured to detect a change in a gaze of a user while the user views the graphical user interface presented on the display. The system also comprises an audio input device configured to receive audio information. The system also comprises one or more processors configured to process the audio information received at the audio input device in accordance with a first mode to direct the received audio information as audio output to one or more audio output devices; and in response to the gaze tracking device detecting the change in the gaze of the user, process the received audio information in accordance with a second mode, the second mode being distinct from the first mode.

Gaze-initiated communications

An exemplary image viewing system includes a display device configured to emit visible light representative of an image provided for display by the image viewing system, an infrared light source configured to emit infrared light, a viewing lens, and a black glass mirror provided between the infrared light source and the viewing lens. The black glass mirror comprises black glass and a coating on an outer surface of the black glass. The coating is configured to reflect a first portion of the visible light emitted from the display device towards the viewing lens, pass a second portion of the visible light emitted from the display device to the black glass, and pass a portion of the infrared light emitted from the infrared light source toward the viewing lens. The black glass is configured to pass the portion of the infrared light and absorb the second portion of the visible light.

Image viewing systems and methods using a black glass mirror

An illustrative apparatus may perform gaze-based auto-exposure management of image frames. For example, the apparatus may determine, for an image frame within an image frame sequence captured by an image capture system, auto-exposure values for pixel units into which the image frame is divided. The apparatus may then assign weight values to the pixel units based on a gaze direction of a viewer viewing the image frame sequence. Based on the auto-exposure values determined for the pixel units and the weight values assigned to the pixel units, the apparatus may update one or more auto-exposure parameters for use by the image capture system to capture an additional image frame within the image frame sequence. Corresponding apparatuses, systems, and methods for gaze-based auto-exposure management of image frames are also disclosed.

Apparatuses, systems, and methods for gaze-based auto-exposure management of image frames

A teleoperational medical system comprises a teleoperational assembly including an operator control system, a first teleoperational manipulator, a first medical instrument, and a processing unit including one or more processors. The first teleoperational manipulators is configured for operation by an operator control device of the operator control system. The first teleoperational manipulator is configured to control the operation of the first medical instrument in a surgical environment. The processing unit is configured to display an image of a field of view of the surgical environment and display a menu proximate to an image of the first medical instrument in the image of the field of view. The menu includes at least one state indicator representing a state of a component of the first medical instrument.

Systems and methods for onscreen menus in a teleoperational medical system

A method comprises displaying an image of an environment on a graphical user interface, the environment including personnel elements visible in the image, detecting a gaze of a user directed at an image of a first personnel element corresponding to a first information conveyor of a plurality of information conveyors, changing a state of the first information conveyor corresponding to the first personnel element from a passive state to a listening state in response to detecting that the gaze is directed at the image of the first personnel element, wherein the first information conveyor includes a first audio output device, receiving audio information through an audio input device, and when the state of the first information conveyor is changed to the listening state, controlling an output operation, wherein the output operation includes directing the received audio information as an audio output through the first audio output device.

A method comprises displaying an image of a field of view of a surgical environment. A first medical instrument in the field of view may be coupled to a first manipulator in a teleoperational assembly. The method may comprise displaying a menu proximate to an image of the first medical instrument in the image of the field of view. The menu may include a plurality of icons wherein each icon is associated with a function for the first medical instrument. The method may also comprise identifying a selected icon from the plurality of icons based upon a movement of an operator control device of a teleoperational operator control system.

A system comprises a teleoperational assembly including an operator control system and a first teleoperational manipulator configured for operation by an operator control device of the operator control system. The first teleoperational manipulator is configured to control the operation of a first medical instrument in a surgical environment. The system also comprises a processing unit including one or more processors. The processing unit is configured to display an image of a field of view of the surgical environment and display a menu proximate to an image of the first medical instrument in the image of the field of view. The menu includes at least one icon representing a function for the first medical instrument.

Growing community of inventors

Cupertino, CA, United States of America

Heath Feather