Intuitive Surgical Operations, Inc.

Pourya Shirazian

Mahdi Azizian

Daniel Proksch

A Jonathan McLeod

Azad Shademan

Maximilian Hunter Allan

An exemplary image management system accesses semantic information regarding an anatomical object in a surgical space and registers, based at least in part on the semantic information regarding the anatomical object, a first dataset of a first visual representation of the surgical space with a second dataset of a second visual representation of the surgical space.

Systems and methods for registering visual representations of a surgical space

A method is provided to model a 3D structure comprising: producing a surface mesh representation of the 3D structure; producing a volume mesh representation of the 3D structure based upon the surface mesh; sorting vertices of the volume mesh into a first sub-list that includes only surface vertices and a second sub-list that includes only internal vertices; applying shading to the surface mesh by accessing only surface vertices in the first sub-list; determining deformation of the volume mesh by accessing both surface vertices in the first sub-list and internal vertices in the second sub-list.

Unified anisotropic volume and surface mesh storage

An exemplary contextual assistance system identifies, during a surgical procedure, an operational context of the surgical procedure while a particular object is present at a surgical site depicted by a first version of an image. Based on the identified operational context, the contextual assistance system augments a depiction of the particular object within a second version of the image that is presented to a user associated with the surgical procedure. Corresponding systems and methods are also disclosed.

Systems and methods for providing surgical assistance based on operational context

An exemplary system is configured to instruct a display device to render a virtual medical element representative of a physical medical element over a target region within an image of an internal space of a patient, the target region depicting an anatomical surface to be covered by the physical medical element; receive, while the virtual medical element is rendered over the target region, user input that sets at least one of a pose of the virtual medical element within the image or a size of the virtual medical element; and determine, based on the user input and on depth data representative of a depth map for the internal space, physical dimensions for the physical medical element.

Physical medical element sizing systems and methods

An illustrative system may be configured to obtain a visual image of an anatomical scene that includes an imaging probe and to augment the visual image with a synthetic element that indicates, within the visual image, an area of the anatomical scene being imaged by the imaging probe. The synthetic element may include spatial markers. The visualization system may be further configured to direct a display device to display the augmented image. The visualization system may be further configured to direct the display device to display, at a position outside of the synthetic element, a probe image captured by the imaging probe.

Anatomical scene visualization systems and methods

A contact detection system is configured to classify each pixel in a plurality of pixels included in an ultrasound image captured by an ultrasound probe located within a patient as either showing tissue or showing non-tissue; determine an average pixel classification representative of a number of pixels classified as showing tissue compared to a number of pixels classified as showing non-tissue; determine, if the average pixel classification is above a first contact state threshold, that the ultrasound probe is in a first contact state that indicates that the ultrasound probe is in operative physical contact with the tissue; and determine, if the average pixel classification is below a second contact state threshold lower than the first contact state threshold, that the ultrasound probe is in a second contact state that indicates that the ultrasound probe is not in operative physical contact with the tissue.

Systems and methods for detecting tissue contact by an ultrasound probe

An exemplary image registration system identifies a subsurface structure at a surgical site based on subsurface imaging data from a subsurface image scan at the surgical site. The image registration system uses the identified subsurface structure at the surgical site for a registration of endoscopic imaging data from an endoscopic imaging modality with additional imaging data from an additional imaging modality. Corresponding systems and methods are also disclosed.

Systems and methods for identifying differentiated blood vessels and registering imaging data from different imaging modalities based on subsurface image scanning

An exemplary system is configured to obtain anatomical characteristic data representative of a characteristic associated with an anatomical surface to be covered by a physical medical element, the anatomical surface within an internal space of a patient and determine, based on the anatomical characteristic data, a placement guidance parameter set. The placement guidance parameter set may include one or more parameters configured to guide a placement of the physical medical element on the anatomical surface with one or more surgical instruments controlled by a computer-assisted surgical system.

Physical medical element placement systems

An exemplary tissue volume detection system accesses, during a surgical procedure involving resecting a piece of tissue from a body, a plurality of depth datasets for the resected piece of tissue. Each of the plurality of depth datasets is captured as a different portion of a surface of the resected piece of tissue is presented to an imaging device by an instrument that holds the resected piece of tissue in a manner that sequentially presents the different portions of the surface to the imaging device. During the surgical procedure and based on the depth datasets, the system generates a three-dimensional (3D) occupancy map that includes a set of voxels identified to be occupied by the resected piece of tissue. Based on the 3D occupancy map and still during the surgical procedure, the system determines an estimated volume of the resected piece of tissue. Corresponding systems and methods are also disclosed.

Systems and methods for determining a volume of resected tissue during a surgical procedure

A composite medical imaging system may direct a display device to display an image captured by an imaging device and showing a view of a surgical area and display an augmentation region within the image and that shows supplemental content. The view of the surgical area shows surface anatomy located at the surgical area and an object located at the surgical area. The augmentation region creates an occlusion over at least a portion of the view of the surgical area. The system may detect an overlap in the image between at least a portion of the object and at least a portion of the augmentation region. In response to the detection of the overlap, the system may adjust the image to decrease an extent of the occlusion within the overlap by the augmentation region.

Composite medical imaging systems and methods

An exemplary system accesses imagery of a surgical space captured by different imaging modalities and, based on the accessed imagery, generates composite imagery that includes integrated representations of the surgical space as captured by the different imaging modalities. An exemplary composite image includes a representation of the surgical space as captured by a first imaging modality augmented with an integrated representation of the surgical space as captured by a second imaging modality. The integrated representation of the surgical space as captured by the second imaging modality may be selectively movable and may be generated based on first imagery of the surgical space captured by the first imaging modality and second imagery of the surgical space captured by the second imaging modality in a manner that provides a visually realistic appearance of depth.

Systems and methods for integrating imagery captured by different imaging modalities into composite imagery of a surgical space

Technology described herein can be embodied in a method of providing visual feedback during surgery. The method includes obtaining a set of vectors representing a model of an anatomical organ, the set produced from a representation of the organ as imaged under one or more imaging modalities. The method includes obtaining a set of pixel positions of a display device, wherein the pixel positions correspond to a view of the organ. The method further includes generating, using at least a portion of the set of vectors, multiple fragment-vectors for each pixel position in the set of pixel positions, the multiple fragment-vectors each representing a corresponding portion of the organ along a line-of-sight through the organ, and generating a pixel value for each pixel position in the set of pixel positions, and presenting, on the display device, an augmented image of the view of the organ using the generated pixel values.

Growing community of inventors

Sunnyvale, CA, United States of America

Pourya Shirazian