Intuitive Surgical Operations, Inc.

Hewlett-packard Development Company, L.p.

Leland Stanford Junior University

Jeffrey M DiCarlo

Ian E McDowall

Wenyi Zhao

Brian David Hoffman

Max J Trejo

Glen Eric Montgomery

Geoff Richmond

Jonathan Michael Sorger

William Jason Culman

Miheer M Bhachech

Zhen He

Tao Zhao

Stephen J Blumenkranz

Steven W Trovinger

Melanie M Gottwals

David D Scott

Gary Joseph Dispoto

Nathan McPherson Moroney

Carl E Picciotto

Casey L Miller

Curtis W Stith

Alexander L Antaris

Xinqiao Liu

Gregory J May

David C Shafer

Anurag Gupta

Abbas A El Gamal

Mark Q Shaw

Timothy Louis Kohler

I-Jong Lin

Heath Feather

Robert L Cobene

Brian A Wandell

Michael D Long

Changmeng Liu

Steven Douglas Stoecker

Les Gehman

Eric Montgomery

An exemplary fluorescence imaging control system may direct, during a surgical procedure performed with a computer-assisted surgical system, an illumination source included in the computer-assisted surgical system to illuminate a scene associated with the surgical procedure with fluorescence excitation illumination configured to excite a fluorophore present at the scene and direct an imaging device included in the computer-assisted surgical system to detect, during the surgical procedure, fluorescence emitted by the fluorophore in response to excitation of the fluorophore by the fluorescence excitation illumination. The fluorescence imaging control system may determine, based on the detected fluorescence, a lifetime of the fluorescence and determine, based on the determined lifetime of the fluorescence, an identity of the fluorophore. The fluorescence imaging control system may configure, during the surgical procedure and based on the determined identity of the fluorophore, operation of the computer-assisted surgical system.

Systems and a method for directing an imaging device to detect fluorescence and for determining a lifetime of the fluorescence

In some embodiments, a controller of a computer-assisted surgical system receives a visible image captured by an image capture unit of a surgical system, receives a plurality of hyperspectral images captured at a same waveband by the image capture unit of the surgical system, generates a composite hyperspectral image from the plurality of hyperspectral images, and spatially registering the composite hyperspectral image with the visible image.

Simultaneous white light and hyperspectral light imaging systems

An exemplary medical imaging system includes a visible light illumination system configured to selectively emit one of a first visible light biased to a first wavelength and a second visible light biased to a second wavelength, a fluorescence excitation illumination system configured to selectively emit one of a first fluorescence excitation illumination having a third wavelength and a second fluorescence excitation illumination having a fourth wavelength, and an illumination source control unit configured to selectively direct the visible light illumination system to emit the second visible light and the fluorescence excitation illumination system to emit the first fluorescence excitation illumination for use with a first fluorescence imaging agent, and selectively direct the visible light illumination system to emit the first visible light and the fluorescence excitation illumination system to emit the second fluorescence excitation illumination for use with a second fluorescence imaging agent.

Medical imaging systems and methods that facilitate use of different fluorescence imaging agents

An illustrative apparatus may determine a color-skew metric for an image frame captured by an image capture system. The color-skew metric may be indicative of an extent to which the image frame skews to a particular color. Based on the color-skew metric and an adaptive target control function, the apparatus may determine a frame auto-exposure target. Based on the frame auto-exposure target, the apparatus may update one or more auto-exposure parameters for use by the image capture system to capture an additional image frame. Corresponding apparatuses, systems, and methods for managing auto-exposure of image frames are also disclosed.

Apparatuses, systems, and methods for managing auto-exposure of image frames depicting color-biased content

Technology described herein can be embodied in a method of displaying a visual representation of a portion of a surgical scene. The method includes receiving data representing information captured using a first sensor of a camera associated with a surgical device, the information being indicative of a first quantity representing an amount of fluorescence emitted from the portion of the surgical scene. The method also includes obtaining information indicative of a second quantity representing an amount of excitation signal causing the fluorescence to be emitted from the portion of the surgical scene, and generating a normalized fluorescence signal as a function of the first quantity and the second quantity. The method further includes generating the visual representation of the portion of the surgical scene based on the normalized fluorescence signal, and presenting the visual representation on a display device associated with the surgical device.

Illumination corrected near-infrared (nir) imaging for image guided surgery

An example method includes causing an illuminator to simultaneously illuminate tissue with a first visible color illumination component, a second visible color illumination component, and a fluorescence excitation illumination component; and causing a display unit to display, based on the illumination, a display scene that includes: a reduced color scene component corresponding to the first and second visible color illumination components, and a highlighted scene component corresponding to the fluorescence excitation illumination component.

Multi-function imaging

An illustrative apparatus may identify, within an image frame captured by an image capture system, a signal region that includes pixels having auto-exposure values exceeding an auto-exposure value threshold. The apparatus may adjust, based on the auto-exposure values of the pixels included within the signal region, one or more auto-exposure parameters used by the image capture system to capture an additional image frame. Additionally, the apparatus may determine, based on a size of the signal region within the image frame, whether to change the auto-exposure value threshold. Corresponding apparatuses, systems, and methods for managing auto-exposure of image frames are also disclosed.

Apparatuses, systems, and methods for managing auto-exposure of image frames based on signal region size

A system directs an imaging device to continuously capture visible light and fluorescence illumination from a surgical area. The system generates a visible light image stream based on the captured visible light and a fluorescence image stream based on the captured fluorescence illumination. The system operates in a first display mode by directing a display device to display a first video stream based on a first set of at least one of the visible light image stream and the fluorescence image stream. In response to detecting an event that occurs within the surgical area, the system switches from operating in the first display mode to operating in a second display mode by directing the display device to display a second video stream based on a second set of at least one of the visible light image stream and the fluorescence image stream, the first Instructions set being different than the second set.

Augmented medical vision systems and methods

An image sensing apparatus including a pixel array comprising two or more photo-sensor elements, a first optical filter disposed on a first photo-sensor element of the pixel array and a second optical filter disposed on a second photo-sensor element of the pixel array. The first optical filter is configured such that a spectral response of the first optical filter includes a first passband in a first wavelength range and a second passband in a second wavelength range, where the first passband and the second passband are separated by a first stop band. The second optical filter is configured such that a spectral response of the second optical filter includes a third passband in the first wavelength range and a fourth passband in the second wavelength range, where the third passband and the fourth passband are separated by a second stop band.

Multiple band pass Bayer pattern filter

An illustrative apparatus may identify, within an image frame captured by an image capture system, a signal region of the image frame and a background region of the image frame. The apparatus may determine one or more of a signal auto-exposure value of the signal region or a background auto-exposure value of the background region. Based on one or more of the signal auto-exposure value or the background auto-exposure value, the apparatus may determine a frame auto-exposure value. Additionally, based on the frame auto-exposure value, the apparatus may adjust one or more auto-exposure parameters used by the image capture system to capture an additional image frame. Corresponding apparatuses, systems, and methods for managing auto-exposure of image frames are also disclosed.

Apparatuses, systems, and methods for managing auto-exposure of image frames depicting signal content against a darkened background

A two-by-two pixel array within an image sensor includes a first pixel, a second pixel, a third pixel, and a fourth pixel. A red filter covers the first pixel, a first blue filter covers the second pixel, a second blue filter covers the third pixel, and a green filter covers the fourth pixel. The red filter is configured to allow the first pixel to collect a red component of visible light and prevent the first pixel from collecting blue and green components of the visible light. The first and second blue filters are configured to allow the second and third pixels to each collect the blue component and prevent the second and third pixels from collecting the red and green components. The green filter is configured to allow the fourth pixel to collect the green component and prevent the fourth pixel from collecting the red and blue components.

Medical imaging systems and methods

An illustrative surgical system may access a plurality of images captured outside a structure within a patient; detect a difference between spectral reflectances of scenes captured in the plurality of images; and identify, based on the detected difference between the spectral reflectances of the scenes captured in the plurality of images, pixels in at least one of the plurality of images that correspond to structure tissue of the structure.

Waveband-selective imaging systems and methods

Technology described herein can be embodied in a method that includes obtaining, using an endoscopic camera, a plurality of images of the surgical scene, each image in the plurality of images being obtained under illumination by electromagnetic radiation in a corresponding wavelength range. The corresponding wavelength ranges are selected in accordance with a set of chromophores present in the surgical scene, and any two of the corresponding wavelength ranges partially non-overlapping. The method also includes determining, based on the plurality of images, a concentration of each of one or more chromophores in the set at various portions of the surgical scene, and generating, by one or more processing devices, based on information about the concentration of each of the one or more chromophores, a representation of the surgical scene. The method further includes presenting the representation of the surgical scene on an output device associated with the surgical device.

Determining chromophore concentrations in tissues

The technology described herein can be embodied in a method that includes receiving data representing information captured using a first sensor and a second sensor of a multi-sensor camera. The sensors are configured to capture a surgical scene illuminated by a light source configured to emit wavelengths in the visible spectrum corresponding to sensing capabilities of the first and second sensors, respectively. The method also includes receiving data representing information captured using a third sensor of the multi-sensor camera, the third sensor configured to capture the surgical scene as illuminated by a near-infrared light source, and generating a first visual representation of the surgical scene based on the data representing the information captured using the first and second sensors. The first visual representation is combined with the information captured using the third sensor to generate a second visual representation, and the second visual representation is presented on a display device.

Combining near-infrared information with colored images in image-guided surgery

An illustrative apparatus may perform gaze-based auto-exposure management of image frames. For example, the apparatus may determine, for an image frame within an image frame sequence captured by an image capture system, auto-exposure values for pixel units into which the image frame is divided. The apparatus may then assign weight values to the pixel units based on a gaze direction of a viewer viewing the image frame sequence. Based on the auto-exposure values determined for the pixel units and the weight values assigned to the pixel units, the apparatus may update one or more auto-exposure parameters for use by the image capture system to capture an additional image frame within the image frame sequence. Corresponding apparatuses, systems, and methods for gaze-based auto-exposure management of image frames are also disclosed.

Growing community of inventors

Palo Alto, CA, United States of America

Jeffrey M DiCarlo