Verily Life Sciences LLC

Nishant Verma

Ryan L Robertson

Aurelien Philippe Bouvier

Thomas Teisseyre

James Polans

Brian Anderson

Systems and methods for specular reflection reduction in endoscope visualizations are described. A method includes receiving an image including a region of specular reflection. The method includes detecting the region of specular reflection in the image. The method includes estimating image information for a portion of the region of specular reflection. The method also includes reconstructing the image including the image information populated into the region of specular reflection.

Specular reflection reduction in endoscope visualization

One example method for scene-adaptive image quality in surgical video includes receiving a first video frame from an endoscope, the first video frame generated from a first raw image captured by an image sensor of the endoscope and processed by an image signal processing ('ISP') pipeline having a plurality of ISP parameters; recognizing, using a trained machine learning (“ML”) model, a first scene type or a first scene feature type based on the first video frame; determining a first set of ISP parameters based on the first scene type or the first scene feature type; applying the first set of ISP parameters to the ISP pipeline; and receiving a second video frame from the endoscope, the second video frame generated from a second raw image captured by the image sensor and processed by the ISP pipeline using the first set of ISP parameters.

Systems and methods for scene-adaptive image quality in surgical video

One example method includes obtaining, from an endoscope and between first and second video frames of a real-time video having a frame rate, a preliminary image of a scene during a surgical procedure, the first image comprising a plurality of pixels, wherein the first and second video frames are consecutive video frames in the real-time video; determining, for each pixel of the plurality of pixels, a depth within the scene; determining first image capture parameters for the scene based on a scene illumination setting and a first set of pixels within a first range of depths in the scene; capturing, between the first and second consecutive output video frames, a first image using the first image capture parameters; determining an illumination correction for a second set of pixels at a second range of depths within the scene; capturing, between the first and second consecutive output video frames, a second image using the illumination correction; generating a composite image based on the first and second images; and outputting the composite image as the second output video frame.

Image signal processing to compensate for scene changes

Methods, endoscopic systems, and non-transitory, machine-readable storage media for adjusting an exposure of an endoscopic system are described. In an embodiment, the methods include emitting light with a light source into a proximal end of an endoscope light pipe; generating received light signals with a photodetector based on light received through the endoscope light pipe by the photodetector; displaying, with a display module, images based on the received light signals having a current image brightness; monitoring the received light signals for changes to the current image brightness; and adjusting a light source illuminance level, and in some embodiments a photodetector gain and exposure time, based on the current image brightness to maintain a target image brightness of the display module. In an embodiment, a brightness adjustment step size is less than a just-noticeable difference in brightness of an eye and is directly proportional to the current image brightness.

Method for adjusting an exposure of an endoscope

Endoscopic systems, non-transitory, machine-readable storage media, and methods for correcting brightness non-uniformity are described. In an embodiment, the endoscopic system includes a light source positioned to emit illumination light onto a scene; a photodetector positioned to receive illumination light reflected off of the scene and configured to generate a scene signal based on the received illumination light; a display; and a controller operatively coupled to the light source, the photodetector, and the display. In an embodiment, the controller including logic that, when executed by the controller, causes the endoscopic system to perform operations including: illuminating the scene with the light source; detecting a scene depth; estimating a scene-specific brightness non-uniformity correction based on the detected scene depth and an endoscopic system brightness non-uniformity profile; and displaying an image of the scene with the display based on the scene signal, the detected scene depth, and the endoscopic system brightness non-uniformity correction.

Adaptive brightness non-uniformity correction in endoscope visualization

An imaging system and method for imaging a scene using a rolling shutter are described. In an embodiment, the method includes illuminating a scene with first and second illumination light; generating frame signals with a photodetector comprising a plurality of pixels arranged in a plurality of rows, wherein the frame signals are based on light received from the scene with sequentially integrated rows of pixels of the plurality of rows, and wherein a frame signal includes signals from pixels of each of the plurality of rows; and generating images of the scene based on an intensity of the frame signals and the proportion of the first illumination light and the second illumination light emitted onto the scene during the first and second frames, wherein a proportion of the first illumination light and the second illumination light in a first frame is different than in a second frame.

Growing community of inventors

Burlingame, CA, United States of America

Nishant Verma