Verily Life Sciences LLC

Ryan L Robertson

Nishant Verma

Thomas Teisseyre

One example method for scene-adaptive image quality in surgical video includes receiving a first video frame from an endoscope, the first video frame generated from a first raw image captured by an image sensor of the endoscope and processed by an image signal processing ('ISP') pipeline having a plurality of ISP parameters; recognizing, using a trained machine learning (“ML”) model, a first scene type or a first scene feature type based on the first video frame; determining a first set of ISP parameters based on the first scene type or the first scene feature type; applying the first set of ISP parameters to the ISP pipeline; and receiving a second video frame from the endoscope, the second video frame generated from a second raw image captured by the image sensor and processed by the ISP pipeline using the first set of ISP parameters.

Systems and methods for scene-adaptive image quality in surgical video

One example method includes obtaining, from an endoscope and between first and second video frames of a real-time video having a frame rate, a preliminary image of a scene during a surgical procedure, the first image comprising a plurality of pixels, wherein the first and second video frames are consecutive video frames in the real-time video; determining, for each pixel of the plurality of pixels, a depth within the scene; determining first image capture parameters for the scene based on a scene illumination setting and a first set of pixels within a first range of depths in the scene; capturing, between the first and second consecutive output video frames, a first image using the first image capture parameters; determining an illumination correction for a second set of pixels at a second range of depths within the scene; capturing, between the first and second consecutive output video frames, a second image using the illumination correction; generating a composite image based on the first and second images; and outputting the composite image as the second output video frame.

Image signal processing to compensate for scene changes

Methods, endoscopic systems, and non-transitory, machine-readable storage media for adjusting an exposure of an endoscopic system are described. In an embodiment, the methods include emitting light with a light source into a proximal end of an endoscope light pipe; generating received light signals with a photodetector based on light received through the endoscope light pipe by the photodetector; displaying, with a display module, images based on the received light signals having a current image brightness; monitoring the received light signals for changes to the current image brightness; and adjusting a light source illuminance level, and in some embodiments a photodetector gain and exposure time, based on the current image brightness to maintain a target image brightness of the display module. In an embodiment, a brightness adjustment step size is less than a just-noticeable difference in brightness of an eye and is directly proportional to the current image brightness.

Growing community of inventors

Sunnyvale, CA, United States of America

Ryan L Robertson