Snap Inc.

Microsoft Technology Licensing, LLC

Microsoft International Holdings B.v.

Sagi Katz

Giora Yahav

Avishai Adler

David Ben Ezra

Ilya Levin

Ashutosh Yugesh Shukla

Matan Zohar

John B Ardisana, Ii

David Cohen

Mathias Andreas Hintermann

Eyal Zak

Nir Daube

Michael Ryner

Nati Kligler

Daniel Wagner

Guy Gilboa

Weston Welge

Guy Melamed

Gilad Refael

Netanel Kligler

John Allen Tardif

Matthias Kalkgruber

Mattias Marder

Shlomo Felzenshtein

Avi Bar-Zeev

Georgios Evangelidis

Branislav Micusik

Eli Larry

Dani Daniel

Zvika Klier

Katie Stone Perez

Zvi Klier

A method for generating reference biometric data based on a bending of a flexible device is described. In one aspect, a method includes forming training data includes bending estimates of a flexible device worn by a first user, training a model based on the training data, and generating reference biometric data for the first user based on the model.

Bending estimation as a biometric signal

A photo filter (e.g., artistic/stylized painting) light field effect system includes an eyewear device having a frame, a temple connected to a lateral side of the frame, and a depth-capturing camera. Execution of programming by a processor configures the stylized image painting effect system to apply a photo filter selection to: (i) a left raw image or a left processed image to create a left photo filter image, and (ii) a right raw image or a right processed image to create a right photo filter image. The stylized image painting effect system generates a photo filter stylized painting effect image with an appearance of a spatial rotation or movement, by blending together the left photo filter image and the right photo filter image based on a left image disparity map and a right image disparity map.

Stylized image painting

A virtual wave creation system comprises an eyewear device that includes a frame, a temple connected to a lateral side of the frame, and a depth-capturing camera. Execution of programming by a processor configures the virtual wave creation system to generate, for each of multiple initial depth images, a respective wave image by applying a transformation function that is responsive to a selected input parameter to the initial three-dimensional coordinates. The virtual wave creation system creates a warped wave video including a sequence of the generated warped wave images. The virtual wave creation system presents, via an image display, the warped wave video.

Input parameter based image waves

A method for increasing a dynamic range of a dual-pixel image sensor is described. The method includes detecting an intensity level of a full pixel from a plurality of pixels of an optical sensor, one or more full pixels of the plurality of pixels includes at least two sub-pixels, detecting an intensity level of one or more sub-pixels, detecting that the intensity level of the full pixel of the optical sensor has reached a saturation level of the full pixel, and in response to detecting that the intensity level of the full pixel of the optical sensor has reached the saturation level of the full pixel, computing an extrapolated intensity level of the full pixel based on the intensity level of the one or more sub-pixels.

High dynamic range for dual pixel sensors

Disclosed are systems, methods, and non-transitory computer-readable media for varied depth determination using stereo vision and phase detection auto focus (PDAF). Computer stereo vision (stereo vision) is used to extract three-dimensional information from digital images. To utilize stereo vison, two optical sensors are displaced horizontally from one another and used to capture images depicting two differing views of a real-world environment from two different vantage points. The relative depth of the objects captured in the images is determined using triangulation by comparing the relative positions of the objects in the two images. For example, the relative positions of matching objects (e.g., features) identified in the captured images are used along with the known orientation of the optical sensors (e.g., distance between the optical sensors, vantage points the optical sensors) to estimate the depth of the objects.

Varied depth determination using stereo vision and phase detection auto focus (PDAF)

A method for correcting a bending of a flexible device is described. In one aspect, the method includes accessing feature data of a first stereo frame that is generated by stereo optical sensors of the flexible device, the feature data generated based on a visual-inertial odometry (VIO) system of the flexible device, accessing depth map data of the first stereo frame, the depth map data generated based on a depth map system of the flexible device, estimating a pitch-roll bias and a yaw bias based on the features data and the depth map data of the first stereo frame, and generating a second stereo frame after the first stereo frame, the second stereo frame based on the pitch-roll bias and the yaw bias of the first stereo frame.

Depth-from-stereo bending correction using visual inertial odometry features

A wallpaper system presents a first wallpaper image of a wallpaper video to a user and receives, via a user input device, one or both of: (i) a spatial user input selection, and (ii) a time user input selection from the user to apply to the wallpaper video. In response to detecting one or both of: (i) the spatial user input selection, and (ii) the time user input selection, the wallpaper system determines one or both of: (i) a respective spatial movement parameter within a wallpaper video associated with the spatial user input selection, and (ii) a respective time coordinate within the wallpaper video associated with the time user input selection. Wallpaper system presents, via the image display, a second wallpaper image associated with one or both of: (i) the respective spatial movement parameter, and (ii) the respective time coordinate.

Creation and user interactions with three-dimensional wallpaper on computing devices

A method for AR-guided depth estimation is described. The method includes identifying a virtual object rendered in a first frame that is generated based on a first pose of an augmented reality (AR) device, determining a second pose of the AR device, the second pose following the first pose, identifying an augmentation area in the second frame based on the virtual object rendered in the first frame, and the second pose, determining depth information for the augmentation area in the second frame, and rendering the virtual object in the second frame based on the depth information.

Augmented reality guided depth estimation

A photo filter (e.g., artistic) light field effect system comprises an eyewear device that includes a frame, a temple connected to a lateral side of the frame, and a depth-capturing camera. Execution of programming by a processor configures the photo filter light field effect system to apply a photo filter selection to: (i) a left raw image or a left processed image to create a left photo filter image, and (ii) a right raw image or a right processed image to create a right photo filter image. The photo filter light field effect system generates, a photo filter light field effect image with an appearance of a spatial rotation or movement, by blending together the left photo filter image and the right photo filter image based on a left image disparity map and a right image disparity map.

Artistic effects for images and videos

Three-dimensional image calibration and presentation for eyewear including a pair of image capture devices is described. Calibration and presentation includes obtaining a calibration offset to accommodate flexure in the support structure for the eyewear, adjusting a three-dimensional rendering offset by the obtained calibration offset, and presenting the stereoscopic images using the three-dimension rendering offset.

Flexible eyewear device with dual cameras for generating stereoscopic images

Systems and method for correcting images including artifacts due to dirty camera lenses of electronic device are disclosed. Correction of images by the systems and methods includes obtaining a first raw pixel image of a scene captured with a first camera, obtaining a second raw image of the scene captured with a second camera separate from the first camera in a camera baseline direction, rectifying the first and second raw pixel images to create respective first and second rectified pixel images, determining disparity correspondence between corresponding image pixel pairs of the first and second rectified images in the camera baseline direction, mapping first and second rectified images into the same domain using the determined disparity, detect image artifact regions within each domain mapped image by comparing corresponding regions of the domain mapped images, determining correction factors for each detected image artifact region, and correcting the rectified first and second images by applying the determined correction factors.

Dirty lens image correction

Systems and methods for determining disparity between two images are disclosed. Such systems and methods include obtaining a first pixel image of a scene from a first viewpoint, obtaining a second pixel image of the scene from a second viewpoint (e.g., separate from the first viewpoint in a camera baseline direction such as horizontal or vertical), modifying the first and second pixel images using component-separated correction to create respective first and second corrected pixel images maintaining pixel scene correspondence in the camera baseline direction from between the first and second pixel images to between the first and second corrected pixel images, determining pixel pairs from corresponding pixels between the first and second corrected pixel images in the camera baseline direction, and determining disparity correspondence for each of the determined pixel pairs from pixel locations in the first and second pixel images corresponding to respective pixel locations of the pixel pairs in the first and second corrected pixel images.

Separable distortion disparity determination

Disclosed are systems, methods, and non-transitory computer-readable media for varied depth determination using, stereo vision and phase detection auto focus (PDAF). Computer stereo vision (stereo vision) is used to extract three-dimensional information from digital images. To utilize stereo vison, two optical sensors are displaced horizontally from one another and used to capture images depicting two differing views of a real-world environment from two different vantage points. The relative depth of the objects captured in the images is determined using triangulation by comparing the relative positions of the objects in the two images. For example, the relative positions of matching objects (e.g., features) identified in the captured images are used along with the known orientation of the optical sensors (e.g., distance between the optical sensors, vantage points the optical sensors) to estimate the depth of the objects.

Growing community of inventors

Yokneam Ilit, Israel

Sagi Katz