Dacuda Ag

Magic Leap, Inc.

Ml Netherlands C.v.

Erik Fonseka

Alexander Ilic

Martin Georg Zahnert

Andreas Breitenmoser

Bradley Vincent Stuart

Daniel Farmer

Shiuli Das

Suraj Manjunath Shanbhag

Peter Weigand

Simon Meier

Samuel A Miller

Lionel Ernest Edwin

Edwin Joseph Selker

Ivan Li Chuen Yeoh

Paul M Greco

Jeffrey Scott Sommers

Lomesh Agarwal

Yonatan Munk

Sergey Fyodorovich Prokushkin

Tiejian Zhang

Georg Zahnert

Systems and methods for eye tracking latency enhancements. An example head-mounted system obtains a first image of an eye of a user. The first image is provided as input to a machine learning model which has been trained to generate iris and pupil segmentation data given an image of an eye. A second image of the eye is obtained. A set of locations in the second image at which one or more glints are shown is detected based on iris segmentation data generated for the first image. A region of the second image at which the pupil of the eye of the user is shown is identified based on pupil segmentation data generated for the first image. A pose of the eye of the user is determined based on the detected set of glint locations in the second image and the identified region of the second image.

Eye tracking latency enhancements

A display system includes a head-mounted display configured to project light, having different amounts of wavefront divergence, to an eye of a user to display virtual image content appearing to be disposed at different depth planes. The wavefront divergence may be changed in discrete steps, with the change in steps being triggered based upon whether the user is fixating on a particular depth plane. The display system may be calibrated for switching depth planes for a main user. Upon determining that a guest user is utilizing the system, rather than undergoing a full calibration, the display system may be configured to switch depth planes based on a rough determination of the virtual content that the user is looking at. The virtual content has an associated depth plane and the display system may be configured to switch to the depth plane of that virtual content.

Depth plane selection for multi-depth plane display systems by user categorization

Forming a 3D representation of an object using a portable electronic device with a camera. A sequence of image frames, captured with the camera may be processed to generate 3D information about the object. This 3D information may be used to present a visual representation of the object as real-time feedback to a user, indicating confidence of 3D information for regions of the object. Feedback may be a composite image based on surfaces in a 3D model derived from the 3D information given visual characteristics derived from the image frames. The 3D information may be used in a number of ways, including as an input to a 3D printer or an input, representing, in whole or in part, an avatar for a game or other purposes. To enable processing to be performed on a portable device, a portion of the image frames may be selected for processing with higher resolution.

Method and system for real-time 3D capture and live feedback with monocular cameras

A smartphone may be freely moved in three dimensions as it captures a stream of images of an object. Multiple image frames may be captured in different orientations and distances from the object and combined into a composite image representing an image of the object. The image frames may be formed into the composite image based on representing features of each image frame as a set of points in a three dimensional point cloud. Inconsistencies between the image frames may be adjusted when projecting respective points in the point cloud into the composite image. Quality of the image frames may be improved by processing the image frames to correct errors. Distracting features, such as the finger of a user holding the object being scanned, can be replaced with background content. As the scan progresses, a direction for capturing subsequent image frames is provided to a user as a real-time feedback.

User feedback for real-time checking and improving quality of scanned image

A computer peripheral that may operate as a scanner. The scanner captures image frames as it is moved across an object. The image frames are formed into a composite image based on computations in two processes. In a first process, fast track processing determines a coarse position of each of the image frames based on a relative position between each successive image frame and a respective preceding image determine by matching overlapping portions of the image frames. In a second process, fine position adjustments are computed to reduce inconsistencies from determining positions of image frames based on relative positions to multiple prior image frames. The peripheral may also act as a mouse and may be configured with one or more navigation sensors that can be used to reduce processing time required to match a successive image frame to a preceding image frame.

Image processing for handheld scanner

A computer peripheral that may operate as a scanner. The scanner captures image frames as it is moved across an object. The image frames are formed into a composite image based on computations in two processes. In a first process, fast track processing determines a coarse position of each of the image frames based on a relative position between each successive image frame and a respective preceding image determine by matching overlapping portions of the image frames. In a second process, fine position adjustments are computed to reduce inconsistencies from determining positions of image frames based on relative positions to multiple prior image frames. As a result, a composite image of an object being scanned may be presented in real time to a user, providing a user feedback or portions of the object that have been scanned and those that have not.

Real-time display of images acquired by a handheld scanner

A computer peripheral that may operate as a scanner. The scanner captures image frames as it is moved across an object. The image frames are formed into a composite image based on computations in two processes. In a first process, fast track processing determines a coarse position of each of the image frames based on a relative position between each successive image frame and a respective preceding image determine by matching overlapping portions of the image frames. In a second process, fine position adjustments are computed to reduce inconsistencies from determining positions of image frames based on relative positions to multiple prior image frames. As additional image frames are added to the composite image, the size and format of the composite image may be automatically adjusted to facilitate ease of use.

Automatic sizing of images acquired by a handheld scanner

A computer peripheral that may operate as a hand-held scanner and, in some embodiments, also as a conventional computer mouse. The components incorporated in the device to support scanning may be compact and may be positioned to allow flexibility in shaping a housing of the device. As a result, the device may be ergonomically shaped to facilitate hand-held use as either a scanner or a mouse. Controls, for mouse or scanner operation, may be readily incorporated into the device. The components may also be low cost. Such a low cost, compact design may be achieved through an appropriate selection of curved reflective surfaces in optical paths through the housing. The curved reflective surface may selectively focus or spread light traveling along an optical path, reducing or eliminating the need for lenses and may enable light emitting elements and an image array to be mounted on the same printed circuit board.

Computer peripheral for scanning

A computer peripheral that may operate as a scanner. The scanner captures image frames as it is moved across an object. The image frames are formed into a composite image based on computations in two processes. In a first process, fast track processing determines a coarse position of each of the image frames based on a relative position between each successive image frame and a respective preceding image determine by matching overlapping portions of the image frames. In a second process, fine position adjustments are computed to reduce inconsistencies from determining positions of image frames based on relative positions to multiple prior image frames. Processing of the image frames may allow a scan of an object to be suspended and resumed automatically when a user lifts the scanner, by deleting out of focus image frames and matching subsequent image frames to previously stored image frames.

Continuous scanning with a handheld scanner

A computer peripheral that may operate as a scanner. The scanner captures image frames as it is moved across an object. The image frames are formed into a composite image based on computations in two processes. In a first process, fast track processing determines a coarse position of each of the image frames based on a relative position between each successive image frame and a respective preceding image determine by matching overlapping portions of the image frames. In a second process, fine position adjustments are computed to reduce inconsistencies from determining positions of image frames based on relative positions to multiple prior image frames. When the determined position of multiple image frames overlap, pixels in the composite image may be formed by combing values of pixels in the image frames, which improves image quality of the composite image.

Handheld scanner with high image quality

A computer peripheral that operates either as a computer mouse or as a scanner. The peripheral includes navigation sensors that generate information on motion of the device and an image array that captures successive image frames of an object being scanned. In a mouse mode, the peripheral periodically transfers readings from the navigation sensors to a computing device so that the computing device can track a position of the device. In a scanner mode, in addition to obtaining navigation information from the navigation sensors, the peripheral also captures image frames as it is moved across the object. Operation of the navigation sensors and image array may be synchronized such that an association between the image data and the navigation information may be generated and maintained as image frames are transferred to the computing device, even if some of the frames are dropped in transmission between the scanner-mouse and a computer.

Growing community of inventors

Zurich, Switzerland

Erik Fonseka