Facebook, Inc.

Facebook Technologies, LLC

Javier San Agustin Lopez

Martin Henrik Tall

Sebastian Sztuk

Benjamin Antoine Georges Lefaudeux

Rasmus Dahl

Henrik Hegner Tomra Skovsgaard

John Paulin Hansen

Sune Loje

Systems and methods for reducing, with minimal loss, optical sensor data to be conveyed to another system for processing. An eye tracking device, such as a head-mounted display (HMD), includes a sensor and circuitry. The sensor generates image data of an eye. The circuitry receives the image data, and assigns pixels of the image data to a feature region of the eye by comparing pixel values of the pixels to a threshold value. A feature region refers to an eye region of interest for eye tracking, such as a pupil or glint. The circuitry generates encoded image data by apply an encoding algorithm, such as a run-length encoding algorithm or contour encoding algorithm, to the image data for the pixels of the feature region. The circuitry transmits the encoded image data, having a smaller data size than the image data received from the sensor, for gaze contingent content rendering.

System and method for data compressing optical sensor data prior to transferring to a host system

Methods and systems to facilitate eye tracking control are provided. A user input is received at a computing device. Point of regard information associated with a user of the computing device is determined while the user input is being received. The point of regard information indicates a location on a display of the computing device at which the user is looking. An operation associated with a display object identified based on the point of regard information is performed when receipt of the user input is determined to have terminated.

Systems and methods of eye tracking control

A display system divides a screen into regions and applies a different set of rendering/encoding parameters to each region. The system applies a first set of parameters to a first region that is being viewed by a fovea of an eye of a user. The system may also apply a second set of parameters to a second region that is being viewed by a parafovea of the eye, and apply a third set of parameters to a third region that is being viewed by the area of the eye outside of the parafovea. The first set of parameters are selected to yield relatively high image quality, while the second set of parameters are yield intermediate quality, and the third set of parameters yield lower quality. As a result, the second region and the third region can be rendered, encoded, and transmitted with less computing power and less bandwidth.

Adaptive parameters in image regions based on eye tracking information

Gaze information of a user can be determined by a computing device that analyzes images of the user. Gaze information of a user includes information such as the user's line of sight, point of regard information, the direction of the user's gaze, the depth of convergence of the user's gaze, and the like. The computing device is able to estimate the distance from the user at which the user is focusing (for example, at a screen near the user or at an object farther away). The visibility and display characteristics of objects displayed on the HUD may be based on the gaze information. For example, content on a heads-up display (HUD) on a windshield may be more transparent while the user is looking through the windshield and more opaque (or otherwise enhanced) while the user is focusing on the HUD.

Display visibility based on eye convergence

Methods and systems to facilitate eye tracking control on mobile devices are provided. An image of a portion of a user is received at an eye tracking device, where the image includes reflections caused by light emitted on the user from one or more light sources located within the eye tracking device. One or more eye features associated with an eye of the user is detected using the reflections. Point of regard information is determined using the one or more eye features, where the point of regard information indicates a location on a display of a computing device coupled to the eye tracking device at which the user was looking when the image of the portion of the user was taken. The point of regard information is sent to an application capable of performing a subsequent operation using the point of regard information.

Systems and methods of eye tracking control on mobile device

A user of a computing device may interact with and control objects and applications displayed on the computing device through the user's eye movement. Detected gaze locations are correlated with actions performed by the user and compared with typical gaze locations for those actions. Based on differences between the detected and expected gaze locations, the eye tracking system can be recalibrated. An area around a gaze location encompassing a set of likely active locations can be enlarged, effectively prompting the user to interact with the desired active location again. The enlarging of the area serves to separate the active locations on the screen, reducing the probability of interpreting the user's gaze incorrectly.

Dynamic eye tracking calibration

An image of a user's eyes and face may be analyzed using computer-vision algorithms. A computing device may use the image to determine the location of the user's eyes and estimate the direction in which the user is looking. The eye tracking technology may be used in a wide range of lighting conditions and with many different and varying light levels. When a user is near a light source, an automatic exposure feature in the camera may result in the user's face and eyes appearing too dark in the image, possibly reducing the likelihood of face and eye detection. Adjusting attributes such as the camera exposure time and the intensity and illumination interval of the light sources based on motion and light levels may improve detection of a user's features.

Systems and methods of light modulation in eye tracking devices

An image of a user's eyes and/or face, captured by a camera on the computing device or on a device coupled to the computing device, may be analyzed using computer-vision algorithms, such as eye tracking and gaze detection algorithms, to determine the location of the user's eyes and estimate the gaze information associated with the user. A user calibration process may be conducted to calculate calibration parameters associated with the user. These calibration parameters may be taken into account to accurately determine the location of the user's eyes and estimate the location on the display at which the user is looking. The calibration process may include determining a plane on which the user's eyes converge and relating that plane to a plane of a screen on which calibration targets are displayed.

Growing community of inventors

Copenhagen, Denmark

Javier San Agustin Lopez