Tobii Ab

Tobil Ab

Erik Lindén

Mark Ryan

Pravin Kumar Rana

Simon Johansson

Rickard Lundahl

Jonas Sjöstrand

Maria Gordon

Geoffrey Cooper

David Mohlin

Computer-generated image data is presented on first and second displays of a binocular headset presuming that a user's left and right eyes are located at first and second positions relative to the first and second displays respectively. At least one updated version of the image data is presented, which is rendered presuming that at least one of the user's left and right eyes is located at a position different from the first and second positions respectively in at least one spatial dimension. In response thereto, a user-generated feedback signal is received expressing either: a quality measure of the updated version of the computer-generated image data relative to computer-generated image data presented previously; or a confirmation command. The steps of presenting the updated version of the computer-generated image data and receiving the user-generated feedback signal are repeated until the confirmation command is received. The first and second positions are defined based on the user-generated feedback signal.

Method, computer program product and binocular headset controller

The present invention relates to a method for establishing the position of an object in relation to a camera in order to enable gaze tracking with a user watching the object, where the user is in view of the camera. The method comprises the steps of showing a known pattern, consisting of a set of stimulus points (s, s, . . . , s), on the object, detecting gaze rays (g, g, . . . , g) from an eye of the user as the user looks at the stimulus points (s, s, . . . , s), and finding, by means of an optimizer, a position and orientation of the object in relation to the camera such that the gaze rays (g, g, . . . , g) approaches the stimulus points (s, s, . . . , s).

Screen estimation

A method of updating a cornea model for a cornea of an eye is disclosed, as well as a corresponding system and storage medium. The method comprises controlling a display to display a stimulus at a first depth, wherein the display is capable of displaying objects at different depths, receiving first sensor data obtained by an eye tracking sensor while the stimulus is displayed at the first depth by the display, controlling the display to display a stimulus at a second depth, wherein the second depth is different than the first depth, receiving second sensor data obtained by the eye tracking sensor while the stimulus is displayed at the second depth by the display, and updating the cornea model based on the first sensor data and the second sensor data.

Updating a cornea model

A gaze tracking model is adapted to predict a gaze ray using an image of the eye. The model is trained using training data which comprises a first image of an eye, reference gaze data indicating a gaze point towards which the eye was gazing when the first image was captured, and images of an eye captured by first and second cameras at a point in time. The training comprises forming a distance between the gaze point and a gaze ray predicted by the model using the first image, forming a consistency measure based on a gaze ray predicted by the model using the image captured by the first camera and a gaze ray predicted by the model using the image captured by the second camera, forming an objective function based on at least the formed distance and the consistency measure, and training the model using the objective function.

Training of a gaze tracking model

A preliminary path for light travelling towards a camera via corneal reflection is estimated based on a preliminary position and orientation of an eye. A position where the reflection would appear in images captured by the camera is estimated. A distance is formed between a detected position of a corneal reflection of an illuminator and the estimated position. A second preliminary path for light travelling through the cornea or from the sclera towards a camera is estimated based on the preliminary position and orientation, and a position where the second preliminary path would appear to originate in images captured by this camera is estimated. A distance is formed between a detected edge of a pupil or iris and the estimated position where the second preliminary path would appear to originate. An updated position and/or orientation of the eye is determined using an objective function formed based on the formed distances.

Gaze tracking via tracing of light paths

A method for forming an offset model is described. The offset model represents an estimated offset between a limbus center of a user eye and a pupil center of the user eye as a function of pupil size. The approach includes sampling a set of limbus center values, sampling a set of pupil center values, and sampling a set of radius values. The offset model is formed by comparing a difference between the set of limbus center values and the set of pupil center values at each of the radius values. A system and a computer-readable storage device configured to perform such a method are also disclosed.

Continuous calibration based on pupil characteristics

A method is disclosed, comprising obtaining a first angular offset between a first eye direction and a first gaze direction of an eye having a first pupil size, obtaining a second angular offset between a second eye direction and a second gaze direction of the eye having a second pupil size, and forming, based on the first angular offset and the second angular offset, a compensation model describing an estimated angular offset as a function of pupil size. A system and a device comprising a circuitry configured to perform such a method are also disclosed.

Pupil radius compensation

Techniques for generating 3D gaze predictions based on a deep learning system are described. In an example, the deep learning system includes a neural network. The neural network is trained with training images generated by cameras and showing eyes of user while gazing at stimulus points. Some of the stimulus points are in the planes of the camera. Remaining stimulus points are not un the planes of the cameras. The training includes inputting a first training image associated with a stimulus point in a camera plane and inputting a second training image associated with a stimulus point outside the camera plane. The training minimizes a loss function of the neural network based on a distance between at least one of the stimulus points and a gaze line.

Training of a neural network for three dimensional (3D) gaze prediction

Techniques for generating 3D gaze predictions based on a deep learning system are described. In an example, the deep learning system includes a neural network. A scaled image is generated from 2D image showing a user face based on a rough distance between the user eyes and a camera that generated the 2D image. Image crops at different resolutions are generated from the scaled image and include a crop around each of the user eyes and a crop around the user face. These crops are input to the neural network. In response, the neural network outputs a distance correction and a 2D gaze vector per user eye. A corrected eye-to-camera distance is generated by correcting the rough distance based on the distance correction. A 3D gaze vector for each of the user eyes is generated based on the corresponding 2D gaze vector and the corrected distance.

Deep learning for three dimensional (3D) gaze prediction

Techniques for generating 3D gaze predictions based on a deep learning system are described. In an example, the deep learning system includes a neural network. The neural network is trained with training images. During the training, calibration parameters are initialized and input to the neural network, and are updated through the training. Accordingly, the network parameters of the neural network are updated based in part on the calibration parameters. Upon completion of the training, the neural network is calibrated for a user. This calibration includes initializing and inputting the calibration parameters along with calibration images showing an eye of the user to the neural network. The calibration includes updating the calibration parameters without changing the network parameters by minimizing the loss function of the neural network based on the calibration images. Upon completion of the calibration, the neural network is used to generate 3D gaze information for the user.

Growing community of inventors

Danderyd, Sweden

Erik Lindén