Facense Ltd.

Other

Affectomatics Ltd.

Active Knowledge Ltd.

Ari M Frank

Gil Thieberger

Arie Tzvieli

Ori Tzvieli

Anat Thieberger Ben-Haim

Tal Thieberger-Navon

Shmuel Jacob Hershkovitz

Systems, methods, and computer programs for efficient use of head-mounted cameras in eye tracking. In one embodiment, a system includes a head-mounted device that takes measurements indicative of positions of an eye of a user, and a head-mounted camera that captures images of the eye. A first range of eye positions trackable from the images is narrower than a second range of eye positions trackable from the measurements. A computer calculates the eye positions based on the measurements. When the eye positions fall within the first range, the computer reads images at a first bitrate from the head-mounted camera. And when the eye positions fall outside the first range, the computer refrains from reading images from the head-mounted camera, or reads the images from the head-mounted camera at a second bitrate that is less than half the first bitrate.

Operating an eye tracker camera according to eye position

Disclosed herein are systems and methods for tracking facial expressions efficiently. In one embodiment, a system includes a non-contact head-mounted electro-optical sensor (e.g., an inward-facing head-mounted camera), which measures reflections from a region on a user's head, which are indicative of facial expressions of the user. The system also includes a computer that detects, based on the reflections, a type of facial expression expressed by the user, which belongs to a group comprising neutral facial expressions and non-neutral facial expressions. The computer uses this detection to determine how to proceed to read the electro-optical sensor: the computer reads the electro-optical sensor at a first average bitrate (b) when the user expresses a facial expression from among the neutral facial expressions, and reads the electro-optical sensor at a second average bitrate (b) when the user expresses a facial expression from among the non-neutral facial expressions, where b>b.

Controlling reading of a head-mounted sensor according to facial expressions

System and method that utilize windowing for efficient capturing of facial landmarks include an inward-facing head-mounted camera that captures images of a region on a user's face utilizing a sensor supporting changing of its region of interest (ROI). The system also includes a computer that detects, based on the images, a type of facial expression expressed by the user, which belongs to a group comprising first and second facial expressions. Responsive to detecting that the user expresses the first facial expression, the computer reads from the camera a first ROI that covers a first subset of facial landmarks relevant to the first facial expression. Responsive to detecting that the user expresses the second facial expression, the computer reads from the camera a second ROI that covers a second subset of facial landmarks relevant to the second facial expression, with the first and second ROIs being different.

Smart windowing for reducing power consumption of a head-mounted camera used for detecting facial expressions

Due to the many interactions that can occur in places of gatherings, such as workplaces, schools, theaters, etc., these locations can be considered dangerous to enter during times of epidemics. It is difficult to keep track of the health state of all the people who visited a location, and thus ascertain if visits to the location pose any risk of contracting a disease. Some embodiments disclosed herein utilize wearable devices that measure physiological signals of their wearer in order to determine whether people who were at a location were healthy, and thus be able to certify the location as contagion-safe.

Wearable-based certification of a premises as contagion-safe

Systems, methods, and computer programs for eye tracking involving a camera that is operated according to eyelid position. In one embodiment, an inward-facing head-mounted camera captures images of an eye. A head-mounted device emits electromagnetic waves towards the eye and measures reflections of these waves from the eye. The average rate of reflection measurements is at least ten times the average rate of image capture. A computer detects the position of at least one of the eyelids covering the eye based on the reflection measurements. And the camera operation is adjusted according to the eyelid position in a manner that decreases, in the captured images, areas depicting the eyelids, and optionally increases areas depicting the iris.

Efficient image capturing based on eyelid position

Systems, methods, and computer programs for eye tracking. In one embodiment, an eye tracking system includes a head-mounted device that takes measurements indicative of a position of the eye of a user. A head-mounted camera captures an image of the eye. A computer calculates the position of the eye based on the measurements, utilizes the position of the eye to crop the image around the pupil, and provides a cropped image to a video-based eye tracker. Optionally, the size of the cropped image is less than a third of the size of its respective uncropped image. Optionally, the head-mounted device includes one or more of the following: a photosensor-oculography device (PSOG), an electrooculography device (EOG), an electromyography device (EMG), an optical flow sensor, and a range sensor. Optionally, the image-based eye tracker calculates, based on the cropped image, at least one of pupil diameter and pupillary response.

Image cropping based on eye position for a video-based eye tracker

Disclosed herein is a doorway that facilitates passage from an inside to an outside, and which includes a barrier that moves between opened and closed positions based on commands sent by a computer. Sensors measure signals indicating whether a first user is on the outside, and whether a second user on the inside. The computer detects that the first user is on the outside and receives, from a device of the first user, an indication indicating the first user is healthy. The computer detects whether the second user is on the inside, and commands the barrier to move to the opened position if: (i) the second user is not on the inside, or (ii) the second user is on the inside and the computer receives, from a device of the second user, an indication indicating the second user is healthy.

Doorway system that utilizes wearable-based health state verifications

Embodiments disclosed herein utilize biometric verification from wearable devices to approve access for shared spaces. Requests to occupy the shared spaces are received and conditional reservations are made pending health checks of the requesters. Prior to occupation, fresh biometric measurements are captured via the user's wearable device, which include physiological signals validating both ongoing good health as well as identity match to the original requester. When recent biometric verification indicates both healthy status and unchanged identity, access to the reserved space is approved. These embodiments securely manage reservations while protecting the health of other occupants through conditional access related to the requester's current health condition.

Reservation booking using wearable-based health state verifications

Described herein are embodiments of systems and methods that utilize images of a user's face to detect alcohol intoxication. One embodiment of a system to detect alcohol intoxication includes first and second inward-facing head-mounted cameras that are located less than 5 cm from a user's face, are sensitive to wavelengths below 1050 nanometer, and are configured to capture images of respective first and second regions on the user's face. The system also includes a computer that calculates, based on baseline images captured with the cameras while the user was not intoxicated, a baseline pattern of hemoglobin concentrations at regions on the face. The computer also calculates, based on a current set of images captured with the cameras, a current pattern of hemoglobin concentrations at the regions, and detects whether the user is intoxicated based on a deviation of the current pattern from the baseline pattern.

Detecting alcohol intoxication from video images

Disclosed herein is photosensor-based detection of facial landmarks that utilizes images in order to detect and account for sensor shifts. In one embodiment, a system that detects positions of facial landmarks includes a head-mounted device comprising: (i) light sources that emit light towards a first region on a user's face, and (ii) discrete photosensors, spread over more than 2 cm, which take measurements of reflections of the light from the first region. The system also includes a head-mounted camera that captures images of a second region on the face. A computer calculates, based on the images, values indicative of a location and/or orientation of the device relative to the face, and detects, based on the reflections and the values, positions of facial landmarks at an average rate higher than an average rate at which the images are captured.

Using a camera to compensate for sensor-shift in a photosensor-based facial expression detector

One embodiment of an unmanned protective vehicle (UPV) includes a chassis, wheels, an engine, a barrier fixed to the chassis, and an autonomous driving system. The UPV has an opening which enables a manned vehicle to enter a space that is protected by the barrier. And the autonomous driving system is configured to drive the UPV in cooperation with the manned vehicle, while the manned vehicle is located inside the space. Having the manned vehicle inside the space of the UPV improves the survivability of the manned vehicle following a collision, while the manned vehicle is located inside the space, compared to the survivability of the manned vehicle following a collision, while the manned vehicle is not located inside the space.

Unmanned protective vehicle for protecting manned vehicles

Software agents collect measurements of affective response and provide them in a selective manner that maintains user privacy. In one embodiment, a sensor takes measurements of affective response of the user. A computer receives a request for an affective value indicative of an emotional response to an experience. The computer selects, from among the measurements, a certain measurement of affective response of the user that corresponds to an event in which the user had the experience. The computer then utilizes a model to calculate, based on the certain measurement, the affective value indicative of the emotional response of the user to experience. Optionally, the model is generated based on previously taken measurements of affective response of the user, taken while the user had various experiences, and indications of emotional responses the user had while said previously taken measurements were taken. The computer sends the affective value to fulfil the request.

Software agents facilitating affective computing applications

Embodiments disclosed herein involve a wearable-based system that can help determine in a privacy-preserving manner whether a user is healthy (and thus poses low risk of contagiousness), and use determinations of this nature to grant passage through a doorway. In one embodiment, a wearable device takes measurements of a user comprising a photoplethysmogram signal (PPG signal) and a temperature measurement. A computer calculates a health score based on a difference between baseline measurements of the user, measured one or more days earlier, and more current measurements of the user. Additionally, the computer calculates an extent of similarity between characteristics of the PPG signal in the current measurements and characteristics of the PPG signal in the baseline measurements. The computer authorizes the user's passage through the doorway responsive to the health score reaching a first threshold and the extent of the similarity reaching a second threshold.

Wearable-based health state verification for physical access authorization

Disclosed herein is utilization of photosensor-oculography along with a video camera to implement a novel form of eye tracking, which can be important for many applications, such as augmented reality (AR) on smartglasses. In one embodiment, an eye tracking system includes a photosensor-oculography device (PSOG) that emits light and takes measurements of reflections of the light from an eye of a user, and a camera that captures images of the eye. A computer calculates values indicative of eye movement velocity (EMV) based on the measurements of the reflections obtained by the PSOG. These values then are used to determine how data is read from the camera, which can save power in some cases: the computer reads data from the camera at a higher bitrate when the values are indicative of the EMV are below a threshold compared to when the values are indicative of the EMV are above the threshold.

Controlling an eye tracking camera according to eye movement velocity

Disclosed herein is robust detection of facial expressions that accounts for interferences on the face. In one embodiment, a system includes a head-mounted device that comprises (i) light sources configured to emit light towards a first region on a user's face, and (ii) discrete photosensors, spread over more than 2 cm, which take measurements of reflections of the light from the first region. Additionally, the system includes a head-mounted camera that captures images of a second region on the face. A computer calculates, based on the images, an extent of presence of hair over a portion of the first region. Optionally, the computer calculates extents of other interferences such as perspiration, applied makeup, and skin infection. The computer then detects facial expressions of the user based on the measurements of the reflections and the calculated extent of the hair and/or other interferences.

Growing community of inventors

Haifa, Israel

Ari M Frank