Tectus Corporation

Ramin Mirjalili

Michael West Wiemer

Abhishek Deepak Bhat

Joseph Czompo

Thomas Llewellyn Owens

Dominic Philip Haine

Erik E Anderson

Hawk Yin Pang

Ben Rafael Kimel Green

Richard Gioscia

Brian Elliot Lemoff

Renaldi Winoto

Sheng Wan

Morrison Ulman

Scott Herz

Jieyang Jia

A system includes an electronic contact lens that can detect eye gestures for initiating various actions. The electronic contact lens includes integrated sensors for obtaining sensor measurements characterizing eye motion. The sensor measurements are processed to detect gestures mapped to specific actions such as changing a power state of the electronic contact lens, activating or deactivating a user interface or other feature, or selecting an item from a virtual menu. The eye gestures may involve the user sequentially performing a first saccade quickly followed by a second saccade in an opposite direction from the first saccade.

Control of an electronic contact lens using eye gestures

A system includes an electronic contact lens that obtains sensor measurements from integrated motion sensors or other types of sensors and a processing module that estimates a mental state of an individual based on the sensor measurements. The processing module identifies patterns of eye movements and analyzes how these patterns change over time. Based on anatomical relationships between eye movement and mental state, the processing module estimates characteristics of the individual such as fatigue, intoxication, injury, or a medical condition that have known effects on eye movement patterns. The electronic contact lens system generates an output indicative of the estimated mental state to alert the individual to the detected condition or to initiate an automated action.

Estimating a mental state based on sensor measurements from an electronic contact lens

A system includes a pair of electronic contact lenses that obtain respective motion sensor measurements in response to eye movements. A tracking module derives estimated orientations for both eyes based on the sensor measurements and a set of correlations and constraints that describe human eye movement. The model describes the limited number of ways that an individual eye can move and relationships between relative movement of the left and right eye. The tracking module performs filtering based on the measurements and the eye model to suppress noise and generate orientation estimates for both eyes.

Two-eye tracking based on measurements from a pair of electronic contact lenses

A system includes an electronic contact lens that can detect eye gestures for initiating various actions. The electronic contact lens includes integrated sensors for obtaining sensor measurements characterizing eye motion. The sensor measurements are processed to detect gestures mapped to specific actions such as changing a power state of the electronic contact lens, activating or deactivating a user interface or other feature, or selecting an item from a virtual menu. The eye gestures may involve the user sequentially stabilizing at a starting pitch, executing a first motion that crosses a first pitch threshold, executing a second motion that crosses a second pitch threshold in an opposite direction from the starting pitch, and stabilizing at an ending pitch.

Control of an electronic contact lens using pitch-based eye gestures

An augmented reality system determines the position and orientation of an eye. The system includes an electronic contact lens that projects images onto a user's retina. The contact lens includes magnetic sensors. The magnetic sensors detect magnetic fields along one axis, or more than one axis, depending on their configuration. The sensors may be a conductive coil, a solenoid, or a tunneling magnetoresistance device. The sensors detect magnetic fields generated by magnetic sources. The magnetic sources may be collocated, or non-collocated, on a wearable device, a device in the environment, or a secondary electronic device. The sources may have different orientations such that they produce magnetic fields along different axes, and the sensors are configured to independently detect the magnetic fields. The system determines the pose of the eye using a combination of the measurements, and the position and orientation of the sensors and sources.

Contact lens-based eye tracking using magnetic fields

Operation of an electronic contact lens takes into account saccadic motion of the eye and reduced visual perception during saccades (saccadic suppression). The user's eye motion is tracked, and onset of a saccade is detected based on the eye's motion. For example, saccades may be detected when the eye's acceleration or jerk exceeds a threshold. The endpoint of the saccade is then predicted in real-time while the saccade is still occurring. This may be the temporal endpoint (i.e., when the saccade ends) and/or the positional endpoint (i.e., the eye position at the end of the saccade). Operation of the electronic contact lens is adjusted based on the predicted endpoint.

Saccade detection and endpoint prediction for electronic contact lenses, with adjusted operation

An augmented reality system recognizes objects in a user's environment and operates an electronic contact lens based on the recognition. The electronic contact lens includes an integrated femtoimager that captures images corresponding to the user's gaze direction. The augmented reality system recognizes objects in the images and generates visual information relevant to the recognized objects that is presented using a femtoprojector integrated with the electronic contact lens. The visual information may include virtual control elements that the user can interact with to control smart devices. The augmented reality system can also configure various calibration parameters of the electronic contact lens based on a recognized environment associated with the recognized objects.

Operating an electronic contact lens based on recognized objects in captured images

Saccade detection and endpoint prediction for electronic contact lenses

An imaging device contained in a contact lens captures images of the external environment, which for convenience will be referred to as real-world images. These real-world images are used to stabilize images produced by a femtoprojector also in the contact lens. For convenience, the images produced by the femtoprojector will be referred to as augmented reality or AR images. The femtoprojector is inward-facing (i.e., facing towards the interior of the eye) and projects the AR images onto the user's retina, creating the appearance of virtual images in the external environment. The imaging device, referred to as a femtoimager for convenience, is outward-facing and captures a sequence of actual real-world images of the external environment. Because the femtoimager and femtoprojector move together, the real-world images captured by the femtoimager reflect the motion of the virtual AR images from the femtoprojector relative to the external environment.

Stabilizing images produced by eye-mounted displays, based on images of the external environment captured by eye-mounted imaging devices

An augmented reality system can include an electronic contact lens and a power source. The source generates a time-varying magnetic field which induces a time-varying current in conductive coils embedded in the electronic contact lens. The electronic contact lens uses the induced current to harvest power or to determine the orientation of the contact lens. The embedded conductive coils are positioned such that the AR system can harvest power and estimate the orientation of the eye at a variety of contact lens orientations. The conductive coils may be embedded within the contact lens at any number of positions and orientations. The embedded coils can encircle a portion of the contact lens and can collectively form an annulus within the contact lens. The conductive coils are embedded such that for at least three conductive coils, no two of the planes defined by the at least three conductive coils are parallel.

Eye-mounted displays including embedded conductive coils

In one approach to eye tracking, a contact lens contains a network of twelve accelerometers. The accelerometers are positioned within the contact lens so that the measurements of acceleration can be used to estimate a position and an orientation of the eye relative to an external reference frame. One advantage of accelerometers is that they can be made relatively small and do not require much power. However, because the contact lens has a curved shape and is relatively thin, the possible locations for the accelerometers are limited. Various geometries for the accelerometer network and approaches to optimizing these geometries are described.

Contact lens-based eye tracking

Humans may exhibit characteristic patterns of eye movements when looking at specific objects. For example, when a person looks at the face of another person, their eyes exhibit a certain pattern of movements and saccades as they look at the face. An electronic contact lens includes eye tracking sensors and an outward looking imaging system that may capture images of the user's environment. When the eye tracking sensors detect the pattern of eye movements characteristic of looking at a face, the imaging system becomes active and captures images and performs facial recognition to identify the face using the captured images. The results of the facial recognition may be displayed to the user using a projector of the electronic contact lens.

Contextual awareness based on eye motion tracking by an eye-mounted system

A transmitter coil inductively couples to a receiver coil contained in a contact lens. In one approach, the transmitter coil is contained in a headgear, for example a head band. When the user wears the headgear, the transmitter coil is positioned on a side of the user's head and between the user's ear and the user's eye opening. In one implementation, a head band loops from one ear behind the user's head to the other ear, and also extends slightly forward of each ear. The transmitter coil(s) may be located in the portion of the headband that extends forward of each ear. This places the transmitter coil close to the receiver coil, typically within 40-50 mm of the user's eye opening, while still maintaining an unobtrusive aesthetic.

Headgear providing inductive coupling to a contact lens

Presented are eye-controlled user-machine interaction systems and methods that, based on input variables that comprise orientation and motion of an electronic contact lens, assist the wearer of the contact lens carrying a femtoprojector to control and navigate a virtual scene that may be superimposed onto the real-world environment. Various embodiments provide for smooth, intuitive, and naturally flowing eye-controlled, interactive operations between the wearer and a virtual environment. In certain embodiments, eye motion information is used to wake a smart electronic contact lens, activate tools in a virtual scene, or any combination thereof without the need for blinking, winking, hand gestures, and use of buttons.

Growing community of inventors

San Jose, CA, United States of America

Ramin Mirjalili