Qualcomm Incorporated

Tectus Corporation

Joseph Czompo

Ramin Mirjalili

Wyatt Thomas Riley

William J Morrison

Michael West Wiemer

Benjamin A Werner

Abhishek Deepak Bhat

Victor Kulik

Guttorm Ringstad Opshaug

Ning Luo

Ming Sun

Erik E Anderson

Zoltan F Biacs

Cristina A Seibert

Ashutosh Joshi

Carlos Manuel Puig

Vesa Veikko Ruuska

Tao Jia

Dominic Gerard Farmer

Jie Wu

Russel A Martin

Ashok Bhatia

Brian Elliot Lemoff

Renaldi Winoto

Ara Bicakci

Kumar Gaurav Chhokra

Thomas Llewellyn Owens

Disha Ahuja

Igor Tchertkov

Mayur Nitinbhai Shah

Dominic Philip Haine

Vidyut Naware

Shanta Pavan Namburi

Scott Herz

Shashank Narayanan

Hithesh Reddivari

Jong Bin D Lee

Manish Kushwaha

Jieyang Jia

An augmented reality system determines the position and orientation of an eye. The system includes an electronic contact lens that projects images onto a user's retina. The contact lens includes magnetic sensors. The magnetic sensors detect magnetic fields along one axis, or more than one axis, depending on their configuration. The sensors may be a conductive coil, a solenoid, or a tunneling magnetoresistance device. The sensors detect magnetic fields generated by magnetic sources. The magnetic sources may be collocated, or non-collocated, on a wearable device, a device in the environment, or a secondary electronic device. The sources may have different orientations such that they produce magnetic fields along different axes, and the sensors are configured to independently detect the magnetic fields. The system determines the pose of the eye using a combination of the measurements, and the position and orientation of the sensors and sources.

Contact lens-based eye tracking using magnetic fields

Operation of an electronic contact lens takes into account saccadic motion of the eye and reduced visual perception during saccades (saccadic suppression). The user's eye motion is tracked, and onset of a saccade is detected based on the eye's motion. For example, saccades may be detected when the eye's acceleration or jerk exceeds a threshold. The endpoint of the saccade is then predicted in real-time while the saccade is still occurring. This may be the temporal endpoint (i.e., when the saccade ends) and/or the positional endpoint (i.e., the eye position at the end of the saccade). Operation of the electronic contact lens is adjusted based on the predicted endpoint.

Saccade detection and endpoint prediction for electronic contact lenses, with adjusted operation

An augmented reality system recognizes objects in a user's environment and operates an electronic contact lens based on the recognition. The electronic contact lens includes an integrated femtoimager that captures images corresponding to the user's gaze direction. The augmented reality system recognizes objects in the images and generates visual information relevant to the recognized objects that is presented using a femtoprojector integrated with the electronic contact lens. The visual information may include virtual control elements that the user can interact with to control smart devices. The augmented reality system can also configure various calibration parameters of the electronic contact lens based on a recognized environment associated with the recognized objects.

Operating an electronic contact lens based on recognized objects in captured images

Saccade detection and endpoint prediction for electronic contact lenses

An imaging device contained in a contact lens captures images of the external environment, which for convenience will be referred to as real-world images. These real-world images are used to stabilize images produced by a femtoprojector also in the contact lens. For convenience, the images produced by the femtoprojector will be referred to as augmented reality or AR images. The femtoprojector is inward-facing (i.e., facing towards the interior of the eye) and projects the AR images onto the user's retina, creating the appearance of virtual images in the external environment. The imaging device, referred to as a femtoimager for convenience, is outward-facing and captures a sequence of actual real-world images of the external environment. Because the femtoimager and femtoprojector move together, the real-world images captured by the femtoimager reflect the motion of the virtual AR images from the femtoprojector relative to the external environment.

Stabilizing images produced by eye-mounted displays, based on images of the external environment captured by eye-mounted imaging devices

In one approach to eye tracking, a contact lens contains a network of twelve accelerometers. The accelerometers are positioned within the contact lens so that the measurements of acceleration can be used to estimate a position and an orientation of the eye relative to an external reference frame. One advantage of accelerometers is that they can be made relatively small and do not require much power. However, because the contact lens has a curved shape and is relatively thin, the possible locations for the accelerometers are limited. Various geometries for the accelerometer network and approaches to optimizing these geometries are described.

Contact lens-based eye tracking

Humans may exhibit characteristic patterns of eye movements when looking at specific objects. For example, when a person looks at the face of another person, their eyes exhibit a certain pattern of movements and saccades as they look at the face. An electronic contact lens includes eye tracking sensors and an outward looking imaging system that may capture images of the user's environment. When the eye tracking sensors detect the pattern of eye movements characteristic of looking at a face, the imaging system becomes active and captures images and performs facial recognition to identify the face using the captured images. The results of the facial recognition may be displayed to the user using a projector of the electronic contact lens.

Contextual awareness based on eye motion tracking by an eye-mounted system

Techniques are provided which may be implemented using various methods and/or apparatuses in a mobile GNSS device to compensate for arm swing. An example of an method for compensating for arm swing according to the disclosure includes determining an arm swing signal, such that the arm swing signal is approximately sinusoidal with a period of approximately T seconds, determining a position signal measurement period, receiving a plurality of positioning signals at intervals corresponding to the position signal measurement period, and determining current position information based on the plurality of positioning signals.

Arm swing compensation techniques

Presented are eye-controlled user-machine interaction systems and methods that, based on input variables that comprise orientation and motion of an electronic contact lens, assist the wearer of the contact lens carrying a femtoprojector to control and navigate a virtual scene that may be superimposed onto the real-world environment. Various embodiments provide for smooth, intuitive, and naturally flowing eye-controlled, interactive operations between the wearer and a virtual environment. In certain embodiments, eye motion information is used to wake a smart electronic contact lens, activate tools in a virtual scene, or any combination thereof without the need for blinking, winking, hand gestures, and use of buttons.

Eye-based activation and tool selection systems and methods

Disclosed embodiments pertain to cardiovascular parameter (e.g. heart rate) measurements when motion is present. Biometric sensor signal measurements may be obtained based on cardiovascular parameters of a user; and motion sensor signal measurements may be obtained based on user motion. An activity type may be determined based on the motion sensor signals. For example, when non-motion related frequencies in a frequency domain representation of the biometric sensor signal are obscured by user motion, an activity type may be determined based on the motion sensor signals. Further, based on the activity type, for each cardiovascular parameter (e.g. heart rate), a corresponding likely cardiovascular parameter value (e.g. a likely heart rate) may be determined. A corresponding fundamental frequency associated with the biometric sensor signal may then be determined for each cardiovascular parameter based on the motion sensor signal measurements and the corresponding likely cardiovascular parameter value.

Cardiovascular parameter estimation in the presence of motion

Embodiments implement a device having a sensor element, where different data streams created as part of a sensor module integrated with the sensor element may create multiple sensor data streams from a single sensor element, and may concurrently convey information from the sensor element to respective different applications having different data parameter requirements such that the data streams each match the parameter requirements of the different applications.

Sensor with concurrent data streaming using various parameters

Techniques for compensating for inertial and/or magnetic interference in a mobile device are provided. The mobile device can include a vibration motor to vibrate the device, a processor, and can include an inertial sensor and/or a magnetometer. The processor can be configured to actuate the vibration motor to induce vibration of the mobile device, to measure motion of the mobile device with the inertial sensor of the device to produce sensor output data and/or to measure a magnetic field generated by the vibration motor to produce magnetometer output data, and to compensate for the vibration of the inertial sensor induced by the vibration motor to produce compensated sensor output data and/or to compensate for a magnetic field generated by the vibration motor when the vibration motor is actuated to produce compensated magnetometer output data.

Device driven inertial interference compensation

Techniques provided herein are directed toward enabling on-device learning to create user-specific movement models that can be used for dead reckoning. Because these moving models are user-specific, they can be later used to identify user-specific motions in a manner that provides for a dead reckoning location estimation. In some embodiments, these models can be focused on pedestrian movement, based on the repetitive motion that occurs when a user takes a stride (walking, jogging, running, etc.) or other repetitive motion (swimming, riding a horse, etc.).

User-specific learning for improved pedestrian motion modeling in a mobile device

The subject matter disclosed herein relates to a system and method for receiving a plurality of signals generated by a plurality of sensors adapted to detect physical movement of a mobile device with respect to a plurality of coordinate axes. A time at which at least one of the received signals is digitized is delayed to provide an output of digitized versions of the received plurality of signals synchronized with respect to a common point in time.

Method and apparatus for accurate acquisition of inertial sensor data

Techniques provided herein are directed toward resolving a direction of travel of a mobile device based on MEMS sensor data by identifying a type of motion the mobile device is subject to and offsetting vertical acceleration data with horizontal acceleration data by a predetermined time offset based on the identified type of motion. The offset vertical and horizontal acceleration data can then be used to resolve, with an increased accuracy, a direction of travel of the mobile device.

Pedestrian sensor assistance in a mobile device during typical device motions

Growing community of inventors

San Jose, CA, United States of America

Joseph Czompo