Magic Leap, Inc.

Em Labs, Inc.

Michael Janusz Woods

Andrew Rabinovich

Eric C Browy

Evan Francis Rynk

James William Palmer

Brian Bucknor

Christopher Lopez

Aly H M Aly

Samuel A Miller

David Charles Lundmark

Sheng Wan

Scott David Nortman

Paul M Greco

Richard Leslie Taylor

Forrest Grinstead

Evan Murphy

Marshall Grinstead

Aly HM Aly

Configurations are disclosed for presenting virtual reality and augmented reality experiences to users. An augmented reality display system comprises a handheld component housing an electromagnetic field emitter, the electromagnetic field emitter emitting a known magnetic field, the head mounted component coupled to one or more electromagnetic sensors that detect the magnetic field emitted by the electromagnetic field emitter housed in the handheld component, wherein a head pose is known, and a controller communicatively coupled to the handheld component and the head mounted component, the controller receiving magnetic field data from the handheld component, and receiving sensor data from the head mounted component, wherein the controller determining a hand pose based at least in part on the received magnetic field data and the received sensor data.

Systems and methods for augmented reality

Head-mounted augmented reality (AR) devices can track pose of a wearer's head to provide a three-dimensional virtual representation of objects in the wearer's environment. An electromagnetic (EM) tracking system can track head or body pose. A handheld user input device can include an EM emitter that generates an EM field, and the head-mounted AR device can include an EM sensor that senses the EM field. EM information from the sensor can be analyzed to determine location and/or orientation of the sensor and thereby the wearer's pose. The EM emitter and sensor may utilize time division multiplexing (TDM) or dynamic frequency tuning to operate at multiple frequencies. Voltage gain control may be implemented in the transmitter, rather than the sensor, allowing smaller and lighter weight sensor designs. The EM sensor can implement noise cancellation to reduce the level of EM interference generated by nearby audio speakers.

Electromagnetic tracking with augmented reality systems

Methods and systems for triggering presentation of virtual content based on sensor information. The display system may be an augmented reality display system configured to provide virtual content on a plurality of depth planes using different wavefront divergences. The system may monitor information detected via the sensors, and based on the monitored information, trigger access to virtual content identified in the sensor information. Virtual content can be obtained, and presented as augmented reality content via the display system. The system may monitor information detected via the sensors to identify a QR code, or a presence of a wireless beacon. The QR code or wireless beacon can trigger the display system to obtain virtual content for presentation.

Systems and methods for reducing error from noisy data received from a high frequency sensor by fusing received input with data received from a low frequency sensor by collecting a first set of dynamic inputs from the high frequency sensor, collecting a correction input point from the low frequency sensor, and adjusting a propagation path of a second set of dynamic inputs from the high frequency sensor based on the correction input point either by full translation to the correction input point or dampened approach towards the correction input point.

An electromagnetic tracking system includes a handheld controller including a first phased array element characterized by a first phase and a second phased array element characterized by a second phase different than the first phase. The first phased array element and the second phased array element are configured to generate a steerable electromagnetic beam characterized by an electromagnetic field pattern. The electromagnetic tracking system also includes a head mounted augmented reality display including an electromagnetic sensor configured to sense the electromagnetic field pattern.

Method and system utilizing phased array beamforming for six degree of freedom tracking for an emitter in augmented reality systems

A handheld controller includes a housing and a first vibration source disposed in the housing and characterized by a first phase. The handheld controller also includes a second vibration source disposed in the housing and characterized by a second phase different than the first phase and a controller disposed in the housing, coupled to the first vibration source and the second vibration source, and configured to vary at least one of the first phase or the second phase.

Method and system for haptic beamforming and haptic effects in a handheld controller

A sensory eyewear system for a mixed reality device can facilitate user's interactions with the other people or with the environment. As one example, the sensory eyewear system can recognize and interpret a sign language, and present the translated information to a user of the mixed reality device. The wearable system can also recognize text in the user's environment, modify the text (e.g., by changing the content or display characteristics of the text), and render the modified text to occlude the original text.

Systems and methods for sign language recognition

Head-mounted augmented reality (AR) devices can track pose of a wearer's head or pose of a hand-held user input device to enable wearer interaction in a three-dimensional AR environment. A pose sensor (e.g., an inertial measurement unit) in the user input device can provide data on pose (e.g., position or orientation) of the user input device. An electromagnetic (EM) tracking system can also provide pose data. For example, the handheld user input device can include an EM emitter that generates an EM field, and the head-mounted AR device can include an EM sensor that senses the EM field. The AR device can combine the output of the pose sensor and the EM tracking system to reduce drift in the estimated pose of the user input device or to transform the pose into a world coordinate system used by the AR device. The AR device can utilize a Kalman filter to combine the output of the pose sensor and the EM tracking system.

Sensor fusion for electromagnetic tracking

Head-mounted augmented reality (AR) devices can track pose of a wearer's head to provide a three-dimensional virtual representation of objects in the wearer's environment. An electromagnetic (EM) tracking system can track head or body pose. A handheld user input device can include an EM emitter that generates an EM field, and the head-mounted AR device can include an EM sensor that senses the EM field. EM information from the sensor can be analyzed to determine location and/or orientation of the sensor and thereby the wearer's pose. An improved or optimized pose can be provided by reverse-estimating a reverse EM measurement matrix and optimizing the pose based on a comparison between the reverse EM measurement matrix and an EM measurement matrix measured by the EM sensor.

Pose estimation using electromagnetic tracking

In various embodiments, a wearable component configured to be worn on a head of a user is disclosed. The wearable component can comprise a wearable support and an electronic component coupled to or disposed within the wearable support. A thermal management structure can be provided in thermal communication with the electronic component. The thermal management structure can be configured to transfer heat from the electronic component away from the head of the user when the wearable support is disposed on the head of the user.

Growing community of inventors

Mountain View, CA, United States of America

Michael Janusz Woods