Apple Inc.

Grant H Mulliken

Izzet B Yildiz

Fletcher R Rothkopf

Sterling R Crispin

Mary A Pyc

Avi Bar-Zeev

Jae Hwang Lee

David A Kalinowski

Nicholas C Soldner

James W VanDyke

Lilli I Jonsson

Akiko Ikkai

Brian Pasley

Arlen M Hartounian

Steven A Marchette

Devin William Chalmers

Arthur Yasheng Zhang

Darshan R Kasar

Ray Ling Chang

Holly Gerhard

Javier Mendez

Paul Meade

David A Schmuck

Eric N Vergo

Nathanael D Parkhill

Ronald J Guglielmone, Jr

Brandon R Neale

Mikaela D Estep

Aaron Mackay Burns

Olivier Soares

Nathan Gitter

Benjamin Hylak

Andrew P Mason

Pau Sastre Miguel

Hermannus J Damveld

Christine Godwin

James Williams Charles Vandyke

Eve Ekman

Lauren Cheung

Anura A Patil

Jonathan Hadida

Raphael A Bernier

Patrick O Eronini

Srinath Nizampatnam

John S McCarten

Thomas Gebauer

James William Charles Vandyke

Holly E Gerhard

A head-mountable device including a display, a housing at least partially surrounding the display, a facial interface attached to the housing, and a cover positioned between the housing and the facial interface, the cover comprising a conductive fabric.

Conductive fabric architecture

Embodiments are directed to aspects of the extended reality environments that are selected or otherwise modified to account for distracting stimuli. Similarly, one or more metrics that reflect, or are otherwise indicative of, a user's ability to focus on a current or upcoming activity may be used to adjust a user's interaction with the extended reality environment. The extended reality environment can generated by an extended reality system that includes a head-mounted display, a set of sensors and a processor configured to enter a focus mode that reduces distraction in an extended reality environment. While in the focus mode, the process or can receive imaging data of a physical environment around a user using the set of sensors and generate the extended reality environment that includes a reproduction of the first region of the physical environment where an identified object is replaced with the additional content.

Operating modes for reducing distraction in extended reality environments

Various implementations disclosed herein include devices, systems, and methods that determine an interaction event during presentation of an interaction element. For example, an example process may include obtaining physiological data associated with a pupil during presentation of an interaction element, determining, based on the obtained physiological data, a pupillary response during the presentation of the interaction element, determining that the pupillary response corresponds to attention response characteristics associated with attention of a region of the regions of the interaction element based on the different illumination characteristics of the regions, and determining an interaction event during the presentation of the interaction element based on determining that the pupillary response corresponds to directing attention to the region during the presentation of the interaction element.

Interaction events based on physiological response to illumination

Various implementations disclosed herein include devices, systems, and methods that determine to record an event based on an emotional state and a scene understanding of an environment. For example, an example process may include obtaining physiological data or behavioral data during a first period of time utilizing the one or more sensors. The process may further include determining an emotional state during the first period of time based on the physiological data or the behavioral data (e.g., elated, happy, etc.). The process may further include determining a scene understanding of an environment based on identifying one or more objects in the environment utilizing the one or more sensors. The process may further include determining to record an event occurring in the environment (e.g., positively valenced events, objects, or situations) based on the emotional state and the scene understanding and record the event utilizing one or more sensors.

Dynamic recording of an experience based on an emotional state and a scene understanding

In at least one example of the present disclosure, a head-mountable device includes a display, a frame at least partially surrounding the display, a facial interface attached to the frame, a cover disposed around the facial interface, and a touch sensitive surface incorporated into the cover to receive input from a user for the head-mountable device.

Touch sensitive input surface

In one implementation, a method of assisting in the rehearsal of a presentation is performed at a device including a display, one or more processors, and non-transitory memory. The method includes obtaining a difficulty level for a rehearsal of a presentation. The method includes displaying, on the display, one or more slides of the presentation. The method includes displaying, on the display in association with a volumetric environment, one or more virtual objects based on the difficulty level.

Virtual presentation rehearsal

Some implementations disclosed herein provide systems, methods, and devices that use physiological data (e.g., indicative of surprise) of a user to determine that an unexpected user interface behavior occurred. In some implementations, a device having a processor implements a method. The method obtains, via a sensor, physiological data of a user during a period of time and uses the physiological data to determine a characteristic of the user during the period of time while the user is using the electronic device. The method identifies an unexpected user interface behavior occurring prior to the period of time based on the characteristic of the user during the period of time.

Detecting unexpected user interface behavior using physiological data

A display system includes a head-mounted display unit and a wake control system. The head-mounted display unit provides content to a user and is operable in a low-power state and a high-power state that consumes more power to provide the content to the user than the low-power state. The wake control system determines when to operate in the high-power state. The wake control system may assess a first wake criterion with low power, assess a second wake criterion with higher power than the first wake criterion upon satisfaction of the first wake criterion, and cause the head-mounted display unit to operate in the high-power state upon satisfaction of the second wake criterion.

Display system

Various implementations disclosed herein include devices, systems, and methods that determine an attentive state of a user during an experience (e.g., visual and/or auditory content that could include real-world physical environment, virtual content, or a combination of each) based on the user's gaze characteristic(s) to enhance the experience. For example, an example process may include obtaining physiological data associated with a gaze of a user during an experience, determine a gaze characteristic during a segment of the experience based on the obtained physiological data, and determine that the user has a first attentive state during the segment of the experience based on classifying the gaze characteristic of the user during the segment of the experience.

Eye-gaze based biofeedback

Some implementations disclosed herein present multimedia content to a user, identify a comfort state of the user (e.g., happy, unhappy, distressed, etc.) based on body data of the user (e.g., facial expressions, hand movements, physiological data, etc.), and update the multimedia content based on the identified comfort state.

User comfort monitoring and notification

Various implementations disclosed herein include devices, systems, and methods that assess physiological data of a user and a scene understanding of a physical environment to determine a retention state. For example, an example process may include obtaining physiological data in an environment during a first period of time. The process may further include identifying one or more of the objects in the environment based on determining a scene understanding of the environment. The process may further include determining, based on the physiological data and the scene understanding, features associated with interaction events for the one or more objects. The process may further include determining, based on the features, a retention state during the first period of time, the retention state associated with retention of a concept associated with an object of the one or more objects. The process may further include providing feedback based on identifying the retention state.

User feedback based on retention prediction

Various implementations disclosed herein include devices, systems, and methods that adjust image content to reduce motion sickness during an experience. For example, an example process may include determining a first zone and a second zone of a display, generating images of a three-dimensional (3D) environment, identifying image content of each of the images corresponding to the second zone of the display, and reducing at least one of contrast or spatial frequency of the image content of each of the images corresponding to the second zone of the display.

Adjusting image content to improve user experience

One exemplary implementation provides an improved user experience on a device by using physiological data to initiate a user interaction for the user experience based on an identified interest or intention of a user. For example, a sensor may obtain physiological data (e.g., pupil diameter) of a user during a user experience in which content is displayed on a display. The physiological data varies over time during the user experience and a pattern is detected. The detected pattern is used to identify an interest of the user in the content or an intention of the user regarding the content. The user interaction is then initiated based on the identified interest or the identified intention.

Pupil modulation as a cognitive control signal

A head-mounted device includes a device housing, a support structure that is connected to the device housing to support the device housing with respect to a user, a display device that is connected to the device housing to display content, an optical system that is associated with the display device, and sensors that generate sensor output signals. The head-mounted device also includes a tension controller that determines a tensioning command based on the sensor output signals, and a tension adjuster that applies tension to the user according to the tensioning command in order to restrain motion of the device housing with respect to the user.

Head-mounted device with tension adjustment

In accordance with some embodiments, exemplary techniques for transitioning from the physical environment to a computer-generated reality environment on a wearable electronic device is described. In accordance with some embodiments, exemplary techniques for transitioning out of a computer-generated reality environment on a wearable electronic device is described.

Transitioning between environments

Growing community of inventors

Sunnyvale, CA, United States of America

Grant H Mulliken