Apple Inc.

Izzet B Yildiz

Grant H Mulliken

Sterling R Crispin

Mary A Pyc

Akiko Ikkai

Fletcher R Rothkopf

Ronald J Guglielmone, Jr

Mikaela D Estep

Olivier Soares

Nicholas C Soldner

Andrew P Mason

Lilli I Jonsson

Brian Pasley

Steven A Marchette

Srinath Nizampatnam

John S McCarten

Thomas Gebauer

Christine Godwin

Various implementations disclosed herein include devices, systems, and methods that determine to record an event based on an emotional state and a scene understanding of an environment. For example, an example process may include obtaining physiological data or behavioral data during a first period of time utilizing the one or more sensors. The process may further include determining an emotional state during the first period of time based on the physiological data or the behavioral data (e.g., elated, happy, etc.). The process may further include determining a scene understanding of an environment based on identifying one or more objects in the environment utilizing the one or more sensors. The process may further include determining to record an event occurring in the environment (e.g., positively valenced events, objects, or situations) based on the emotional state and the scene understanding and record the event utilizing one or more sensors.

Dynamic recording of an experience based on an emotional state and a scene understanding

Some implementations disclosed herein provide systems, methods, and devices that use physiological data (e.g., indicative of surprise) of a user to determine that an unexpected user interface behavior occurred. In some implementations, a device having a processor implements a method. The method obtains, via a sensor, physiological data of a user during a period of time and uses the physiological data to determine a characteristic of the user during the period of time while the user is using the electronic device. The method identifies an unexpected user interface behavior occurring prior to the period of time based on the characteristic of the user during the period of time.

Detecting unexpected user interface behavior using physiological data

Various implementations disclosed herein include devices, systems, and methods that determine an attentive state of a user during an experience (e.g., visual and/or auditory content that could include real-world physical environment, virtual content, or a combination of each) based on the user's gaze characteristic(s) to enhance the experience. For example, an example process may include obtaining physiological data associated with a gaze of a user during an experience, determine a gaze characteristic during a segment of the experience based on the obtained physiological data, and determine that the user has a first attentive state during the segment of the experience based on classifying the gaze characteristic of the user during the segment of the experience.

Eye-gaze based biofeedback

Some implementations disclosed herein present multimedia content to a user, identify a comfort state of the user (e.g., happy, unhappy, distressed, etc.) based on body data of the user (e.g., facial expressions, hand movements, physiological data, etc.), and update the multimedia content based on the identified comfort state.

User comfort monitoring and notification

Various implementations disclosed herein include devices, systems, and methods that assess physiological data of a user and a scene understanding of a physical environment to determine a retention state. For example, an example process may include obtaining physiological data in an environment during a first period of time. The process may further include identifying one or more of the objects in the environment based on determining a scene understanding of the environment. The process may further include determining, based on the physiological data and the scene understanding, features associated with interaction events for the one or more objects. The process may further include determining, based on the features, a retention state during the first period of time, the retention state associated with retention of a concept associated with an object of the one or more objects. The process may further include providing feedback based on identifying the retention state.

User feedback based on retention prediction

One exemplary implementation displays a visual characteristic associated with an object on a display of a device and utilizes a sensor of the device to obtain physiological data associated with a pupillary response of a user to the visual characteristic. The device adjusts the visual characteristic based on the obtained physiological data to enhance pupillary responses of the user to the object and displays the adjusted visual characteristic to the user. For example, the adjusted visual characteristic may be selected based on previously identified pupillary responses of the user to particular visual characteristics.

Biofeedback method of modulating digital content to invoke greater pupil radius response

Sensor data indicating a user's response to an avatar experience in which the user experiences a rendered avatar model is obtained. A perceptual quality metric value corresponding to the rendered avatar model is determined based on the sensor data and a determined relationship between the sensor data and the perceptual quality metric value. The avatar model is re-rendered for display based on the perceptual quality metric value.

Perceptually and physiologically constrained optimization of avatar models

Devices, systems, and methods that provide content that distracts a user's attention away from discomfort, for example, occurring due to a prior injury or ongoing medical condition. The content is adjusted over time based on tracking the user's attentiveness towards the content. Specifically, the user's current attentive state may be tracked using physiological sensors and used to adapt content to mitigate the perception of discomfort. In some implementations, the user's stress is also assessed and used an indication of the user's current discomfort level to better track when the user's attentive state is shifting from the content to the discomfort and adjust the content accordingly.

Adaptation of electronic content

Some implementations disclosed herein present a computer-generated reality (CGR) environment in which a user participates in an activity, identify a cognitive state of the user (e.g., working, learning, resting, etc.) based on data regarding the user's body (e.g., facial expressions, hand movements, physiological data, etc.), and update the environment with a surrounding environment that promotes a cognitive state of the user for the activity.

Creation of optimal working, learning, and resting environments on electronic devices

Identifying a state of a user (e.g., attentive, distracted, mind wandering, etc.) based on the user's physiological (e.g., pupillary) response to luminance change events in which a portion or all of the content quickly becomes brighter or dimmer.

Growing community of inventors

Sunnyvale, CA, United States of America

Izzet B Yildiz