Apple Inc.

Daniele Casaburo

Adrian Peter Lindberg

Anselm Grundhoefer

Eshan Verma

Pedro Manuel Da Silva Quelhas

Omar Elafifi

Daniel R Kurz

Sebastian Knorr

Phillip N Smith

In one implementation, a method of remedying a medical impairment of a user is performed by a device including a processor, non-transitory memory, one or more biometric sensors, an image sensor, and a display. The method includes detecting, based on data from at least one of the image sensor and the one or more biometric sensors, a medical impairment of a user of the head-mounted device from a plurality of potential medical impairments associated with a plurality of remedies. The method includes selecting, from the plurality of remedies, a remedy of the medical impairment of the user. The method includes controlling the display to effect the remedy of the medical impairment of the user.

Method and device for health monitoring

In some implementations, representations of applications are identified, positioned, and configured in a computer generated reality (CGR) environment based on context. The location at which the representation of the application is positioned may be based on the context of the CGR environment. The context may be determined based on non-image data that is separate from image data of the physical environment being captured for the CGR environment. As examples, the non-image data may relate to the user, a user preferences, a user attribute, a user gesture, motion, activity, or interaction, semantics related to user input or an external source of information, the current time, date, or time period, information from another device involved in the CGR, etc.

Context-based application placement

Implementations of the subject technology provide for image composition for extended reality systems. Image composition may include combining virtual content from virtual images with physical content from images captured by one or more cameras. The virtual content and the physical content can be combined to form a composite image using depth information for the virtual content and the physical content. An adjustment mask may be generated to indicate edges or boundaries between virtual and physical content at which artifact correction for the composite image can be performed.

Image composition for extended reality systems

Systems, devices, and methods disclosed herein represent existing photo or video content within computer generated reality (CGR) environments. In some implementations, a representation of an existing photo or video is positioned within a CGR environment based on the position of the camera of the existing photo/video at the time the existing photo or video was captured. For example, a user experiencing CGR while walking along a hiking trail may see a floating representation of a photo that the user took on his or her last trip on that hiking trail. The representation may be positioned and oriented in the same position and orientation that the camera was positioned and oriented in when the photo was first captured, e.g., 5 feet above the ground, three feet north of the hiking trail welcome sign. The representation may be a version of the image itself or provide information about the image.

Visualization of existing photo or video content

Various implementations disclosed herein include devices, systems, and methods that create additional depth frames where a depth camera runs at a lower frame rate than a light intensity camera. Rather than upconverting the depth frames by simply repeating a previous depth camera frame, additional depth frames are created by adjusting some of the depth values of a prior frame based on the RGB camera data (e.g., by 'dragging' depths from their positions in the prior depth frame to new positions for a new frame). Specifically, a contour image is generated, and changes in the contour image are used to determine how to adjust (e.g., drag) the depth values for the additional depth frames. The contour image may be based on a mask (e.g., occlusions masks identifying where the hand occludes the virtual cube).

Depth rate up-conversion

Various implementations disclosed herein include devices, systems, and methods that determines changes in depth category of pixels from one image to the next. For example, changes in depth category may include determining that a given pixel has changed from foreground to background or vice versa. The technique is based on creating a histogram or other model using a mask identifying a depth category of pixels in the first image. The histogram or model is used to assess a pixel in the first frame and the second frame and identify a change in its depth category. One use of this technique is to determine a change in occlusion.

Occlusion detection

Various implementations disclosed herein include devices, systems, and methods that create additional depth frames in the circumstance wherein a depth camera runs at a lower frame rate than a light intensity camera. Rather than upconverting the depth frames by simply repeating a previous depth camera frame, additional depth frames are created by adjusting some of the depth values of a prior frame based on the RGB camera data (e.g., by 'dragging' depths from their positions in the prior depth frame to new positions for a new frame). Specifically, a contour image (e.g., identifying interior and exterior outlines of a hand with respect to a virtual cube that the hand occludes) is generated based on a mask (e.g., occlusions masks identifying where the hand occludes the virtual cube). Changes in the contour image are used to determine how to adjust (e.g., drag) the depth values for the additional depth frames.

An augmented safety restraint system includes a first restraint operable to secure a first passenger. The first restraint has an exposed surface facing away from a body of the first passenger. A gesture-sensing device is disposed on the exposed surface of the first restraint and is operable to receive an input from the first passenger.

Growing community of inventors

Cupertino, CA, United States of America

Daniele Casaburo