Apple Inc.

Eric S Beale

Xiaohuan Corina Wang

Zehang Sun

Brian Edward Walsh

Kevin M O'Neil

Greg Dudey

Stuart Mark Pomerantz

Toshihiro Horie

Jean-Michel Berthoud

Omid Khalili

Nathalie Castel

Angus M Taggart

Andy Harding

Marc Robins

Joe Weil

Systems, methods, apparatuses and non-transitory, computer-readable storage mediums are disclosed for generating AR self-portraits or 'AR selfies.' In an embodiment, a method comprises: capturing, by a first camera of a mobile device, live image data, the live image data including an image of a subject in a physical, real-world environment; receiving, by a depth sensor of the mobile device, depth data indicating a distance of the subject from the camera in the physical, real-world environment; receiving, by one or more motion sensors of the mobile device, motion data indicating at least an orientation of the first camera in the physical, real-world environment; generating a virtual camera transform based on the motion data, the camera transform for determining an orientation of a virtual camera in a virtual environment; and generating a composite image data, using the image data, a matte and virtual background content selected based on the virtual camera orientation.

Augmented reality self-portraits

Systems, methods, apparatuses and non-transitory, computer-readable storage mediums are disclosed for generating AR self-portraits or 'AR selfies.' In an embodiment, a method comprises: capturing, by a first camera of a mobile device, image data, the image data including an image of a subject in a physical, real-world environment; receiving, by a depth sensor of the mobile device, depth data indicating a distance of the subject from the camera in the physical, real-world environment; receiving, by one or more motion sensors of the mobile device, motion data indicating at least an orientation of the first camera in the physical, real-world environment; generating a virtual camera transform based on the motion data, the camera transform for determining an orientation of a virtual camera in a virtual environment; and generating a composite image data, using the image data, a matte and virtual background content selected based on the virtual camera orientation.

Creating augmented reality self-portraits using machine learning

A novel video system that detects events in a video sequence that causes such distortions and switch off optical flow based frame interpolation is provided. The system detects sudden changes in light intensity and switch to non-optical flow based frame interpolation when such sudden change is detected. When there is no such drastic change in light intensity, the system reverts back to using optical flow frame interpolation. Specifically, the system detects a flash event by computing a sum of differences in histograms of intensity levels between the pair of consecutive actual video frames. When the sum of differences is above a certain threshold, the video system switch to non-optical flow based frame interpolation. When the sum of differences is below the threshold, the video system reverts back to optical flow frame interpolation.

Generating synthetic video frames using optical flow

A novel method of using optical flow algorithm that maximizes the benefit of optical flow synthetic frames while minimizing the associated computation cost is provided. When using optical flow to produce synthetic frames between two actual/recorded frames, the method computes a set of estimates of optical flows (or a flow estimate) between the two frames. These flow estimates are then used to compute all synthetic frames that are needed between the two actual frames by interpolation, which creates each synthetic frame based on its temporal distances from the pair of actual frames.

Growing community of inventors

Santa Monica, CA, United States of America

Eric S Beale