Apple Inc.

Xiaohuan Corina Wang

Vijay Sundaram

Kevin M O'Neil

Giovanni M Agnoli

Brian Edward Walsh

Omid Khalili

Randy Ubillos

Brian Everett Meaney

David Robert Black

Colleen M Pendergast

Xiaojin Shi

Marco Zuliani

Stephen Sheeler

Francesco Rossi

Eric J Graves

Peter Chou

Richard H Salvador

Zehang Sun

David N Chen

Bartlomiej W Rymkowski

Paul Thomas Schneider

Andrew L Harding

Shaun M Poole

James Brasure

Joseph-Alexander P Weil

Greg Dudey

Katherine K Ernst

William D Cummings

Stuart Mark Pomerantz

Toshihiro Horie

Eric S Beale

Jean-Michel Berthoud

Alexey Marinichev

Nathalie Castel

Andy Harding

Joash S Berkeley

Marc Robins

Robert L Clark

Joe Weil

Mike Stern

Benjamin Poulain

Michael P Stern

Systems, methods, apparatuses and non-transitory, computer-readable storage mediums are disclosed for generating AR self-portraits or 'AR selfies.' In an embodiment, a method comprises: capturing, by a first camera of a mobile device, live image data, the live image data including an image of a subject in a physical, real-world environment; receiving, by a depth sensor of the mobile device, depth data indicating a distance of the subject from the camera in the physical, real-world environment; receiving, by one or more motion sensors of the mobile device, motion data indicating at least an orientation of the first camera in the physical, real-world environment; generating a virtual camera transform based on the motion data, the camera transform for determining an orientation of a virtual camera in a virtual environment; and generating a composite image data, using the image data, a matte and virtual background content selected based on the virtual camera orientation.

Augmented reality self-portraits

In some implementations, a user device can be configured to create media messages with automatic titling. For example, a user can create a media messaging project that includes multiple video clips. The video clips can be generated based on video data and/or audio data captured by the user device and/or based on pre-recorded video data and/or audio data obtained from various storage locations. When the user device captures the audio data for a clip, the user device can obtain a speech-to-text transcription of the audio data in near real time and present the transcription data (e.g., text) overlaid on the video data while the video data is being captured or presented by the user device.

Media message creation with automatic titling

Artistic styles extracted from one or more source images may be applied to one or more target images, e.g., in the form of stylized images and/or stylized video sequences. The extracted artistic style may be stored as a plurality of layers in a neural network, which neural network may be further optimized, e.g., via the fusion of various elements of the network's architectures. An optimized network architecture may be determined for each processing environment in which the network will be applied. The artistic style may be applied to the obtained images and/or video sequence of images using various optimization methods, such as the use of scalars to control the resolution of the unstylized and stylized images, temporal consistency constraints, as well as the use of dynamically adjustable or selectable versions of Deep Neural Networks (DNN) that are responsive to system performance parameters, such as available processing resources and thermal capacity.

Flexible resolution support for image and video style transfer

Systems, methods, apparatuses and non-transitory, computer-readable storage mediums are disclosed for generating AR self-portraits or 'AR selfies.' In an embodiment, a method comprises: capturing, by a first camera of a mobile device, image data, the image data including an image of a subject in a physical, real-world environment; receiving, by a depth sensor of the mobile device, depth data indicating a distance of the subject from the camera in the physical, real-world environment; receiving, by one or more motion sensors of the mobile device, motion data indicating at least an orientation of the first camera in the physical, real-world environment; generating a virtual camera transform based on the motion data, the camera transform for determining an orientation of a virtual camera in a virtual environment; and generating a composite image data, using the image data, a matte and virtual background content selected based on the virtual camera orientation.

Creating augmented reality self-portraits using machine learning

Real-time selection of DNN style transfer networks from DNN sets

A novel method for retiming a portion of a media content (e.g., audio data, video data, audio and video data, etc.) in a media-editing application is provided. The media editing application includes a user interface for defining a range in order to select a portion of the media content. The media editing application performs retiming by applying a speed effect to the portion of the media content selected by the defined range. For a faster speed effect, the media editing application retimes the selected portion of the media content by sampling the media content at a faster rate. For a slower speed effect, the media editing application retimes the selected portion of the media content by sampling the content at a slower rate.

Retiming media presentations

Some embodiments provide a method that receives the addition of a video clip having a first set of spatial properties to a composite video project having a second set of spatial properties. When the first set of spatial properties and the second set of spatial properties are different, the method automatically applies a spatial conform effect to the video clip to conform images of the video clip to the second set of spatial properties. The method receives input to transform images of the video clip as displayed in the composite video project. The method stores the spatial conform effect and the received transform as separate effects for the video clip.

Spatial conform operation for a media-editing application

A novel method for creating multimedia presentations in a media editing application is provided. The method selects a portion of a media clip and retimes the selected portion of the media clip from a first playback speed to a second playback speed. The method defines a transition interval for a discontinuity between a first section of the media clip with the first playback speed and a second section of the media clip with the second playback speed. The media clip is played back according to a smoothed curve that is constrained by the transition interval, the first playback speed, and the second playback speed. The smoothed curve transitions gradually from the first playback speed at the start of the transition interval to the second playback speed at the end of the transition interval.

User interface for retiming in a media authoring tool

Some embodiments provide a method that receives the addition of a video clip to a composite video project. The video clip has a sequence of video images at a first frame rate and the composite video project has a second frame rate for outputting video images. When the first frame rate does not match the second frame rate but is within a threshold of the second frame rate, the method generates output video images for a particular duration of the composite video project at the second frame rate by using each of the video images of the video clip once during the particular duration. When the first frame rate is not within the threshold, generating output video images for the particular duration of the composite video project at the second frame rate by using at least one of the video images for each output video image over the particular duration.

Rate conform operation for a media-editing application

Growing community of inventors

Los Altos, CA, United States of America

Xiaohuan Corina Wang