Apple Inc.

Zachary Z Becker

Michelle Chua

Robin-Yann Joram Storm

Etienne H Guerard

Adam James Bolton

David Adam Lipton

Adam Michael O'Hern

Eric Gregory Thivierge

Jue Wang

Michelle Tze Hiang Chua

Karen Natalie Wong

David Andrew Brasgalla

Bradley Warren Peebler

Peter Justin Dollar

Gerald Louis Guyomard

Tobias Rick

Shem Nguyen

Mahdi Nabiyouni

Robin Yann Joram Storm

Scott Bassett

Philip M Simmons

Rafael Saracchini

Christopher I Wilson

Allison W Dryer

Aaron Mackay Burns

Thorsten Gernoth

Arun Rakesh Yoganandan

Tyler L Casella

Benjamin R Blachnitzky

Ryan S Burgoyne

Novaira Masood

Arnold H Cachelin

Jeremy Ray Bernstein

Kyle Ellington Fisher

Anette L Freiin Von Kapri

Eric Geusz

Soravis Prakkamakul

Omar Shaik

Michael P Johnson

Eric Chiu

Jacob Ty Sawyer

Christopher I Word

Eduardo Sanchez Santos

Etienne Guerard

Some examples of the disclosure are directed to object manipulators and associated processes for manipulating an object representation in a three-dimensional environment. The object representation may correspond to a scan of a real-world object in a real-world environment. The object manipulators may include an object cleanup manipulator and a text annotation manipulator. The object cleanup manipulator may be selectable to display one or more control affordances providing functionality for selectively removing portions of the object representation in the three-dimensional environment and/or selectively adjusting one or more parameters of the object representation in the three-dimensional environment. The text annotation manipulator may be selectable to display one or more control affordances providing functionality for selectively generating one or more text labels in the three-dimensional environment. The one or more text labels may be associated with the object representation in the three-dimensional environment.

System and method of three-dimensional object cleanup and text annotation

In one implementation, a method of casting a scene is performed at a first device including a display, an input device, one or more processors, and non-transitory memory. The method includes displaying, on the display, a graphical user interface for composing an XR scene, wherein the graphical user interface includes an assets region displaying a list of assets of the XR scene and a casting affordance. The method includes detecting, via the input device, a user input selecting the casting affordance. The method includes, in response to detecting the user input selecting the casting affordance, determining a second device to receive a container file, wherein the second device is a stereoscopic device. The method includes storing the assets of the XR scene as the container file. The method includes transmitting the container file to the second device.

Container file viewing

Display screen or portion thereof with icon

A method is performed at an electronic device with a non-transitory memory, one or more processors, and a display. The method includes obtaining positional information regarding an auxiliary device. The method includes determining an anchor point of the auxiliary device on a volumetric computer-generated object. Determining the anchor point includes setting the anchor point to a first position of the auxiliary device in accordance with a determination that dimensions of the volumetric computer-generated satisfy a first criterion. Determining the anchor point further includes setting the anchor point to a second position of the auxiliary device in accordance with a determination that the dimensions of the volumetric computer-generated object satisfy a second criterion different from the first criterion. The first position is different from the second position. The method includes anchoring, based on the positional information, the volumetric computer-generated object to the anchor point of the auxiliary device.

Anchoring to multiple positions of an auxiliary device

Some examples of the disclosure are directed to media editing methods and graphical user interfaces. In some examples, the media editing user interface includes a plurality of user interface options and tools for capturing and editing media generated by a plurality of media recording devices. In some examples, states of the media recording devices are modified to add respective content to a media stream. In some examples, the media editing user interface includes representations of media content from the orientation of a respective media recording device. In some examples, the media editing user interface can present controls to alter contents of the media stream and publish and/or export the contents of the media stream.

Systems, methods and graphical user interfaces for media capture and editing applications

In one implementation, a method of displaying files is performed at a device including a display, an input device, one or more processors, and non-transitory memory. The method includes displaying, in a first area of the display, a plurality of file representations respectively representing a plurality of files. The method includes detecting, via the input device, selection of a first file representation representing a first file from the plurality of file representations, wherein the first file is a first container file including a plurality of first constituent files. The method includes determining a first file type. The method includes while displaying the plurality of file representations, displaying, in a second area of the display, a first set of first constituent file representations respectively representing each of the plurality of first constituent files having the first file type.

Generating a three-dimensional virtual representation of a three-dimensional physical object can be based on capturing or receiving a capture bundle or a set of images. In some examples, generating the virtual representation of the physical object can be facilitated by user interfaces for identifying a physical object and capturing a set of images of the physical object. Generating the virtual representation can include previewing or modifying a set of images. In some examples, generating the virtual representation of the physical object can include generating a first representation of the physical object (e.g., a point cloud) and/or generating a second three-dimensional virtual representation of the physical object (e.g., a mesh reconstruction). In some examples, a visual indication of the progress of the image capture process and/or the generation of the virtual representation of the three-dimensional object can be displayed, such as in a capture user interface.

Systems, methods, and user interfaces for generating a three-dimensional virtual representation of an object

A three-dimensional preview of content can be generated and presented at an electronic device in a three-dimensional environment. The three-dimensional preview of content can be presented concurrently with a two-dimensional representation of the content in a content generation environment presented in the three-dimensional environment. While the three-dimensional preview of content is presented in the three-dimensional environment, one or more affordances can be provided for interacting with the one or more computer-generated virtual objects of the three-dimensional preview. The one or more affordances may be displayed with the three-dimensional preview of content in the three-dimensional environment. The three-dimensional preview of content may be presented on a three-dimensional tray and the one or more affordances may be presented in a control bar or other grouping of controls outside the perimeter of the tray and/or along the perimeter of the tray.

Devices, methods and graphical user interfaces for three-dimensional preview of objects

A computer-generated virtual light source manipulator having one or more affordances for manipulating a computer-generated virtual light sources is disclosed. Selection of a virtual light source can cause a light source manipulator tailored for that virtual light source to be displayed over the virtual light source. The light source manipulator can include various lines, circles and the like that can define starting boundaries (e.g., surfaces that represent the start location and initial aperture of light emission from the virtual light source), ending boundaries (e.g., surfaces that represent the extent or reach (i.e., end location or projection distance) and final aperture of light transmission from the virtual light source), and fade boundaries (e.g., surfaces that represent the beginning of the fading of the virtual light source). The light source manipulators can also include one or more disc or spherical affordances for adjusting these boundaries.

Techniques for manipulating computer graphical light sources

A computer-generated virtual object manipulator having one or more affordances for manipulating a computer-generated virtual object is disclosed. Selection of a virtual object can cause an object manipulator to be displayed over the virtual object. The object manipulator can include a cone-shaped single-axis translation affordance for each of one or more object axes, a disc-shaped single-axis scale affordance for each of the one or more object axes, an arc-shaped rotation affordance for rotation about each of the one or more object axes, and a center of object affordance for free space movement of the virtual object. The object manipulator can also include a slice-shaped two-axis translation affordance that can be displayed after hovering over an area in a particular plane.

Techniques for manipulating computer graphical objects

Various implementations disclosed herein include devices, systems, and methods that facilitate the creation of a 3D model for object detection based on a scan of the object. Some implementations provide a user interface that a user interacts with to facilitate a scan of an object to create 3D model of the object for later object detection. The user interface may include an indicator that provides visual or audible feedback to the user indicating the direction that the capturing device is facing relative to the object being scanned. The direction of the capture device may be identified using sensors on the device (e.g., inertial measurement unit (IMU), gyroscope, etc.) or other techniques (e.g., visual inertial odometry (VIO)) and based on the user positioning the device so that the object is in view.

Object scanning for subsequent object detection

Various implementations disclosed herein include devices, systems, and methods that generates a three-dimensional (3D) model based on a selected subset of the images and depth data corresponding to each of the images of the subset. For example, an example process may include acquiring sensor data during movement of the device in a physical environment including an object, the sensor data including images of a physical environment captured via a camera on the device, selecting a subset of the images based on assessing the images with respect to motion-based defects based on device motion and depth data, and generating a 3D model of the object based on the selected subset of the images and depth data corresponding to each of the images of the selected subset.

Single-pass object scanning

In one implementation, a method of playing audio is performed at a device including one or more processors coupled to non-transitory memory. The method includes playing audio associated with an object at a frequency-dependent volume based on a distance between a user location and an object location, an object orientation with respect to the user location, and the frequency-dependent three-dimensional audio emission pattern. In various implementations, methods of visualizing the frequency-dependent three-dimensional audio emission pattern and changing properties thereof are disclosed.

Methods and systems for visualizing audio properties of objects

Various implementations disclosed herein include a method performed by a device. While executing a CGR application, the method includes displaying a three-dimensional object in a three-dimensional space, wherein the three-dimensional space is defined by a three-dimensional coordinate system. The method also includes: detecting a first user input directed to the three-dimensional object; and in response to detecting the first user input, displaying a spatial manipulation user interface element including a set of spatial manipulation affordances respectively associated with a set of spatial manipulations of the three-dimensional object, wherein each of the set of spatial manipulations corresponds to a translational movement of the three-dimensional object along a corresponding axis of the three-dimensional space.

Device, method, and graphical user interface for manipulating 3D objects on a 2D screen

Growing community of inventors

Santa Clara, CA, United States of America

Zachary Z Becker