Apple Inc.

Geoffrey Grant Stahl

Nathaniel C Begeman

Timothy Robert Oriol

Avi Bar-Zeev

Patrick L Coffman

Ian C Hendry

Kenneth Christian Dyke

Arthur Yasheng Zhang

Steve S Ko

Tobias Eble

Teck Yang Lee

Robert Beretta

Nicholas William Burns

Phillip Kent Miller

Guy Cote

Jim Chen Chou

Marc Jason Krochmal

Hao Pan

John Harper

Sheng Zhang

John Joseph Iarocci

Christopher S Linn

Gurjeet S Saund

Ray Ling Chang

Norman Nuo Wang

Alexander R Ledwith

Justin A Hensley

Ling Su

Jaques P Gasselin De Richebourg

Anselm Grundhoefer

Michael J Rockwell

Edwin Iskandar

Cody J White

Christopher C Niederauer

Helmut Garstenauer

Martin Garstenauer

Jonathan P Ive

Alan C Dye

Stephen Olivier Lemay

Julian Jaede

Fletcher R Rothkopf

Craig Michael Federighi

Cheng Kai Chen

Jeremy C Franklin

William A Sorrentino, Iii

Michael F Culbert

Joshua H Shaffer

Jason C Sauers

Wey-Jiun Lin

Ivan S Maric

Phil M Hobson

Jeremy Sandmel

Tim Millet

Graham Butler Myhre

Bruno M Sommer

Forrest C Wang

Howard Aron Miller

Srinivas Vedula

Yoonhoo Jo

Jacques P Gasselin De Richebourg

Muhammad F Hossain

Marinus Meursing

Jia Tao

Yanghai Tsin

David J Redman

Tom Sengelaub

Joel N Kerr

Assana Fard

Kathrin Berkner Cieslicki

Jeffrey S Norris

Bertrand Nepveu

Thomas W Costa

Jan K Quijalvo

Thomas Post

Julian Hoenig

Adrian T Sheppard

Jason L Slupeiks

Avi Bar-zeev

Toby C Paterson

Eve Brasfield

Amanda R Rainer

Mike Nugent

Ingrid Aligaen

Chris Niederauer

Peter Kaufmann

Branko Petljanski

Toby C Patterson

Thibaut Weise

James W Vandyke

Hesam Najafi

John S Harper

Joshua H Schaffer

Various implementations disclosed herein include devices, systems, and methods that provide shared extended reality (XR) experiences in which two or more users interact with their own sets of physical objects (e.g., cards, game pieces, dice, chips, etc.) during the shared experiences. Each user may have multiple physical objects, each of those physical objects having the same generic shape, size, etc., but having a unique fiducial marker. The unique fiducial marker of each physical object can be assigned to represent one of multiple virtual content items.

Tracking objects with fiducial markers in multiple environments to provide shared experiences

A mixed reality system including a head-mounted display (HMD) and a base station. Information collected by HMD sensors may be transmitted to the base via a wired or wireless connection. On the base, a rendering engine renders frames including virtual content based in part on the sensor information, and an encoder compresses the frames according to an encoding protocol before sending the frames to the HMD over the connection. Instead of using a previous frame to estimate motion vectors in the encoder, motion vectors from the HMD and the rendering engine are input to the encoder and used in compressing the frame. The motion vectors may be embedded in the data stream along with the encoded frame data and transmitted to the HMD over the connection. If a frame is not received at the HMD, the HMD may synthesize a frame from a previous frame using the motion vectors.

Video compression methods and apparatus

A head-mountable display device includes a housing defining a front opening and a rear opening, a display screen disposed in the front opening, a display assembly disposed in the rear opening, a first securement strap coupled to the housing, the first securement strap including a first electronic component, a second securement strap coupled to the housing, the second securement strap including a second electronic component, and a securement band extending between and coupled to the first securement strap and the second securement strap.

Head mountable display

Various implementations disclosed herein include devices, systems, and methods for compositing an affordance in association with a CGR object representing a physical article. In various implementations, a device includes a display, a non-transitory memory, and one or more processors coupled with the display and the non-transitory memory. In some implementations, a method includes displaying a computer-generated reality (CGR) object in a CGR environment. In some implementations, the CGR object represents a physical article. In some implementations, the method includes compositing an affordance in association with the CGR object. In some implementations, the method includes detecting an input directed to the affordance. In some implementations, the method includes, in response to detecting the input, adding an identifier identifying the physical article to a user selection queue.

Controlling a user selection queue

A mixed reality system that includes a device and a base station that communicate via a wireless connection The device may include sensors that collect information about the user's environment and about the user. The information collected by the sensors may be transmitted to the base station via the wireless connection. The base station renders frames or slices based at least in part on the sensor information received from the device, encodes the frames or slices, and transmits the compressed frames or slices to the device for decoding and display. The base station may provide more computing power than conventional stand-alone systems, and the wireless connection does not tether the device to the base station as in conventional tethered systems. The system may implement methods and apparatus to maintain a target frame rate through the wireless link and to minimize latency in frame rendering, transmittal, and display.

Video pipeline

A device for providing operating system managed group communication sessions may include a memory and at least one processor. The at least one processor may be configured to receive, by an operating system level process executing on a device and from an application process executing on a device, a request to initiate a group session between a user associated with the device and another user. The at least one processor may be further configured to identify, by the operating system level process, another device associated with the other user. The at least one processor may be further configured to initiate, by the operating system level process, the group session with the user via the other device. The at least one processor may be further configured to manage, by the operating system level process, the group session.

Operating system level management of group communication sessions

In accordance with some implementations, a method is performed at an electronic device including one or more processors, a non-transitory memory, a positional sensor, a rendering system, and a display. The method includes while displaying, on the display, first textual content according to an initial viewing angle, determining an expected viewing angle based on the initial viewing angle and positional data from the positional sensor. The positional data indicates a positional change of the electronic device. The initial viewing angle is different from the expected viewing angle. The method includes, in accordance with a determination that the expected viewing angle satisfies a render criterion, generating, via the rendering system, second textual content based on the expected viewing angle. The method includes displaying, on the display, the second textual content according to the expected viewing angle.

Generating textual content based on an expected viewing angle

The present disclosure relates to providing a computer-generated reality (CGR) platform for generating CGR environments including virtual and augmented reality environments. In some embodiments, information related to an object to be simulated and rendered in the CGR environment is provided to the CGR platform, and a three-dimensional representation of the object is displayed in the CGR environment.

Computer-generated reality platform for generating computer-generated reality environments

In one implementation, a method includes: displaying a UI element as an overlay in a UI associated with a first FOV, wherein the first FOV is characterized by a first viewing vector of a physical environment; detecting a change from the first FOV to a second FOV, wherein the second FOV is characterized by a second viewing vector of the physical environment; and in response to detecting the change from the first FOV to the second FOV, determining a prominence-display value for the UI element; if the prominence-display value for the UI element exceeds a prominence threshold, displaying the UI element as the overlay in the UI associated with the second FOV; and if the prominence-display value for the UI element does not exceed the prominence threshold, ceasing display of the UI element in the UI associated with the second FOV.

Application programming interface for setting the prominence of user interface elements

Various implementations disclosed herein include devices, systems, and methods that enable improved display of virtual content in computer generated reality (CGR) environments. In some implementations, the virtual content is viewed by an observer and includes multiple content elements, each of which may be rendered at an independent rate based on a change of view of the observer.

View dependent content updated rates

The present disclosure relates to providing a computer-generated reality (CGR) platform for generating CGR environments including virtual and augmented reality environments. In some embodiments, the platform includes an operating-system-level (OS-level) process that simulates and renders content in the CGR environment, and one or more application-level processes that provide information related to the content to be simulated and rendered to the OS-level process.

In one implementation, a method of determining a visual appearance of a user interface (UI) element overlaid in an underlying physical environment is performed at a device. The method includes: obtaining environmental data corresponding to a physical environment; determining environmental characteristics of the physical environment based on the environmental data; in response to obtaining a request to display a UI element, determining whether a prominence-display value satisfies a prominence criterion; in response to determining that the prominence-display value satisfies the prominence criterion: modifying visual characteristics of the UI element based on the environmental characteristics of the physical environment, and displaying the UI element as an overlay on the physical environment, wherein the UI element includes the modified visual characteristics; and in response to determining that the prominence-display value for the UI element does not satisfy the prominence criterion, foregoing modifying the visual characteristics of the UI element.

Growing community of inventors

San Jose, CA, United States of America

Geoffrey Grant Stahl