Microsoft Technology Licensing, LLC

Snap Inc.

Luke Cartwright

Ilteris Kaan Canberk

Dmytro Kucher

Richard William Neal

Richard Zhuang

Alton Kwok

Misbah Uraizee

Matthew Hallberg

Marcelo Alonso Mejia Cobo

William Miles Miller

Jeremy Littel

Charles Miller

Alexander James Thaman

Alton Hau Kwong Kwok

Samuel Geoffrey Finding

Sambu Patach Arrojula

Hunter Araujo

The subject technology receives, by one or more hardware processors implementing a local wireless network, a request from a client device to mirror media content displayed on a screen of the client device on a wearable device. In response to the request, the subject technology causes a display of the media content in a mirroring lens of the wearable device. While the media content is being displayed in the mirroring lens of the wearable device, the subject technology tracks hand gestures of a user wearing the wearable device and viewing the media content displayed in the mirroring lens of the wearable device. The subject technology processes navigational or manipulation data based on the tracked hand gestures and sends a navigation or manipulation instruction to the client device or a mirroring lens processor of the wearable device based on the tracked hand gestures.

Mirroring and navigating content in augmented reality messaging systems

Systems, methods, and computer readable media for 3D content display using head-wearable apparatuses. Example methods include a head-wearable apparatus that is configured to determine a position for a content item on a closest curved line, of a plurality of curved lines, to the head-wearable apparatus that has space for the content item. The method includes adjusting a shape of the content item based on the position of the content item on the closest curved line and a user view of a user of the head-wearable apparatus. The method includes causing the adjusted content item to be displayed on a display of the head-wearable apparatus at the position on the closest curved line. The curved lines are either higher or lower as the curved lines goes away from the head-wearable apparatus. Additionally, the curved line or the content item may be adjusted with a random movement for an organic appearance.

3D content display using head-wearable apparatuses

A head-wearable apparatus determines an imaginary reference plane intersecting a head of a user viewing augmented content in a viewing pane having vertical and lateral dimensions in a display of the head-wearable apparatus. The imaginary reference plane coincides with a first viewing direction of the head of the user. The apparatus detects a rotational movement of the head of the user in a vertical direction while viewing the augmented content. In response to the detected rotational movement, the apparatus determines a second viewing direction of the head of the user when viewing the augmented content in the second viewing direction and determines a reference angle between the imaginary reference plane and the second viewing direction. Based on the reference angle, the apparatus assign one of a billboard display mode and a headlock display mode (or combination) to the augmented content presented in the display.

Orientation of augmented content in interaction systems

A method performed by a computing system for directing a voice command to a function associated with a visual target includes receiving a set of time-variable sensor-based data streams, including an audio data stream and a targeting data stream. The targeting data stream is stored in a buffer as buffered targeting data. Presence of a spoken utterance is identified within the audio data stream and is associated with a temporal identifier corresponding in time to the set of sensor-based data streams. A voice command corresponding to the spoken utterance is identified. A visual targeting vector within the buffered targeting data and a visual target of that visual targeting vector is identified at a time corresponding to the temporal identifier. The voice command is directed to a function associated with the visual target to generate an output.

Time-based visual targeting for voice commands

A computing system is provided. The computing system includes a processor of a display device configured to execute one or more programs. The processor is configured to receive a command from a user by way of natural language input. The processor is configured to identity a set of candidate objects within or adjacent a user's field of view having associated spatialized regions on which the command can be executed, the set of candidate objects identified at least partially by using a machine learning model. The processor is configured to use visual or audio indicators associated with the candidate objects and query the user for disambiguation input. The processor is configured to receive the disambiguation input from the user that selects a target object, executing the command on the target object. The processor is configured to train the machine learning model using the disambiguation input and data about the spatialized regions.

Natural language input disambiguation for spatialized regions

A computing system is provided. The computing system includes a processor of a display device configured to execute one or more programs. The processor is configured to receive, from a user, a voice command, a first auxiliary input from a first sensor, and a second auxiliary input from a second sensor. The processor is configured to, for each of a plurality of objects in the user's field of view in an environment, determine a first set of probability factors with respect to the first auxiliary input and a second set of probability factors with respect to the second auxiliary input. Each probability factor in the first and second sets indicates a likelihood that respective auxiliary inputs are directed to one of the plurality of objects. The processor is configured to determine a target object based upon the probability factors and execute the command on the target object.

Voice command execution from auxiliary input

A method for presenting a physical environment in a virtual environment includes presenting a virtual environment to a user with a near-eye display, imaging a physical environment of the user, positioning a passthrough portal in the virtual environment, fixing a position of the passthrough portal in the virtual environment relative to the physical environment, and presenting a video feed of the physical environment in the passthrough portal in the virtual environment.

Growing community of inventors

Seattle, WA, United States of America

Luke Cartwright