Microsoft Technology Licensing, LLC

Elwha LLC

Other

Perkinelmer Health Sciences, Inc.

Microsoft Corporation

Andrew David Wilson

Hrvoje Benko

Steven Nabil Bathiche

Desney S Tan

Jeannette M Wing

Eyal Ofek

Christian Holz

Sasa Junuzovic

Kenneth Paul Hinckley

Lowell L Wood, Jr

Roderick A Hyde

Clarence T Tegreene

Jordin T Kare

Nathan P Myhrvold

Muriel Y Ishikawa

Victoria Y H Wood

Charles Whitmer

Robert C Petroski

Jesse R Cheatham, Iii

Craig J Mundie

Patrick Markus Baudisch

Eric D Rudder

Anoop Gupta

William Thomas Blank

Nigel Stuart Keam

Meredith June Morris

Michael Jack Sinclair

Yong Rui

Daniel Chaim Robbins

Daniel Morris

Joel P Dehlin

Nikunj Raghuvanshi

Stephen Edward Hodges

Shahram Izadi

Raman Kumar Sarin

Nuria M Oliver

William Arthur Stewart Buxton

Eric J Horvitz

Zhengyou Zhang

Steven Mark Drucker

Michael Anthony Hall

Philip Andrew Chou

James Morris Alkove

Michel Pahud

Boyd Cannon Multerer

John C Tang

Neil Scott Fishman

Ravin Balakrishnan

Michael Koenig

Abigail Jane Sellen

Edward Bryan Cutrell

Jon Marcus Randall Whitten

Bret P O'Rourke

David J Kurlander

Pourang Polad Irani

Francois Guimbretiere

Edward Sean Lloyd Rintel

Douglas Christopher Burger

Michael F Cohen

Matthew Graham Dyor

Shai Guday

Billy P Chen

Dawson Yee

Ivan Jelev Tashev

Julia Schwarz

David Alexander Butler

Asta Jane Roseway

David W Russo

F David Jones

Otmar Hilliges

John Mark Miller

Ian Sands

Shane Williams

Mihai R Jalobeanu

Chris Harrison

Peter N Glaskowsky

Kathryn Stone Perez

Ryen William White

Kentaro Toyama

Blaise Aguera Y Arcas

John R Lewis

Brian James Mount

Kurt Jenkins

Oscar E Murillo

Matthew Bret MacLaurin

Alex A Kipman

Anthony James Ambrus

Tim Paek

Glen C Larsen

Aleksandar Uzelac

Maneesh Agrawala

Sumit Basu

Weidong Zhao

Mathew Julian Lamb

Alex Olwal

James Thomas Kajiya

Ali M Vassigh

James E Allard

Gregg Robert Wygonik

Sophie Stellmach

A computer-implemented method is described which comprises generating a representation of a digital space and a representation of the physical space using an audiovisual feed received from a camera proximate to a display located in the physical space. The representation of the digital space is generated using user information identifying a remote user associated with the display and presence information relating to the remote user and the digital representation comprises an avatar of the remote user. The representation of the digital space is output to the display located in the physical space and the representation of the physical space it output to a computing device associated with the remote user. The method further comprises dynamically updating the representation of the digital space and/or physical space in response to changes in the user information and presence information.

Collaborative system

Aspects of this present disclosure relate to hybrid conference user interface. The hybrid conference interface provides an establishing shot before the meeting begins that places meeting attendees in a specific spatial arrangement, such as in specific seats around a conference table. Upon starting the conference, the hybrid user interface renders an appropriate perspective view of the meeting that is tailored to each attendee's perspective while also being spatially consistent for the entire group of attendees. Allowing attendees to place themselves where they want gives attendees a sense of physical space that helps them stay spatially oriented relative to the other people and resources in the room.

Established perspective user interface and user experience for video meetings

Technologies for transitioning between two-dimensional (2D) and three-dimensional (3D) display views for video conferencing are described herein. Video conferencing applications can have multiple display views for a user participating in a video conference. In certain situations a user may want to transition from a 2D display view of the video conference to a more immersive 3D display view. These transitions can be visually jarring and create an uncomfortable user experience. The transition from a 2D display view to a 3D display view can be improved by executing the transition to a 3D display view by manipulating visual properties of a virtual camera that is employed to generate the display views.

Computer-implemented video conference application that is configured to transition between 2D and 3D views

The interactive and shared surface technique described herein employs hardware that can project on any surface, capture color video of that surface, and get depth information of and above the surface while preventing visual feedback (also known as video feedback, video echo, or visual echo). The technique provides N-way sharing of a surface using video compositing. It also provides for automatic calibration of hardware components, including calibration of any projector, RGB camera, depth camera and any microphones employed by the technique. The technique provides object manipulation with physical, visual, audio, and hover gestures and interaction between digital objects displayed on the surface and physical objects placed on or above the surface. It can capture and scan the surface in a manner that captures or scans exactly what the user sees, which includes both local and remote objects, drawings, annotations, hands, and so forth.

Interactive and shared surfaces

Systems and methods for representing two-dimensional representations as three-dimensional avatars are provided herein. In some examples, one or more input video streams are received. A first subject, within the one or more input video streams, is identified. Based on the one or more input video streams, a first view of the first subject is identified. Based on the one or more input video streams, a second view of the first subject is identified. The first subject is segmented into a plurality of planar object. The plurality of planar objects are transformed with respect to each other. The plurality of planar objects are based on the first and second views of the first subject. The plurality of planar objects are output in an output video stream. The plurality of planar objects provide perspective of the first subject to one or more viewers.

Representing two dimensional representations as three-dimensional avatars

Aspects of the present disclosure relate to distributed virtual reality. In examples, a depth buffer and a color buffer are generated at a presenter device as part of rendering a virtual environment. The virtual environment may be perceived by a user in three dimensions (3D), for example via a virtual reality (VR) headset. Virtual environment information comprising the depth buffer and the color buffer may be transmitted to a viewer device, where it is used to render the virtual environment for display to a viewer. For example, the viewer may similarly view the virtual environment in 3D via a VR headset. A viewer perspective (e.g., from which the virtual environment is generated for the viewer) may differ from a presenter perspective and may be manipulated by the viewer, thereby decoupling the viewer's perception of the virtual environment from that of the presenter.

Distributed virtual reality

A computer device is provided that includes a display device, and a sensor system configured to be mounted adjacent to a user's head and to measure an electrical potential near one or more electrodes of the sensor system. The computer device further includes a processor configured to present a periodic motion-based visual stimulus having a changing motion that is frequency-modulated for a target frequency or code-modulated for a target code, detect changes in the electrical potential via the one or more electrodes, identify a corresponding visual evoked potential feature in the detected changes in electrical potential that corresponds to the periodic motion-based visual stimulus, and recognize a user input to the computing device based on identifying the corresponding visual evoked potential feature.

Periodic motion-based visual stimulus

A 'Concurrent Projector-Camera' uses an image projection device in combination with one or more cameras to enable various techniques that provide visually flicker-free projection of images or video, while real-time image or video capture is occurring in that same space. The Concurrent Projector-Camera provides this projection in a manner that eliminates video feedback into the real-time image or video capture. More specifically, the Concurrent Projector-Camera dynamically synchronizes a combination of projector lighting (or light-control points) on-state temporal compression in combination with on-state temporal shifting during each image frame projection to open a 'capture time slot' for image capture during which no image is being projected. This capture time slot represents a tradeoff between image capture time and decreased brightness of the projected image. Examples of image projection devices include LED-LCD based projection devices, DLP-based projection devices using LED or laser illumination in combination with micromirror arrays, etc.

Real-time mapping of projections onto moving 3D objects

Various embodiments are provided herein for tracking a user's physical environment, to facilitate on-the-fly blending of a virtual environment with detected aspects of the physical environment. Embodiments can be employed to facilitate virtual roaming by compositing virtual representations of detected physical objects into virtual environments. A computing device coupled to a HMD can select portions of a depth map generated based on the user's physical environment, to generate virtual objects that correspond to the selected portions. The computing device can composite the generated virtual objects into an existing virtual environment, such that the user can traverse the virtual environment while remaining aware of their physical environment. Among other things, the computing device can employ various blending techniques for compositing, and further provide image pass-through techniques for selective viewing of the physical environment while remaining fully-immersed in virtual reality.

Blending virtual environments with situated physical reality

A computer-implemented method for determining focus of a user is provided. User input is received. An intention image of a scene including a plurality of interactive objects is generated. The intention image includes pixels encoded with intention values determined based on the user input. An intention value indicates a likelihood that the user intends to focus on the pixel. An intention score is determined for each interactive object based on the intention values of pixels that correspond to the interactive object. An interactive object of the plurality of interactive objects is determined to be a focused object that has the user's focus based on the intention scores of the plurality of interactive objects.

Intention image analysis for determining user focus

A multi-factor probabilistic model evaluates user input to determine if the user input was intended for an on-screen user interface control. When user input is received, a probability is computed that the user input was intended for each on-screen user interface control. The user input is then associated with the user interface control that has the highest computed probability. The probability that user input was intended for each user interface control may be computed utilizing a multitude of factors including the probability that the user input is near each user interface control, the probability that the motion of the user input is consistent with the user interface control, the probability that the shape of the user input is consistent with the user interface control, and that the size of the user input is consistent with the user interface control.

Multi-factor probabilistic model for evaluating user input

Systems and methods related to engaging with a virtual assistant via ancillary input are provided. Ancillary input may refer to non-verbal, non-tactile input based on eye-gaze data and/or eye-gaze attributes, including but not limited to, facial recognition data, motion or gesture detection, eye-contact data, head-pose or head-position data, and the like. Thus, to initiate and/or maintain interaction with a virtual assistant, a user need not articulate an attention word or words. Rather the user may initiate and/or maintain interaction with a virtual assistant more naturally and may even include the virtual assistant in a human conversation with multiple speakers. The virtual assistant engagement system may utilize at least one machine-learning algorithm to more accurately determine whether a user desires to engage with and/or maintain interaction with a virtual assistant. Various hardware configurations associated with a virtual assistant device may allow for both near-field and/or far-field engagement.

Non-verbal engagement of a virtual assistant

Growing community of inventors

Seattle, WA, United States of America

Andrew David Wilson