Microsoft Technology Licensing, LLC

Microsoft Technology Licenisng, LLC

Ryan S Menezes

Hongli Deng

Gabriel Blanco Saldana

Zicheng Liu

Martin Ellis

Juri Reitel

Andrei Birjukov

ZhiCheng Miao

Taroon Mandhana

Aaron Wesley Cunningham

Brandon V Taylor

Austin Seungmin Lee

Henry Yao-Tsu Chen

Mark Robert Swift

Joseph Milan Filcik

Dhiraj P Gandhi

Shankar Seal

Ali Reza Emami

Hao Yan

Tony Ducheng Jin

Ming-Chieh Lee

Jason Thomas Faulkner

Abhishek Abhishek

Shawn Crispin Wright

Sriram Srinivasan

Mei-Hsuan Lu

Hongwu Huai

Bofan Hsu

Robert Aichner

Ermin Kozica

Chenyang Li

Schuaib Rasool

Shuaib Rasool

Duong Nguyen

Zhicheng Miao

Improved techniques for determining an object's 3D orientation. An image is analyzed to identify a 2D object and a first set of key points. The first set defines a first polygon. A 3D virtual object is generated. This 3D virtual object has a second set of key points defining a second polygon representing an orientation of the 3D virtual object. The second polygon is rotated a selected number of times. For each rotation, each rotated polygon is reprojected into 2D space, and a matching score is determined between each reprojected polygon and the first polygon. A specific reprojected polygon is selected whose corresponding matching score is lowest. The orientation of the 3D virtual object is set to an orientation corresponding to the specific reprojected polygon. Based on the orientation of the 3D virtual object, an area of focus of the 2D object is determined.

Determining an object's 3D orientation from a single camera's image

A method to predict a traversal-time interval for traversal of a service queue comprises receiving video of a region including the service queue, recognizing in the video, via machine vision, a plurality of persons awaiting service within the region, estimating an average crossing-time interval between successive crossings, by the plurality of persons, of a fixed boundary along the service queue, wherein such estimating is based on features of the service queue and of the one or more persons awaiting service, and returning an estimate of the traversal-time interval based on a count of the persons awaiting service and on the average crossing-time interval as estimated.

Machine-vision person tracking in service environment

Techniques for inferring whether an event is occurring in 3D space based on 2D image data and for maintaining a camera's calibration are disclosed. An image of an environment is accessed. Input is received, where the input includes a 2D rule imposed against a ground plane. The 2D rule includes conditions indicative of an event. A bounding box is generated and encompasses a detected object. A point within the bounding box is projected from a 2D-space image plane of the image into 3D space to generate a 3D-space point. Based on the 3D-space point, a 3D-space ground contact point is generated. That 3D-space ground contact point is reprojected onto the ground plane of the image to generate a synthesized 2D ground contact point. A location of the synthesized 2D ground contact point is determined to satisfy the conditions.

Accurate video event inference using 3D information

The description relates to cameras, and camera calibration for enhancing user experiences. One example can receive a first image of a user at a first location relative to a camera. The first image can include the user's upper body but does not include the user from head to toe. The example can receive a second image of the user at a second location relative to a camera. The second image can include the user's upper body but does not include the user from head to toe. The example can estimate a distance of the second location from the first location relative to the camera and calibrate a height and tilt angle of the camera from the first image, the second image, and the estimated distance and without a full body image of the user.

Camera calibration

The description relates to cameras, such as security cameras, and providing guidance for positioning cameras to achieve desired goals. One example can receive an image of a scene overlaid with transparent indicators that reflect accuracy of object detection in individual regions of the image. The example can correlate input received from the user on the display relative to the regions. The example can analyze subsequent images of the scene with rules derived from the input from the user.

Camera placement guidance

Techniques for improved camera calibration are disclosed. An image is analyzed to identify a first set of key points for an object. A virtual object is generated. The virtual object has a second set of key points. A reprojected version of the second set is fitted to the first set in 2D space until a fitting threshold is satisfied. To do so, a 3D alignment of the second set is generated in an attempt to fit (e.g., in 2D space) the second set to the first set. Another operation includes reprojecting the second set into 2D space. In response to comparing the reprojected second set to the first set, another operation includes determining whether a fitting error between those sets satisfies the fitting threshold. A specific 3D alignment of the second set is selected. The camera is calibrated based on resulting reprojection parameters.

Auto calibrating a single camera from detectable objects

In a device including a processor and a memory in communication with the processor is described, the memory includes executable instructions that, when executed by the processor, cause the processor to control the device to perform functions of: generating, based on a plurality of local 3D models, a global 3D model representing a portion of a real-world environment; determining a location of a 3D virtual object in the global 3D model; and generating augmentation data for rendering the 3D virtual object to be seen at a location of the real-world environment corresponding to the location of the 3D virtual object in the global 3D model.

Augmented reality

A user device comprises a network interface, a rendering module, and a scene modification module. The network interface is configured to receive a video signal from another device via a network. The rendering module is configured to control display apparatus of the user device to display a virtual element to a user of the user device, the virtual element comprising a video image derived from the video signal. The modification module is configured to generate rendering data for displaying a modified version of the virtual element at the other device. The modified version does not include said video image. The network interface is configured to transmit the rendering data to the other device via the network. Alternatively or in addition, the rendering data can be modified at the other device to the same end.

Communication system

An augmented reality (AR) system receives a plurality of local 3D models of a part of a real-world environment, each having been generated by a different AR device when located in the real-world environment. The local 3D models are combined to generate a global 3D model, at least part of which is transmitted to a device remote from the real-world environment. The global 3D model represents a greater portion of the real-environment than any of the local 3D models individually. The AR system receives rendering data from the remote device, and transmits it to an AR device when the AR device is located in the real-world environment. Alternatively, the rendering data may be transmitted from the remote device to the AR device via a network directly. The rendering data is for use in rendering a virtual object at the AR device in the real-world environment.

A user device within a communication architecture, the user device comprising: an image capture device configured to determine image data for the creation of a video channel defining the shared scene; an intrinsic/extrinsic data determiner configured to determine intrinsic/extrinsic capture device data associated with the image capture device; and a video encoder configured to encode the image data and intrinsic/extrinsic capture device data within the video channel.

Shared scene mesh data synchronisation

Within a communication system, packets are received for a video channel defining a shared scene and a surface reconstruction channel associated with surface reconstruction data of image data. The image data and intrinsic and extrinsic capture device data are from an image capture device and are encoded and packetized within the video channel. The surface reconstruction data is encoded and packetized within the surface reconstruction channel. The image data and the intrinsic and extrinsic capture device data are extracted from within the video channel. The surface reconstruction data is extracted from within the surface reconstruction channel. The shared scene is rendered using the image data, the intrinsic and extrinsic capture device data, and the surface reconstruction data.

Shared scene mesh data synchronization

Growing community of inventors

Woodinville, WA, United States of America

Ryan S Menezes