Microsoft Technology Licensing, LLC

Microsoft Corporation

Zicheng Liu

Zhengyou Zhang

Michael F Cohen

Charles Edward Jacobs

Philip Andrew Chou

Michael Jack Sinclair

Cha Zhang

Rajesh Hegde

Yinpeng Chen

Alejandro Acero

David W Williams

Hongli Deng

Ross Garrett Cutler

Christian F Huitema

Yuan Kong

Xuedong D Huang

James G Droppo

Li-wei He

Ying Shan

Sasa Junuzovic

Ryan S Menezes

Gabriel Blanco Saldana

Amarnag Subramanya

Kun Luo

Jin Li

Gang Hua

Michel Pahud

William A S Buxton

Qin Cai

Kori Marie Quinn

Fan Yang

Li Deng

Anoop Gupta

Yong Rui

Xin Tong

Michael John Conrad

Ratul Mahajan

Silviu-Petru Cucerzan

David J Kurlander

Behrooz Chitsaz

Chun-Te Chu

Lijuan Wang

Jaeyeon Jung

Nikolaos Karianakis

Kevin Shude Lin

Ying Jin

Liangliang Cao

Jonathan E Paff

Yang Liu

Heung-Yeung Shum

Henrique Sarmento Malvar

Baining Guo

Christopher A Meek

Chunhui Zhang

Mariusz Hieronim Jakubowski

Dinei A Florencio

Yacov Yacobi

Michael Lewis Seltzer

David G Molyneaux

Yuqun Chen

Daniel Allen Rosenfeld

Kyle James Krum

Yasuyuki Matsushita

Yang Wang

Jingyu Qiu

Jay P Kapur

Kori Inkpen Quinn

Eric Chang

Minghua Chen

Emily Yang

Nirupama Chandrasekaran

Qing Zhang

Geoff Hulten

Minmin Gong

Mei Chen

Dongdong Chen

Sean Eron Anderson

John E Paff

Dijia Wu

Lu Yuan

Xu Miao

Javier Hernandez Rivera

Ye Yu

Xiyang Dai

Mengchen Liu

Xuedong David Huang

Norberto Goussies

Hsiang-Tao Wu

Rajesh Hedge

Mike Sinclair

Pei Qun Yu

David DeBarr

Jiang Wang

Yingli Tian

Wanqing Li

Aswin Sankaranarayanan

Jacek Dmochowski

Mingxuan Sun

Ya Chang

Yanli Zheng

A method for environment-specific training of a machine learning model, comprises receiving, for a local environment, a data stream including a plurality of sequential data snippets. Programmed labels are generated for each data snippet using a student version of a machine learning model. A portion of data snippets and associated programmed labels are selected and uploaded to a server-side computing device for evaluation by a teacher version of the machine learning model. An environment-specific training update is received from the server-side computing device. This training update is based on a comparison of the selected programmed labels and pseudolabels generated for the selected portion of data snippets by the teacher version. The environment-specific training update is applied to the student version to generate an updated student version. The updated student version of the machine learning model is then used to generate programmed labels for newly received data snippets.

Environment-specific training of machine learning models

Improved techniques for determining an object's 3D orientation. An image is analyzed to identify a 2D object and a first set of key points. The first set defines a first polygon. A 3D virtual object is generated. This 3D virtual object has a second set of key points defining a second polygon representing an orientation of the 3D virtual object. The second polygon is rotated a selected number of times. For each rotation, each rotated polygon is reprojected into 2D space, and a matching score is determined between each reprojected polygon and the first polygon. A specific reprojected polygon is selected whose corresponding matching score is lowest. The orientation of the 3D virtual object is set to an orientation corresponding to the specific reprojected polygon. Based on the orientation of the 3D virtual object, an area of focus of the 2D object is determined.

Determining an object's 3D orientation from a single camera's image

A computer device for automatic feature detection comprises a processor, a communication device, and a memory configured to hold instructions executable by the processor to instantiate a dynamic convolution neural network, receive input data via the communication network, and execute the dynamic convolution neural network to automatically detect features in the input data. The dynamic convolution neural network compresses the input data from an input space having a dimensionality equal to a predetermined number of channels into an intermediate space having a dimensionality less than the number of channels. The dynamic convolution neural network dynamically fuses the channels into an intermediate representation within the intermediate space and expands the intermediate representation from the intermediate space to an expanded representation in an output space having a higher dimensionality than the dimensionality of the intermediate space. The features in the input data are automatically detected based on the expanded representation.

Dynamic matrix convolution with channel fusion

To improve the accuracy and efficiency of object detection through computer digital image analysis, the detection of some objects can inform the sub-portion of the digital image to which subsequent computer digital image analysis is directed to detect other objects. In such a manner object detection can be made more efficient by limiting the image area of a digital image that is analyzed. Such efficiencies can represent both computational efficiencies and communicational efficiencies arising due to the smaller quantity of digital image data that is analyzed. Additionally, the detection of some objects can render the detection of other objects more accurate by adjusting confidence thresholds based on the detection of those related objects. Relationships between objects can be utilized to inform both the image area on which subsequent object detection is performed and the confidence level of such subsequent object detection.

Focused computer detection of objects in images

Examples are disclosed herein that relate to automatically calibrating cameras based on human detection. One example provides a computing system comprising instructions executable to receive image data comprising depth image data and two-dimensional image data of a space from a camera, detect a person in the space via the image data, determine a skeletal representation for the person via the image data, determine over a period of time a plurality of locations at which a reference point of the skeletal representation is on a ground area in the image data, determine a ground plane of the three-dimensional representation based upon the plurality of locations at which the reference point of the skeletal representation is on the ground area in the image data, and track a location of an object within the space relative to the ground plane.

Calibrating cameras using human skeleton

Techniques for improved camera calibration are disclosed. An image is analyzed to identify a first set of key points for an object. A virtual object is generated. The virtual object has a second set of key points. A reprojected version of the second set is fitted to the first set in 2D space until a fitting threshold is satisfied. To do so, a 3D alignment of the second set is generated in an attempt to fit (e.g., in 2D space) the second set to the first set. Another operation includes reprojecting the second set into 2D space. In response to comparing the reprojected second set to the first set, another operation includes determining whether a fitting error between those sets satisfies the fitting threshold. A specific 3D alignment of the second set is selected. The camera is calibrated based on resulting reprojection parameters.

Auto calibrating a single camera from detectable objects

An object re-identifier. For each of a plurality of frames of a video, a quality of the frame is assessed and a confidence that a previously-recognized object is present in the frame is determined. The determined confidence for the frame is weighted based on the assessed quality of the frame such that frames with higher relative quality are weighted more heavily than frames with lower relative quality. An overall confidence that the previously-recognized object is present in the video is assessed based on the weighted determined confidences.

Depth-based object re-identification

A computing system is provided. The computing system includes a processor configured to execute a convolutional neural network that has been trained, the convolutional neural network including a backbone network that is a concatenated pyramid network, a plurality of first head neural networks, and a plurality of second head neural networks. At the backbone network, the processor is configured to receive an input image as input and output feature maps extracted from the input image. The processor is configured to: process the feature maps using each of the first head neural networks to output corresponding keypoint heatmaps; process the feature maps using each of the second head neural networks to output corresponding part affinity field heatmaps; link the keypoints into one or more instances of virtual skeletons using the part affinity fields; and output the instances of the virtual skeletons.

Neural network for skeletons from input images

The large display interaction implementations described herein combine mobile devices with people tracking to enable new interactions including making a non-touch-sensitive display touch-sensitive and allowing personalized interactions with the display. One implementation tracks one or more mobile computing device users relative to a large computer-driven display, and configures content displayed on the display based on a distance a given mobile computing device user is from the display. Another implementation personalizes user interactions with a large display. One or more mobile computing device users are tracked relative to a display. The identity of each of the one or more mobile computing device users is obtained. Content displayed on the display is configured based on a distance an identified mobile computing device user is from the display and the identity of the user that provides the content.

Combining mobile devices with people tracking for large display interactions

An apparatus that automatically monitors a display device includes a photo sensor configured to receive light from a display screen of the display device. The photo sensor provides signals representing detected light levels to a processor. The processor is coupled to the display device and is configured to cause the display device to present a test sequence including a plurality of images on the display screen. The processor is configured to capture data from the photo sensor during the presentation of the test sequence and to compare the captured data to an expected sequence corresponding to the test sequence displayed by a well-functioning display. The processor is further configured to report any mismatch between the captured data and the expected sequence as a possible malfunction of the display device.

Automated presentation equipment testing

Growing community of inventors

Bellevue, WA, United States of America

Zicheng Liu