Microsoft Technology Licensing, LLC

Christian Palmer Larson

Mei Ling Wilson

Sang Kun Choe

Yin Li

Naveen Thumpudi

Fabin Shen

Louis-Philippe Bourret

Mehmet Kucukgoz

Aleksey S Kabanov

Trevor Baron

Michael J Ruete

Sang K Choe

The present disclosure relates to systems, methods, and computer-readable media for selectively identifying image frames from an input video to provide to an image processing model based on camera statistics. For example, systems disclosed herein include receiving an input video and associated camera statistics from a video capturing device. The systems disclosed herein further include identifying select image frames to provide to the image processing model based on the camera statistics and based on an application of the image processing model. The systems disclosed herein further include selectively identifying and providing camera statistics to the image processing model. By selectively providing data to the image processing model based on camera statistics, the systems disclosed herein can leverage capabilities of video capturing devices to significantly reduce the expense of processing resources when utilizing a variety of image processing models.

Pre-processing image frames based on camera statistics

The present disclosure relates to systems, methods, and computer-readable media for selectively identifying pixel data to provide as an input to an image processing model based on motion data associated with the content of a digital video. For example, systems disclosed herein include receiving a compressed digital video and decompressing the compressed digital video to generate a decompressed digital video. The systems disclosed herein further include extracting or otherwise identifying motion data while decompressing the compressed digital video. The systems disclosed herein also include analyzing the motion data to determine a subset of pixel data from the decompressed digital video to provide as input to an image processing model trained to generate an output based on input pixel data.

Selectively identifying data based on motion data from a digital video to provide as input to an image processing model

Described are examples for providing stream augmenting components. A stream augmenting component having an augmentation function indicating a plurality of interfaces to one or more devices and an output stream can be initialized. At least a portion of the plurality of interfaces for the one or more devices can be accessed to at least one of obtain stream output from the one or more devices or control the one or more devices. A plurality of stream outputs from the one or more devices can be virtualized to the output stream at least in part by modifying or correlating data received over the plurality of stream outputs. The stream augmenting component can output the output stream to one or more other stream augmenting components or applications via at least another portion of the one or more interfaces.

Pluggable components for augmenting device streams

Described are examples for providing stream augmenting components. A stream augmenting component having an augmentation function indicating a plurality of interfaces to one or more devices and an output stream can be initialized. One or more functions for the one or more devices to at least one of obtain stream output from the one or more devices or control the one or more devices can be accessed. A plurality of stream outputs from the one or more devices can be virtualized to the output stream at least in part by modifying or correlating data received over the plurality of stream outputs. The output stream can be output to one or more other stream augmenting components or applications.

Described are examples for allocating buffers for multiple components. A stream server can provide an interface to a centralized memory allocator for allocating at least one buffer in a memory to each of the multiple components. The stream server can initialize an instance of the centralized memory allocator based at least in part on a request received from a component of the multiple components via the interface. The stream server can allocate, via the instance of the centralized memory allocator, the at least one buffer for the component in the memory. The stream server can receive, via the instance of the centralized memory allocator, data for storing in the at least one buffer. The stream server can modify the data to generate modified data stored in the at least one buffer.

Centralized memory management for multiple device streams

Methods and devices for allocating memory for use with a camera resources may include receiving camera resource requests from application plurality of applications to use a camera resource. The methods and devices may include determining a memory type to allocate to the applications for the camera resource based on the camera resource request and compatibility information of the camera resource. The methods and devices may include determining a buffer and a buffer type to provide each of the applications based on an access mode of the camera resource, wherein the buffer type comprises one or more of a shared type, a copy type, and a secure type. The methods and devices may include providing the application access to the buffer based on the determination.

Memory allocation type for media buffer

Described are examples for outputting stream data from a plurality of stream sources. A first data stream output for a first stream source and a second data stream output for a second stream source can be received. First data of the first data stream output can be correlated, in time, with second data of the second data stream output. One or more output frames can be output including the first data and the second data as correlated in time.

Growing community of inventors

Kirkland, WA, United States of America

Christian Palmer Larson