Microsoft Technology Licensing, LLC

Matthew Richard Wozniak

Bakkama Srinath Reddy

Shyam Sadhwani

Yongjun Wu

Mehmet Kucukgoz

Glenn Frederick Evans

Vladislav Alexandrov

David H Mebane

Dibyajyoti Roy

Amr Martini

Naveen Thumpudi

Simon John Baker

Kevin A Chin

Yuechuan Li

Holly Helene Pollock

Isuru Chamara Pathirana

Rafael Vincent Prieto Vertido

Catalin Alexandru Negrila

Venkata S K Kamal Lanka

Ian Glen Neal

Brian Kohlwey

Marcin Stankiewicz

Anton Victor Polinger

Sandeep Kanumuri

Abhijit Sarkar

Kumar Rajeev

Jesse Hakanen

Victor Cherepanov

Jitesh Krishnan

Isha Sharma

Aaron Oneal

Sang Choe

The techniques disclosed herein enable a guest operating system (OS) to access and use a color space conversion component on a host OS. The guest OS provides, via an application programming interface, a request for the host OS to generate media data in a color space format that is used by the guest OS. To generate the media data, the host OS uses a color space conversion component on the host OS, which is more performant than a corresponding color space conversion component on the guest OS because the color space conversion component on the host OS has access to hardware-accelerated functionality. Accordingly, the color space conversion component on the host OS converts media data into the color space format that is used by the guest OS, and stores the media data in memory that is accessible to the guest OS.

Providing host media processing functionality to a guest operating system

Innovations in stream configuration operations and playback operations for adaptive streaming of panoramic video are described. The innovations include features of adaptive streaming of panoramic video with composite pictures. For example, a stream configuration tool splits an input picture of panoramic video into multiple sections and creates multiple composite pictures. A composite picture includes one of the sections as well as a low-resolution version of the input picture. A playback tool reconstructs one or more composite pictures. Under normal operation, the playback tool can use the reconstructed section(s) of the composite picture(s) to render high-quality views of the panoramic video. If the view window dramatically changes, however, or if encoded data for a section is lost or corrupted, the playback tool can use the low-resolution version of the input picture to render lower-quality details for views of the panoramic video, without disruption of playback.

Adaptive panoramic video streaming using composite pictures

The techniques disclosed herein enable a guest operating system (OS) to access and use a media processing component configured on a host OS. The guest OS provides, via an API, a request to create an instance of the media processing component (e.g., a codec, an encryption/decryption component, a DRM component). In association with the request, the guest OS allocates space in memory for media data that is to be processed by the instance of the media processing component configured on the host OS. The guest OS stores the input media data in the allocated memory and provides, via the API, reference(s) to locations of the allocated memory. The reference(s) to the locations of the allocated memory enable the host OS to retrieve the input media data and process the input media data using the instance of the media processing component configured on the host OS.

A computing device is provided, including a display and a processor. The processor may, for each frame of a plurality of frames included in a video, determine a brightness characterization for a region of interest of the frame. The processor may determine one or more additional brightness characterizations for the region of interest of one or more additional frames of the plurality of frames. The processor may determine that a high brightness condition is present in the region of interest. The processor may generate a filter for the frame based on the brightness characterization and the one or more additional brightness characterizations. The processor may apply the filter to the region of interest of the frame to generate a filtered frame. The high brightness condition is not present in the region of interest of the filtered frame. The processor may output the filtered frame for display at the display.

Video frame brightness filter

A computing device is configured to selectively adjust a brightness of a video based on a user-specific response to light. A user-specific light response parameter is determined. For each frame of a plurality of frames included in the video, (1) a brightness characterization for the frame is determined, (2) one or more additional brightness characterizations for one or more additional frames of the plurality of frames is determined, (3) a filter for the frame is generated based on the brightness characterization, the one or more additional brightness characterizations, and the user-specific light response parameter, (4) the filter is applied to the frame to generate a filtered frame, and (5) the filtered frame is output by the computing device. The filter is configured such that if a high brightness condition is present in the frame, then the high brightness condition is mitigated or removed from the filtered frame.

User-specific video frame brightness filter

Innovations in stream configuration operations and playback operations for adaptive streaming of panoramic video are described. The innovations include features of adaptive streaming of panoramic video with overlapping sections. For example, a stream configuration tool splits an input picture of panoramic video into overlapping sections. A playback tool reconstructs one or more of the sections. Because of overlap between adjacent sections, for gradual changes in a view window, the playback tool can, in effect, preemptively fetch a new section as the view window moves out of a current section into the new section. By the time the view window reaches a non-overlapping part of the adjacent, new section, the playback tool has retrieved encoded data for the new section and can reconstruct the new section. This hides network latency and stream switching latency from the viewer, and disruption of playback is avoided.

Adaptive panoramic video streaming using overlapping partitioned sections

Innovations in reconstruction and rendering of panoramic video are described. For example, a view-dependent operation controller of a panoramic video playback system receives an indication of a view direction for an application and, based at least in part on the view direction, identifies a section of a picture of panoramic video in an input projection. The view-dependent operation controller limits operations of a color converter, video decoder, and/or streaming controller to the identified section. In this way, the panoramic video playback system can avoid performing operations to reconstruct sections of the picture of panoramic video that will not be viewed. As another example, a mapper of a panoramic video playback system re-projects at least some sample values in an input flat projection towards a center location for a view direction, producing an output flat projection, which an application can use to generate one or more screen projections.

Re-projecting flat projections of pictures of panoramic video for rendering by application

Innovations in reconstruction and rendering of panoramic video are described, including the use of a platform rendering engine to provide a screen projection based on a view direction specified for an application through an interface. For example, based at least in part on the view direction specified for the application, at least a section of panoramic video in an input projection is identified. At least some of sample values of the at least a section of the picture of panoramic video in the input projection are mapped to a screen projection. The screen projection is output for display to a buffer for the application. Thus, an application may use panoramic video, including updating a view direction, without itself having to render a screen projection for the panoramic video.

Interface for application-specified playback of panoramic video

View-dependent operations during playback of panoramic video

Innovations in rendering of high dynamic range ('HDR') video on a display device having enhanced dynamic range ('EDR'). The peak brightness for an EDR display device is lower than the peak brightness for a typical HDR display device but higher than the peak brightness for a typical display device having standard dynamic range. The increased range of brightness values in an EDR display device can be utilized effectively to show bright highlights of the HDR video. For example, decision logic is configured to evaluate a peak brightness of a target display device and select an HDR-to-EDR tone mapping mode. A tone mapper is configured to apply tone mapping to input values for the sample values of HDR video, according to a tone mapping function, thereby producing output values for sample values of EDR video.

Architecture for rendering high dynamic range video on enhanced dynamic range display devices

In one embodiment, a video processing systemmay filter a video data set to correct skew and wobble using a central processing unitand a graphical processing unit. The video processing systemmay apply a rolling shutter effect correction filter to an initial version of a video data set. The video processing systemmay simultaneously apply a video stabilization filter to the initial version to produce a final version video data set.

Joint video stabilization and rolling shutter correction on a generic platform

Innovations in rendering of high dynamic range ('HDR') content are described. A playback system can adjust a tone mapping function used when rendering the HDR content, thereby improving quality when rendering the HDR content. For example, the playback system includes decision logic, which is configured to receive summary information (e.g., histogram information) for sample values of a picture of HDR content, and set a tone mapping function based at least in part on the summary information. The tone mapping function can be selected from among multiple available tone mapping functions, and parameters of the selected tone mapping function can be determined based at least in part on the summary information. A tone mapper is configured to apply tone mapping to input values for the sample values of the picture of HDR content, according to the tone mapping function, and thereby produce output values.

Content-adaptive adjustments to tone mapping operations for high dynamic range content

Growing community of inventors

Bellevue, WA, United States of America

Matthew Richard Wozniak