Microsoft Technology Licensing, LLC

Peter Bodik

Paramvir Bahl

Ganesh Ananthanarayanan

Matthai Philipose

Srikanth Kandula

Shane Kumar Mainali

Moises Goldszmidt

Ishai Menache

Rushi Srinivas Surla

Tsuwang Hsieh

Qifan Pu

Srinivasa Aditya Akella

Yang Lu

Lenin Ravindranath Sivalingam

Eric Boutin

Hans Christian Andersen

Donald Frank Brinkman, Jr

Max Artemov

Haoyu Zhang

Chien-Chun Hung

Andrew D Ferguson

Suvamsh Shivaprasad

Donald Frank Brinkman

A method can include classifying, using a compressed and specialized convolutional neural network (CNN), an object of a video frame into classes, clustering the object based on a distance of a feature vector of the object to a feature vector of a centroid object of the cluster, storing top-k classes, a centroid identification, and a cluster identification, in response to receiving a query for objects of class X from a specific video stream, retrieving image data for each centroid of each cluster that includes the class X as one of the top-k classes, classifying, using a ground truth CNN (GT-CNN), the retrieved image data for each centroid, and for each centroid determined to be classified as a member of the class X providing image data for each object in each cluster associated with the centroid.

Querying video data with reduced latency and cost

Described herein is a mechanism for creating a dynamic video highlight from a plurality of video segments. A metadata collection agent collects metadata comprising attributes about a video, segments within the video where one or more events occur, attributes about the creator of the video and so forth. The metadata is collected and used to create highlight video definitions comprising a set of metadata attribute-value pairs. The highlight video definitions can be created in an interactive fashion by presenting a user interface allowing selection of a combination of attribute-value pairs to include/exclude segments from the definition and/or manual selection of custom segments. The highlight video definitions can be stored and/or shared among users. The video highlight definitions are utilized to instantiate one or more video players to play the video segments in an identified order without assembling the video segments into a separate video.

Dynamic video highlight

In an embodiment, a partition cost of one or more of the plurality of partitions and a data block cost for one or more data blocks that may be subjected to a garbage collection operation are determined. The partition cost and the data block cost are combined into an overall reclaim cost by specifying both the partition cost and the data block cost in terms of a computing system latency. A byte constant multiplier that is configured to modify the overall reclaim cost to account for the amount of data objects that may be rewritten during the garbage collection operation may be applied. The one or more partitions and/or one or more data blocks that have the lowest overall reclaim cost while reclaiming an acceptable amount of data block space may be determined and be included in a garbage collection schedule.

Cost-based garbage collection scheduling in a distributed storage environment

Latency in responding to queries directed to geographically distributed data can be reduced by allocating individual steps, of a multi-step compute operation requested by the query, among the geographically distributed computing devices so as to reduce the duration of shuffling of intermediate data among such devices, and, additionally, by pre-moving, prior to the receipt of the query, portions of the distributed data that are input to a first step of the multistep compute operation, to, again, reduce the duration of the exchange of intermediate data. The pre-moving of input data occurring, and the adaptive allocation of intermediate steps, are prioritized for high-value data sets. Additionally, a threshold increase in a quantity of data exchanged across network communications can be established to avoid incurring network communication usage without an attendant gain in latency reduction.

Latency reduction with pre-moving of distributed data and adaptive allocating of compute operations

Various technologies described herein pertain to performing video analytics. The approaches set forth herein support live video analytics at scale with approximate and delay-tolerant processing. Video streams can be captured by multiple cameras and continuously streamed to a video analytics computing system; the video streams can be received at the video analytics computing system. Multiple video analytics queries can be executed on the video streams. The multiple video analytics queries can be concurrently executed by the video analytics computing system on the video streams as the video streams are continuously streamed to the video analytics computing system. The multiple video analytics queries can be executed utilizing resources of the video analytics computing system allocated between the multiple video analytics queries. Execution of the multiple video analytics queries can return respective results for the multiple video analytics queries. The results for the multiple video analytics queries can be outputted.

Live video analytics at scale

A global manager communicates with various local managers to receive and process video queries. The video queries identify components that process live video streams, placement options for where the components of the video query may be executed, and various video query plans. The video query plans include options such as framerate and video quality. As the global manager processes the video queries, the global manager determines an initial set of video query configurations that identify a video query plan and placement option for each component of a given video query. Using the initial set of video query configurations, the global manager then determines an optimal set of video query configurations for the received set of video queries. The global manager communications instructions to the local managers to execute the components of the video queries using the video query plans and placement options from the optimal set of video query configurations.

Processing live video streams over hierarchical clusters

A computing cluster operated according to a resource allocation policy based on a predictive model of completion time. The predictive model may be applied in a resource control loop that iteratively updates resources assigned to an executing job. At each iteration, the amount of resources allocated to the job may be updated based on of the predictive model so that the job will be scheduled to complete execution at a target completion time. The target completion time may be derived from a utility function determined for the job. The utility function, in turn, may be derived from a service level agreement with service guarantees and penalties for late completion of a job. Allocating resources in this way may maximize utility for an operator of the computing cluster while minimizing disruption to other jobs that may be concurrently executing.

Computing cluster with latency control

Methods for automatically identifying and classifying a crisis state occurring in a system having a plurality of computer resources. Signals are received from a device that collects the signals from each computer resource in the system. For each epoch, an epoch fingerprint is generated. Upon detecting a performance crisis within the system, a crisis fingerprint is generated consisting of at least one epoch fingerprint. The technology is able to identify that a performance crisis has previously occurred within the datacenter if a generated crisis fingerprint favorably matches any of the model crisis fingerprints stored in a database. The technology may also predict that a crisis is about to occur.

Automated identification of performance crisis

The present invention extends to methods, systems, and computer program products for automatically generating and refining health models. Embodiments of the invention use machine learning tools to analyze historical telemetry data from a server deployment. The tools output fingerprints, for example, small groupings of specific metrics-plus-behavioral parameters, that uniquely identify and describe past problem events mined from the historical data. Embodiments automatically translate the fingerprints into health models that can be directly applied to monitoring the running system. Fully-automated feedback loops for identifying past problems and giving advance notice as those problems emerge in the future is facilitated without any operator intervention. In some embodiments, a single portion of expert knowledge, for example, Key Performance Indicator (KPI) data, initiates health model generation. Once initiated, the feedback loop can be fully automated to access further telemetry and refine health models based on the further telemetry.

Growing community of inventors

Kirkland, WA, United States of America

Peter Bodik