Microsoft Technology Licensing, LLC

Ganesh Ananthanarayanan

Paramvir Bahl

Yuanchao Shu

Peter Bodik

Matthai Philipose

Ramarathnam R Venkatesan

Srikanth Kandula

Sean Blagsvedt

Tsuwang Hsieh

Landon Prentice Cox

Yu Yan

Nikolaos Karianakis

Tsu-wang Hsieh

Romil Bhardwaj

Qifan Pu

Srinivasa Aditya Akella

Philip Andrew Chou

Albert Gordon Greenberg

Kentaro Toyama

Yi Lu

Venkata N Padmanabhan

Bozidar Radunovic

Edward Bryan Cutrell

Manikanta Kotaru

Xenofon Foukas

Stavros Volos

Daehyeok Kim

Martin Ellis

Eric M Lawrence

Yoganand Rajasekaran

Srinath T V Setty

Edward David Harris

Zhuqi Li

Kiran Muthabatulla

Anand Padmanabha Iyer

Sunava Dutta

Rajdeep Das

Bikas Saha

Swagath Kannan

Michael Harrison Rudow

Haoyu Zhang

Rishabh Poddar

Chien-Chun Hung

Junchen Jiang

Longfei Shangguan

Arthi Hema Padmanabhan

Srinath TV Setty

A context analysis system receives a query from a user. The context analysis system generates one or multiple context profiles and generates a prompt for a foundation model for each of the context profiles. The context analysis system analyzes each of the context profiles and generates a relevancy score. The context analysis system selects one of the context profiles based on the relevancy score. In some examples, the context analysis system iteratively determines predicted latencies and relevancies of processing a query in conjunction with a generated context and, based on the predicted latencies and/or relevancies, processes the query using a foundation model, such as a large language model (LLM).

Generating and implementing context profiles in processing queries using foundation models

A real-time radio intelligent controller (RIC) executes in parallel with one or more virtual radio access network functions to provide real-time analytics and control of the virtual radio access network functions. At least a first processor core is configured to execute a radio network virtual function. The radio network virtual function is configured with a codelet to output selected operational data to a first stream associated with a first stream ID and receive control information from a control stream associated with a second stream ID. At least a second processor core is configured to execute the real-time RIC isolated from the at least the first processor core. The real-time RIC includes one or more dynamically loaded programs configured to: access the first stream; perform processing on the operational data; and write commands for the radio network virtual function to the control stream.

Real-time radio intelligent controller

Systems and methods are provided for continuous learning of models across hierarchies under a multi-access edge computing. In particular, an on-premises edge server, using a model, generates inference data associated with captured stream data. A data drift determiner determines a data drift in the inference data by comparing the data against reference data generated using a golden model. The data drift indicates a loss of accuracy in the inference data. A gateway model maintains one or more models in a model cache for update the model. The gateway model instructs the one or more servers to train the new model. The gateway model transmits the trained model to update the model in the on-premises edge server. Training the new model includes determining an on-premises edge server with computing resources available to train the new model while generating other inference data for incoming stream data in the data analytic pipeline.

Continuous learning models across edge hierarchies

Systems and methods are provided for performing privacy transformation of data to protect privacy in data analytics under the multi-access edge computing environment. In particular, a policy receiver in an edge server receives privacy instructions. Inference determiner in the edge server in a data analytics pipeline receives data from an IoT device and evaluates the data to recognize data associated with personally identifiable information. Privacy data transformer transforms the received data with inference for protecting data privacy by preventing exposure of private information from the edge server. In particular, the privacy data transformer dynamically selects a technique among techniques for removing information that is subject to privacy protection and transforms the received data using the technique. The techniques includes reducing resolution of image data such that inference enables object recognition without sufficient details to prevent other servers in the data analytics pipeline to determine identifies of the object deeper inferences.

Privacy transformations in data analytics

The techniques described herein use an edge device to manage the security for a data stream being ingested by a tenant and a cloud platform. The creation of the data stream for ingestion occurs in an environment that is trusted by a tenant (e.g., an on-premises enterprise network). The cloud platform that is part of the data stream ingestion process is outside this trusted environment, and thus, the tenant loses an element of security when ingesting data streams for cloud storage and/or cloud processing. Accordingly, the edge device is configured on a trust boundary so that the data stream ingestion process associated with a cloud platform is secured, or trusted by the tenant. The edge device is configured to encrypt the data stream using a data encryption key and/or manage the protection of the data encryption key.

Secure data ingestion with edge computing

Provided are aspects relating to methods and computing devices for allocating computing resources and selecting hyperparameter configurations during continuous retraining and operation of a machine learning model. In one example, a computing device configured to be located at a network edge between a local network and a cloud service includes a processor and a memory storing instructions executable by the processor to operate a machine learning model. During a retraining window, a selected portion of a video stream is selected for labeling. At least a portion of a labeled retraining data set is selected for profiling a superset of hyperparameter configurations. For each configuration of the superset of hyperparameter configurations, a profiling test is performed. The profiling test is terminated, and a change in inference accuracy that resulted from the profiling test is extrapolated. Based upon the extrapolated inference accuracies, a set of selected hyperparameter configurations is output.

Allocating computing resources during continuous retraining

Systems and methods are provided for reducing stream data according to a data streaming protocol under a multi-access edge computing. In particular, an IoT device, such as a video image sensing device, may capture stream data and generate inference data by applying a machine-learning model trained to infer data based on the captured stream data. The inference data represents the captured stream data in a reduced data size based on performing data analytics on the captured data. The IoT device formats the inference data according to the data streaming protocol. In contrast to video data compression, the data streaming protocol includes instructions for transmitting the reduced volume of inference data through a data analytics pipeline.

Data streaming protocols in edge computing

Computing resources are managed in a computing environment comprising a computing service provider and an edge computing network. The edge computing network comprises computing and storage devices configured to extend computing resources of the computing service provider to remote users of the computing service provider. The edge computing network collects capacity and usage data for computing and network resources at the edge computing network. The capacity and usage data is sent to the computing service provider. Based on the capacity and usage data, the computing service provider, using a cost function, determines a distribution of workloads pertaining to a processing pipeline that has been partitioned into the workloads. The workloads can be executed at the computing service provider or the edge computing network.

Orchestrating edge service workloads across edge hierarchies

Identifying, by a sender and for each frame i of a plurality of frames of a video stream, a partition of a set of video data symbols D[i] into a first set of video data symbols U[i] and a second set of video data symbols V[i]. Generating, by the sender and for each frame i, a set of one or more streaming forward error correction (FEC) code parity symbols P[i] based on the symbols: V[i&#x2212;&#x3c4;] through V[i&#x2212;1], U[i&#x2212;&#x3c4;], and the symbols D[i], wherein &#x3c4; is a function of a maximum tolerable latency of the video stream expressed as a whole number of frames. Encoding, by the sender and for each frame i, packets carrying the symbols D[i], and P[i]. Transmitting, by the sender, each frame i of encoded packets in frame order to one or more receivers.

Loss recovery using streaming codes in forward error correction

Methods and computing devices for allocating computing resources and selecting hyperparameter configurations during continuous retraining and operation of a machine learning model. In one example, a computing device configured to be located at a network edge between a local network and a cloud service includes a processor and a memory storing instructions executable by the processor to operate a machine learning model. During a retraining window, a selected portion of a video stream is selected for labeling. At least a portion of a labeled retraining data set is selected for profiling a superset of hyperparameter configurations. For each configuration of the superset of hyperparameter configurations, a profiling test is performed. The profiling test is terminated, and a change in inference accuracy that resulted from the profiling test is extrapolated. Based upon the extrapolated inference accuracies, a set of selected hyperparameter configurations is output.

A method can include classifying, using a compressed and specialized convolutional neural network (CNN), an object of a video frame into classes, clustering the object based on a distance of a feature vector of the object to a feature vector of a centroid object of the cluster, storing top-k classes, a centroid identification, and a cluster identification, in response to receiving a query for objects of class X from a specific video stream, retrieving image data for each centroid of each cluster that includes the class X as one of the top-k classes, classifying, using a ground truth CNN (GT-CNN), the retrieved image data for each centroid, and for each centroid determined to be classified as a member of the class X providing image data for each object in each cluster associated with the centroid.

Querying video data with reduced latency and cost

A multi-hop relay network comprises a set of data nodes arranged in a relay topology, wherein the set of data nodes communicate with one another via a data plane comprising a set of high frequency data links, such as mmWave links. An edge node communicates with one or more of the set of data nodes via the data plane and communicates with the set of data nodes over a control plane comprising a set of low frequency wireless links, such as a Wi-Fi network. The edge node determines a path utilization for the set of high frequency wireless links. When one of the high frequency wireless links is over-utilized, the edge node communicates, to a data node in the set of data nodes via the control plane, a command to change the relay topology. The edge node also determines whether the relay topology is operating at a target accuracy. When it is not, the edge node adjusts a data analytics parameter for a node to improve the overall accuracy of the network.

Live video analytics over high frequency wireless networks

This document relates to performing video analytics on a cloud device that preserves privacy. One example uses data-oblivious algorithms to process input video data, where the data-oblivious algorithms can assist with preventing attackers from exploiting side-channels induced by data-dependent access patterns.

Private data analytics

Growing community of inventors

Bellevue, WA, United States of America

Ganesh Ananthanarayanan