Sri International

Latent Ai, Inc.

Syracuse University

Sri Inlernational

Sek Meng Chai

Aswin Nadamuni Raghavan

David Chao Zhang

Michael Raymond Piacentino

Michael Anthony Isnardi

John Robertson Tower

Mohamed R Amer

Timothy J Shields

Jesse Hostetler

Jagadeesh Kandasamy

Peter Alan Levine

Bhaskar S Ramamurthy

Ruby B Lee

Peter Jeffrey Burt

Girish Acharya

Douglas A Bercow

Scott Oberg

John Benjamin Southall

Gooitzen Sieman Van Der Wal

Michael E Locasto

Zecheng He

Aaron Spaulding

Erik Frank Matlin

Thomas Lee Vogelsong

Senem Velipasalar

David Berends

Samyak Parajuli

Indranil Sur

Abrar Abdullah Rahman

Nicholas Vitovitch

The disclosed embodiments relate to a system that generates and executes a deep neural network (DNN) based on target runtime parameters. During operation, the system receives a trained original model and a set of target runtime parameters for the DNN, wherein the target runtime parameters are associated with one or more of the following for the DNN: desired operating conditions, desired resource utilization, and desired accuracy of results. Next, the system generates a context-specific model based on the original model and the set of target runtime parameters. The system also generates an operational plan for executing both the original model and the context-specific model to meet requirements of the target runtime parameters. Finally, the system controls execution of the original model and the context-specific model based on the operational plan.

Generating and executing context-specific neural network models based on target runtime parameters

The disclosed embodiments relate to a system that optimizes execution of a DNN based on operational performance parameters. During operation, the system collects the operational performance parameters from the DNN during operation of the DNN, wherein the operational performance parameters include parameters associated with operating conditions for the DNN, parameters associated with resource utilization during operation of the DNN, and parameters associated with accuracy of results produced by the DNN. Next, the system uses the operational performance parameters to update the DNN model to improve performance and efficiency during execution of the DNN.

Optimizing execution of a neural network based on operational performance parameters

Artificial neural network systems involve the receipt by a computing device of input data that defines a pattern to be recognized (such as faces, handwriting, and voices). The computing device may then decompose the input data into a first subband and a second subband, wherein the first and second subbands include different characterizing features of the pattern in the input data. The first and second subbands may then be fed into first and second neural networks being trained to recognize the pattern. Reductions in power expenditure, memory usage, and time taken, for example, allow resource-limited computing devices to perform functions they otherwise could not.

Low precision neural networks using subband decomposition

A low-cost, low-power, stand-alone sensor platform having a visible-range camera sensor, a thermopile array, a microphone, a motion sensor, and a microprocessor that is configured to perform occupancy detection and counting while preserving the privacy of occupants. The platform is programmed to extract shape/texture from images in spatial domain; motion from video in time domain; and audio features in frequency domain. Embedded binarized neural networks are used for efficient object of interest detection. The platform is also programmed with advanced fusion algorithms for multiple sensor modalities addressing dependent sensor observations. The platform may be deployed for (i) residential use in detecting occupants for autonomously controlling building systems, such as HVAC and lighting systems, to provide energy savings, (ii) security and surveillance, such as to detect loitering and surveil places of interest, (iii) analyzing customer behavior and flows, (iv) identifying high performing stores by retailers.

Low power and privacy preserving sensor platform for occupancy detection

A method, apparatus and system for malware characterization includes receiving data identifying a presence of at least one anomaly of a respective portion of a processing function captured by at least one of each of at least two different sensor payloads and one sensor payload at two different times, determining a correlation between the at least two anomalies identified by the data captured by the at least one sensor payloads, and determining a presence of malware in the processing function based on the determined correlation. The method, apparatus and system can further include predicting an occurrence of at least one anomaly in the network based on at least one of current sensor payload data or previously observed and stored sensor payload data, recommending and/or initiating a remediation action and reporting a result of the malware characterization to a user.

Method of malware characterization and prediction

Techniques are disclosed for training machine learning systems. An input device receives training data comprising pairs of training inputs and training labels. A generative memory assigns training inputs to each archetype task of a plurality of archetype tasks, each archetype task representative of a cluster of related tasks within a task space and assigns a skill to each archetype task. The generative memory generates, from each archetype task, auxiliary data comprising pairs of auxiliary inputs and auxiliary labels. A machine learning system trains a machine learning model to apply a skill assigned to an archetype task to training and auxiliary inputs assigned to the archetype task to obtain output labels corresponding to the training and auxiliary labels associated with the training and auxiliary inputs assigned to the archetype task to enable scalable learning to obtain labels for new tasks for which the machine learning model has not previously been trained.

Generative memory for lifelong machine learning

In general, the disclosure describes techniques for creating runtime-throttleable neural networks (TNNs) that can adaptively balance performance and resource use in response to a control signal. For example, runtime-TNNs may be trained to be throttled via a gating scheme in which a set of disjoint components of the neural network can be individually 'turned off' at runtime without significantly affecting the accuracy of NN inferences. A separate gating neural network may be trained to determine which trained components of the NN to turn off to obtain operable performance for a given level of resource use of computational, power, or other resources by the neural network. This level can then be specified by the control signal at runtime to adapt the NN to operate at the specified level and in this way balance performance and resource use for different operating conditions.

Runtime-throttleable neural networks

A method, apparatus and system for anomaly detection in a processor based system includes training a deep learning sequence prediction model using observed baseline behavioral sequences of at least one processor behavior of the processor based system, predicting baseline behavioral sequences from the observed baseline behavioral sequences using the sequence prediction model, determining a baseline reconstruction error distribution profile using the baseline behavioral sequences and the predicted baseline behavioral sequences, predicting test behavioral sequences from observed, test behavioral sequences using the sequence prediction model, determining a testing reconstruction error distribution profile using the observed test behavioral sequences and the predicted test behavioral sequences, and comparing the baseline reconstruction error distribution profile to the testing reconstruction error distribution profile to determine if an anomaly exists in a processor behavior of the processor based system.

Anomalous behavior detection in processor based systems

Techniques are disclosed for training a deep neural network (DNN) for reduced computational resource requirements. A computing system includes a memory for storing a set of weights of the DNN. The DNN includes a plurality of layers. For each layer of the plurality of layers, the set of weights includes weights of the layer and a set of bit precision values includes a bit precision value of the layer. The weights of the layer are represented in the memory using values having bit precisions equal to the bit precision value of the layer. The weights of the layer are associated with inputs to neurons of the layer. Additionally, the computing system includes processing circuitry for executing a machine learning system configured to train the DNN. Training the DNN comprises optimizing the set of weights and the set of bit precision values.

Dynamic adaptation of deep neural networks

Modules and control units cooperate to simultaneously and independently control and adjust pixel parameters non-uniformly at regional increments across an entire image captured in an image frame by pixels in a pixel array. Pixel parameter changes for pixels in a given region occur, based on i) a contextual understanding of what contextually was happening in the one or more prior image frames and ii) whether salient items are located within that region. Additionally, guidance is sent to the sensor control unit to i) increase or decrease pixel parameters within those regions with salient items and then either to i) maintain, ii) increase or iii) decrease pixel parameters within regions without salient items in order to stay within any i) bandwidth limitations ii) memory storage, and/or iii) power consumptions limitations imposed by 1) one or more image sensors or 2) the communication loop between the sensor control unit and the image processing unit.

Methods for enhanced imaging based on semantic processing and dynamic scene modeling

Operations of computing devices are managed using one or more deep neural networks (DNNs), which may receive, as DNN inputs, data from sensors, instructions executed by processors, and/or outputs of other DNNs. One or more DNNs, which may be generative, can be applied to the DNN inputs to generate DNN outputs based on relationships between DNN inputs. The DNNs may include DNN parameters learned using one or more computing workloads. The DNN outputs may be, for example, control signals for managing operations of computing devices, predictions for use in generating control signals, warnings indicating an acceptable state is predicted, and/or inputs to one or more neural networks. The signals enhance performance, efficiency, and/or security of one or more of the computing devices. DNNs can be dynamically trained to personalize operations by updating DNN weights or other parameters.

Systems and methods for optimizing operations of computing devices using deep neural networks

In general, techniques are described for processing a set of high-resolution images of an integrated circuit, the images captured at different locations with respect to the integrated circuit, to automatically align and 'stitch' the set of high-resolution images into a larger composite image. For example, an imaging system as described herein may use sampled feature points distributed across different grid tiles within overlap regions for pairs of images to match feature points to inform the alignments of a pair with respect to each image in the pair. The system may in some cases further apply a bundle adjustment to iteratively align and refine the alignment results for each image in a set of images being processed. In some examples, the bundle adjustment is a best-fit adjustment based on minimizing the net error associated with the alignment of the set of images.

Integrated circuit image alignment and stitching

Device logic in a mobile device configures a processor to capture a series of images, such as a video, using a consumer-grade camera, and to analyze the images to determine the best-focused image, of the series of images, that captures a region of interest. The images may be of a textured surface, such as facial skin of a mobile device user. The processor sets a focal length of the camera to a fixed position for collecting the images. The processor may guide the user to position the mobile device for capturing the images, using audible cues. For each image, the processor crops the image to the region of interest, extracts luminance information, and determines one or more energy levels of the luminance via a Laplacian pyramid. The energy levels may be filtered, and then are compared to energy levels of the other images to determine the best-focused image.

Selecting optimal image from mobile device captures

A method and apparatus for optimizing image quality based on scene content comprising a sensor for generating a sequence of frames where each frame in the sequence of frames comprises content representing a scene and a digital processor, coupled to the sensor, for performing scene content analysis and for establishing a window defining a number of input frames from the sensor and processed output frames, and for aligning and combining the number of frames in the window to form an output frame, wherein sensor parameters and frame combination parameters are adjusted based on scene content.

Growing community of inventors

Princeton, NJ, United States of America

Sek Meng Chai