Nvidia Corporation

Sun Microsystems, Inc.

Conexant Systems, Inc.

Oracle America, Inc.

Parthasarathy Sriram

Farzin Aghdasi

Subramania Sudharsanan

Milind R Naphade

Fnu Ratnesh Kumar

Sujit Biswas

Anil Ubale

Anurag Bist

Arman Toorians

Bhanu Nagendra Pisupati

Aaron Bartholomew

Yan Zhai

Subhashree Radhakrishnan

Ratnesh Kumar

Vinay Kolar

Amit Gulati

Aleksandr Frid

Edwin Weill

Steven E Molnar

James Leroy Deming

Wei Jia

Rohit Puri

Brian Lawrence Smith

Jay Huang

Gordon Grigor

Greg Heinrich

Jay Gupta

Shu-Jen Fang

Robert Quan

Tayfun Gurel

Stephane Le Provost

Han Chou

Shuo Wang

Zeyu Zhao

Shangru Li

Evan McLaughlin

Arihant Jain

Varun Praveen

Jessica Fang

Systems and methods disclosed relate to generating training data. In one embodiment, the disclosure relates to systems and methods for generating training data to train a neural network to detect and classify objects. A simulator obtains 3D models of objects, and simulates 3D environments comprising the objects using physics-based simulations. The simulations may include applying real-world physical conditions, such as gravity, friction, and the like on the objects. The system may generate images of the simulations, and use the images to train a neural network to detect and classify the objects from images.

Object simulation using real-world environments

Input layers of an element-wise operation in a neural network can be pruned such that the shape (e.g., the height, the width, and the depth) of the pruned layers matches. A pruning engine identifies all of the input layers into the element-wise operation. For each set of corresponding neurons in the input layers, the pruning engine equalizes the metrics associated with the neurons to generate an equalized metric associated with the set. The pruning engine prunes the input layers based on the equalized metrics generated for each unique set of corresponding neurons.

Pruning neural networks that include element-wise operations

Calibration of various sensors may be difficult without specialized software to process intrinsic and extrinsic information about the sensors. Certain types of input files, such as image files, may also lack certain information, like depth information, to effectively translate regions of interest between images taken from a different perspective. Landmarks can be used to establish points for associating regions of interest between images taken from a different perspective and provided as an overlay to verify sensor calibration.

Sensor calibration for space translation

In various examples, a neural network may be trained for use in vehicle re-identification tasks—e.g., matching appearances and classifications of vehicles across frames—in a camera network. The neural network may be trained to learn an embedding space such that embeddings corresponding to vehicles of the same identify are projected closer to one another within the embedding space, as compared to vehicles representing different identities. To accurately and efficiently learn the embedding space, the neural network may be trained using a contrastive loss function or a triplet loss function. In addition, to further improve accuracy and efficiency, a sampling technique—referred to herein as batch sample—may be used to identify embeddings, during training, that are most meaningful for updating parameters of the neural network.

Training neural networks for vehicle re-identification

In various examples, sensor data—such as masked sensor data—may be used as input to a machine learning model to determine a confidence for object to person associations. The masked sensor data may focus the machine learning model on particular regions of the image that correspond to persons, objects, or some combination thereof. In some embodiments, coordinates corresponding to persons, objects, or combinations thereof, in addition to area ratios between various regions of the image corresponding to the persons, objects, or combinations thereof, may be used to further aid the machine learning model in focusing on important regions of the image for determining the object to person associations.

Determining associations between objects and persons using machine learning models

The present disclosure provides various approaches for smart area monitoring suitable for parking garages or other areas. These approaches may include ROI-based occupancy detection to determine whether particular parking spots are occupied by leveraging image data from image sensors, such as cameras. These approaches may also include multi-sensor object tracking using multiple sensors that are distributed across an area that leverage both image data and spatial information regarding the area, to provide precise object tracking across the sensors. Further approaches relate to various architectures and configurations for smart area monitoring systems, as well as visualization and processing techniques. For example, as opposed to presenting video of an area captured by cameras, 3D renderings may be generated and played from metadata extracted from sensors around the area.

Scenario recreation through object detection and 3D visualization in a multi-sensor environment

In various examples, sensor data&#x2014;such as masked sensor data&#x2014;may be used as input to a machine learning model to determine a confidence for object to person associations. The masked sensor data may focus the machine learning model on particular regions of the image that correspond to persons, objects, or some combination thereof. In some embodiments, coordinates corresponding to persons, objects, or combinations thereof, in addition to area ratios between various regions of the image corresponding to the persons, objects, or combinations thereof, may be used to further aid the machine learning model in focusing on important regions of the image for determining the object to person associations.

In various examples, cloud computing systems may store frames of video streams and metadata generated from the frames in separate data stores, with each type of data being indexed using shared timestamps. Thus, the frames of a video stream may be stored and/or processed and corresponding metadata of the frames may be stored and/or generated across any number of devices of the cloud computing system (e.g., edge and/or core devices) while being linked by the timestamps. A client device may provide a request or query to dynamically annotate the video stream using a particular subset of the metadata. In processing the request or query, the timestamps may be used to retrieve video data representing frames of the video stream and metadata extracted from those frames across the data stores. The retrieved metadata and video data may be used to annotate the frames for display on the client device.

Overlaying metadata on video streams on demand for intelligent video analysis

In various examples, a neural network may be trained for use in vehicle re-identification tasks&#x2014;e.g., matching appearances and classifications of vehicles across frames&#x2014;in a camera network. The neural network may be trained to learn an embedding space such that embeddings corresponding to vehicles of the same identify are projected closer to one another within the embedding space, as compared to vehicles representing different identities. To accurately and efficiently learn the embedding space, the neural network may be trained using a contrastive loss function or a triplet loss function. In addition, to further improve accuracy and efficiency, a sampling technique&#x2014;referred to herein as batch sample&#x2014;may be used to identify embeddings, during training, that are most meaningful for updating parameters of the neural network.

Smart area monitoring with artificial intelligence

A programmable SoC (system on a chip) having optimized power domains and power islands. The SoC is an integrated circuit device including a plurality of power domains, each of the power domains having a respective voltage rail to supply power to the power domain. A plurality of power islands are included within the integrated circuit device, wherein each power domain includes at least one power island. A plurality of functional blocks are included within the integrated circuit device, wherein each power island includes at least one functional block. Each functional block is configured to provide a specific device functionality. The integrated circuit device adjusts power consumption in relation to a requested device functionality by individually turning on or turning off power to a selected one or more power domains, and for each turned on power domain, individually power gating one or more power islands.

Integrated circuit device having power domains and partitions based on use case power optimization

A system and method are provided for protecting data. In operation, a request to read data from memory is received. Additionally, it is determined whether the data is stored in a predetermined portion of the memory. If it is determined that the data is stored in the predetermined portion of the memory, the data and a protect signal are returned for use in protecting the data. In certain embodiments of the invention, data stored in the predetermined portion of the memory may be further processed and written hack to the predetermined portion of the memory. In other embodiments of the invention, such processing may involve unprotected data stored outside the predetermined portion of the memory.

System and method for protecting data by returning a protect signal with the data

Efficient and effective permission confidential information protection systems and methods are described. The secure information protection systems and methods facilitate storage of confidential information in a manner safe from rogue software access. In one embodiment, a confidential information protection method is implemented in hardware and facilitates protection against software and/or Operating System hacks. In one exemplary implementation, a confidential information protection method includes setting a permission sticky bit flag to a default state upon system set up. The permission sticky bit flag access permission indication is adjusted at system reset in accordance with an initial application instruction. Access to the confidential information is restricted in accordance with the permission sticky bit and the permission sticky bit is protected from adjustments attempting to violate the permission indication. For example, another software application can not access or alter confidential information (e.g., an encryption key, initialization vector, etc.) if a permission sticky bit is designated as the highest security rating (e.g., disabling read permission and write permission until system reset).

Confidential information protection system and method

Growing community of inventors

Los Altos, CA, United States of America

Parthasarathy Sriram