Nec Corporation

Nec Laboratories America, Inc.

Yi Yang

Srimat T Chakradhar

Giuseppe Coviello

Min Feng

Biplob Kumar Debnath

Murugan Sankaradas

Kunal Rao

Nishkam Ravi

Oliver Po

Utsav Drolia

Renqiang Min

Wang-Pin Hsiung

Yang Gao

Linnan Wang

Neisarg Dave

Amit Redkar

Chao Li

Ravi Kailasam Rajendran

Nitin Agrawal

Tarang Chugh

Bin Ren

A method for free flow fever screening is presented. The method includes capturing a plurality of frames from thermal data streams and visual data streams related to a same scene to define thermal data frames and visual data frames, detecting and tracking a plurality of individuals moving in a free-flow setting within the visual data frames, and generating a tracking identification for each individual of the plurality of individuals present in a field-of-view of the one or more cameras across several frames of the plurality of frames. The method further includes fusing the thermal data frames and the visual data frames, measuring, by a fever-screener, a temperature of each individual of the plurality of individuals within and across the plurality of frames derived from the thermal data streams and the visual data streams, and generating a notification when a temperature of an individual exceeds a predetermined threshold temperature.

Free flow fever screening

A method for employing a semi-supervised learning approach to improve accuracy of a small model on an edge device is presented. The method includes collecting a plurality of frames from a plurality of video streams generated from a plurality of cameras, each camera associated with a respective small model, each small model deployed in the edge device, sampling the plurality of frames to define sampled frames, performing inference to the sampled frames by using a big model, the big model shared by all of the plurality of cameras and deployed in a cloud or cloud edge, using the big model to generate labels for each of the sampled frames to generate training data, and training each of the small models with the training data to generate updated small models on the edge device.

Dynamic, contextualized AI models

A method for real-time cross-spectral object association and depth estimation is presented. The method includes synthesizing, by a cross-spectral generative adversarial network (CS-GAN), visual images from different data streams obtained from a plurality of different types of sensors, applying a feature-preserving loss function resulting in real-time pairing of corresponding cross-spectral objects, and applying dual bottleneck residual layers with skip connections to accelerate real-time inference and to accelerate convergence during model training.

Real-time cross-spectral object association and depth estimation

A method for implementing a self-optimized video analytics pipeline is presented. The method includes decoding video files into a sequence of frames, extracting features of objects from one or more frames of the sequence of frames of the video files, employing an adaptive resource allocation component based on reinforcement learning (RL) to dynamically balance resource usage of different microservices included in the video analytics pipeline, employing an adaptive microservice parameter tuning component to balance accuracy and performance of a microservice of the different microservices, applying a graph-based filter to minimize redundant computations across the one or more frames of the sequence of frames, and applying a deep-learning-based filter to remove unnecessary computations resulting from mismatches between the different microservices in the video analytics pipeline.

Self-optimizing video analytics pipelines

Methods and systems for face clustering include determining a quality score for each of a set of input images. A first subset of the input images is clustered, having respective quality scores that exceed a predetermined threshold, to form an initial set of clusters. A second subset of the input images is clustered, having respective quality scores below the predetermined threshold. An action is performed responsive to the clustered images after the second subset is added to the initial set of clusters.

Face clustering with image uncertainty

Systems and methods for video analytic processing with neuro-symbolic artificial intelligence are provided. These systems and methods include detecting and extracting one or more objects from one or more video frames, and identifying the attributes associated with each of the one or more objects. These further include extracting context from a question, and compiling a series of inquiries to identify the information needed to answer the question and identify missing information. These further include storing intermediate information about the extracted objects and identified attributes, and determining whether the question requires further modeling of data to obtain missing information. These further include mining the one or more video frames for missing information, and compiling the intermediate information from the data storage and missing information based on the context of the question to produce a final answer.

Video analytic processing with neuro-symbolic artificial intelligence

A computer-implemented method executed by at least one processor for detecting tattoos on a human body is presented. The method includes inputting a plurality of images into a tattoo detector, selecting one or more images of the plurality of images including tattoos, extracting, via a feature extractor, tattoo feature vectors from the tattoos found in the one or more images of the plurality of images including tattoos, applying a deep learning tattoo matching model to determine potential matches between the tattoo feature vectors and preexisting tattoo images stored in a tattoo training database, and generating a similarity score between the tattoo feature vectors and one or more of the preexisting tattoo images stored in the tattoo training database.

Deep learning tattoo match system based

Systems and methods for demographic determination using image recognition. The method includes analyzing an image with a pre-trained lightweight neural network model, where the lightweight neural network model generates a confidence value, and comparing the confidence value to a threshold value to determine if the pre-trained lightweight neural network model is sufficiently accurate. The method further includes analyzing the image with a pre-trained heavyweight neural network model for the confidence value below the threshold value, wherein the pre-trained heavyweight neural network model has above about one million trainable parameters and the pre-trained lightweight neural network model has a number of trainable parameters below one tenth the heavyweight model, and displaying demographic data to a user on a user interface, wherein the user modifies store inventory based on the demographic data.

Combined light and heavy models for image filtering

A computer-implemented method executed by at least one processor for person identification is presented. The method includes employing one or more cameras to receive a video stream including a plurality of frames to extract features therefrom, detecting, via an object detection model, objects within the plurality of frames, detecting, via a key point detection model, persons within the plurality of frames, detecting, via a color detection model, color of clothing worn by the persons, detecting, via a gender and age detection model, an age and a gender of the persons, establishing a spatial connection between the objects and the persons, storing the features in a feature database, each feature associated with a confidence value, and normalizing, via a ranking component, the confidence values of each of the features.

Person search system based on multiple deep learning models

A computer-implemented method includes obtaining a usecase specification and a usecase runtime specification corresponding to the usecase. The usecase includes a plurality of applications each being associated with a micro-service providing a corresponding functionality within the usecase for performing a task. The method further includes determining that at least one instance of the at least one of the plurality of applications can be reused during execution of the usecase based on the usecase specification and the usecase runtime specification, and reusing the at least one instance during execution of the usecase.

Usecase specification and runtime execution

Systems and methods to specify and execute real-time streaming applications are provided. The method includes specifying an application topology for an application including spouts, bolts, connections, a global hash table, and a topology manager. Each spout receives input data and each bolt transforms the input data, the global hash table allows in memory communication between each spout and bolt to others of the spouts and the bolts. The topology manager manages the application topology. The method includes compiling the application into a shared or static library for applications, and exporting a special symbol associated with the application. The runtime system can be used to retrieve the application topology from the shared or static library based on the special symbol and execute the application topology on a single node or distribute across multiple nodes.

Specification and execution of real-time streaming applications

A computer-implemented method includes obtaining a usecase specification and a usecase runtime specification corresponding to the usecase. The usecase includes a plurality of applications each being associated with a micro-service providing a corresponding functionality within the usecase for performing a task. The method further includes managing execution of the usecase within a runtime system based on the usecase and usecase runtime specifications to perform the task by serving an on-demand query and dynamically scaling resources based on the on-demand query, including using a batch helper server to employ the usecase specification to load dynamic application instances and connect the dynamic application instances to existing instances, and employ a batch helper configuration to load nodes/machines for execution of the on-demand query.

Usecase specification and runtime execution to serve on-demand queries and dynamically scale resources

A computer-implemented method executed by at least one processor for detecting tattoos on a human body is presented. The method includes inputting a plurality of images into a tattoo detection module, selecting one or more images of the plurality of images including tattoos with at least three keypoints, the at least three keypoints having auxiliary information related to the tattoos, manually labeling tattoo locations in the plurality of images including tattoos to create labeled tattoo images, increasing a size of the labeled tattoo images identified to be below a predetermined threshold by padding a width and height of the labeled tattoo images, training two different tattoo detection deep learning models with the labeled tattoo images defining tattoo training data, and executing either the first tattoo detection deep learning model or the second tattoo detection deep learning model based on a performance of a general-purpose graphical processing unit.

Deep learning based tattoo detection system with optimized data labeling for offline and real-time processing

Aspects of the present disclosure describe techniques for training a convolutional neural network using an inconsistent stochastic gradient descent (ISGD) algorithm. Training effort for training batches used by the ISGD algorithm are dynamically adjusted according to a determined loss for a given training batch which are classified into two sub states&#x2014;well-trained or under-trained. The ISGD algorithm provides more iterations for under-trained batches while reducing iterations for well-trained ones.

Accelerating deep neural network training with inconsistent stochastic gradient descent

Aspects of the present disclosure are directed to techniques that improve performance of CNN systems through the effect of improved memory efficiencies for CNNs operating on GPUs. Aspects of the disclosure demonstrate that off-chip memory in such CNN systems is underutilized due to at least three characteristics namely, data layout, data locality and inter-kernel redundancy. Aspects of the disclosure examine the performance impact of different data layouts and then describe a method to produce data layout selection for various layers of the CNN including a fast transformation implementation. Disclosed are improvements to data locality from working set expansion, elimination of inter-kernel redundancy and increase of TLP using kernel reconstruction techniques including kernel fusion and thread injection. Disclosed experimental results show that our optimizations are very effective to boost the performance of CNNs by amounts up to 9.76 times for a single kernel and 2.05 times for a network.

Growing community of inventors

Princeton, NJ, United States of America

Yi Yang