Nec Corporation

Nec Laboratories America, Inc.

Giuseppe Coviello

Srimat T Chakradhar

Kunal Rao

Murugan Sankaradas

Yi Yang

Srihari Cadambi

Biplob Kumar Debnath

Min Feng

Cheng-Hong Li

Oliver Po

Rajat Phull

Wang-Pin Hsiung

Murugan Sankaradass

Sibendu Paul

Souvik Bhattacherjee

Arash Rezaei

Utsav Drolia

Debayan Deb

Yang Gao

Valentina Pelliccia

Nitin Agrawal

Methods and systems for camera configuration include configuring an image capture configuration parameter of a camera according to a multi-objective reinforcement learning aggregated reward function. Respective quality estimates for analytics are determined after configuring the image capture parameters. The aggregated reward function is updated based on the quality estimates.

Camera parameter enhancement for multiple analytics

Systems and methods are provided for dynamically tuning camera parameters in a video analytics system to optimize analytics accuracy. A camera captures a current scene, and optimal camera parameter settings are learned and identified for the current scene using a Reinforcement Learning (RL) engine. The learning includes defining a state within the RL engine as a tuple of two vectors: a first representing current camera parameter values and a second representing measured values of frames of the current scene. Quality of frames is estimated using a quality estimator, and camera parameters are adjusted based on the quality estimator and the RL engine for optimization. Effectiveness of tuning is determined using perceptual Image Quality Assessment (IQA) to quantify a quality measure. Camera parameters are adaptively tuned in real-time based on learned optimal camera parameter settings, state, quality measure, and set of actions, to optimize the analytics accuracy for video analytics tasks.

Adaptive perceptual quality based camera tuning using reinforcement learning

A method for automatically adjusting camera parameters to improve video analytics accuracy during continuously changing environmental conditions is presented. The method includes capturing a video stream from a plurality of cameras, performing video analytics tasks on the video stream, the video analytics tasks defined as analytics units (AUs), applying image processing to the video stream to obtain processed frames, filtering the processed frames through a filter to discard low-quality frames and dynamically fine-tuning parameters of the plurality of cameras. The fine-tuning includes passing the filtered frames to an AU-specific proxy quality evaluator, employing State-Action-Reward-State-Action (SARSA) reinforcement learning (RL) computations to automatically fine-tune the parameters of the plurality of cameras, and based on the reinforcement computations, applying a new policy for an agent to take actions and learn to maximize a reward.

Reinforcement-learning based system for camera parameter tuning to improve analytics

A method for free flow fever screening is presented. The method includes capturing a plurality of frames from thermal data streams and visual data streams related to a same scene to define thermal data frames and visual data frames, detecting and tracking a plurality of individuals moving in a free-flow setting within the visual data frames, and generating a tracking identification for each individual of the plurality of individuals present in a field-of-view of the one or more cameras across several frames of the plurality of frames. The method further includes fusing the thermal data frames and the visual data frames, measuring, by a fever-screener, a temperature of each individual of the plurality of individuals within and across the plurality of frames derived from the thermal data streams and the visual data streams, and generating a notification when a temperature of an individual exceeds a predetermined threshold temperature.

Free flow fever screening

Methods and systems for executing an application include extending a container orchestration system application programming interface (API) to handle objects that specify components of an application. An application representation is executed using the extended container orchestration system API, including the instantiation of one or more services that define a data stream path from a sensor to a device.

Data exchange and transformation in stream computing systems

A pull-based communication method for microservices-based real-time streaming video analytics pipelines is provided. The method includes receiving a plurality of frames from a plurality of cameras, each camera including a camera sidecar, arranging a plurality of detectors in layers such that a first detector layer includes detectors with detector sidecars and detector business logic, and the second detector layer includes detectors with only sidecars, arranging a plurality of extractors in layers such that a first extractor layer includes extractors with extractor sidecars and extractor business logic, and the second extractor layer includes extractors with only sidecars, and enabling a mesh controller, during registration, to selectively assign inputs to one or more of the detector sidecars of the first detector layer and one or more of the extractor sidecars of the first extractor layer to pull data items for processing.

Flexible and efficient communication in microservices-based stream analytics pipeline

A method for implementing a self-optimized video analytics pipeline is presented. The method includes decoding video files into a sequence of frames, extracting features of objects from one or more frames of the sequence of frames of the video files, employing an adaptive resource allocation component based on reinforcement learning (RL) to dynamically balance resource usage of different microservices included in the video analytics pipeline, employing an adaptive microservice parameter tuning component to balance accuracy and performance of a microservice of the different microservices, applying a graph-based filter to minimize redundant computations across the one or more frames of the sequence of frames, and applying a deep-learning-based filter to remove unnecessary computations resulting from mismatches between the different microservices in the video analytics pipeline.

Self-optimizing video analytics pipelines

A method for implementing application self-optimization in serverless edge computing environments is presented. The method includes requesting deployment of an application pipeline on data received from a plurality of sensors, the application pipeline including a plurality of microservices, enabling communication between a plurality of pods and a plurality of analytics units (AUs), each pod of the plurality of pods including a sidecar, determining whether each of the plurality of AUs maintains any state to differentiate between stateful AUs and stateless AUs, scaling the stateful AUs and the stateless AUs, enabling communication directly between the sidecars of the plurality of pods, and reusing and resharing common AUs of the plurality of AUs across different applications.

System for application self-optimization in serverless edge computing environments

A method is provided for managing applications for sensors. In one embodiment, the method includes loading a plurality of applications and links for communicating with a plurality of sensors on a platform having an interface for entry of a requested use case; and copying a configuration from a grouping of application instances being applied to a first sensor performing in a function comprising of the requested use case. The method may further include applying the configuration for the grouping of application instances to a second set of sensors to automatically conform the plurality of sensors on the platform to perform the requested use case.

Managing applications for sensors

Methods and systems for face clustering include determining a quality score for each of a set of input images. A first subset of the input images is clustered, having respective quality scores that exceed a predetermined threshold, to form an initial set of clusters. A second subset of the input images is clustered, having respective quality scores below the predetermined threshold. An action is performed responsive to the clustered images after the second subset is added to the initial set of clusters.

Face clustering with image uncertainty

A computer-implemented method executed by at least one processor for detecting tattoos on a human body is presented. The method includes inputting a plurality of images into a tattoo detector, selecting one or more images of the plurality of images including tattoos, extracting, via a feature extractor, tattoo feature vectors from the tattoos found in the one or more images of the plurality of images including tattoos, applying a deep learning tattoo matching model to determine potential matches between the tattoo feature vectors and preexisting tattoo images stored in a tattoo training database, and generating a similarity score between the tattoo feature vectors and one or more of the preexisting tattoo images stored in the tattoo training database.

Deep learning tattoo match system based

Methods and systems for managing communications include identifying a system condition in a distributed computing system comprising a first microservice in communication with a second microservice. A communications method is identified responsive to the identified system condition using a reinforcement learning model that associates communication methods with system conditions. The identified communications method is implemented for communications between the first microservice and the second microservice, such that the first microservice and the second microservice use the identified communications method to transmit data.

Dynamic microservice intercommunication configuration

A method for optimal placement of microservices of a micro-services-based application in a multi-tiered computing network environment employing 5G technology is presented. The method includes accessing a centralized server or cloud to request a set of services to be deployed on a plurality of sensors associated with a plurality of devices, the set of services including launching an application on a device of the plurality of devices, modeling the application as a directed graph with vertices being microservices and edges representing communication between the microservices, assigning each of the vertices of the directed graph with two cost weights, employing an edge monitor (EM), an edge scheduler (ES), an alerts-manager at edge (AM-E), and a file transfer (FT) at the edge to handle partitioning of the microservices, and dynamically mapping the microservices to the edge or the cloud to satisfy application-specific response times.

Eco: edge-cloud optimization of 5G applications

Methods and systems for image clustering include matching a new image to a representative image of a cluster. The new image is set as a representative of the cluster with a first time limit. The new image is set as a representative of the cluster with a second time limit, responsive to a determination that the new image has matched at least one incoming image during the first time limit.

Online face clustering

A computer-implemented method executed by at least one processor for person identification is presented. The method includes employing one or more cameras to receive a video stream including a plurality of frames to extract features therefrom, detecting, via an object detection model, objects within the plurality of frames, detecting, via a key point detection model, persons within the plurality of frames, detecting, via a color detection model, color of clothing worn by the persons, detecting, via a gender and age detection model, an age and a gender of the persons, establishing a spatial connection between the objects and the persons, storing the features in a feature database, each feature associated with a confidence value, and normalizing, via a ranking component, the confidence values of each of the features.

Growing community of inventors

Plainsboro, NJ, United States of America

Giuseppe Coviello