Nec Corporation

Nec Laboratories America, Inc.

Murugan Sankaradas

Srimat T Chakradhar

Kunal Rao

Giuseppe Coviello

Yi Yang

Min Feng

Oliver Po

Utsav Drolia

Biplob Kumar Debnath

Srihari Cadambi

Cheng-Hong Li

Wang-Pin Hsiung

Rajat Phull

Sibendu Paul

Erik Kruus

Venkata S Jakkula

Anousheh Gholami

Amit Redkar

Ravi Kailasam Rajendran

Vivek Aswal

Systems and methods are provided for dynamically tuning camera parameters in a video analytics system to optimize analytics accuracy. A camera captures a current scene, and optimal camera parameter settings are learned and identified for the current scene using a Reinforcement Learning (RL) engine. The learning includes defining a state within the RL engine as a tuple of two vectors: a first representing current camera parameter values and a second representing measured values of frames of the current scene. Quality of frames is estimated using a quality estimator, and camera parameters are adjusted based on the quality estimator and the RL engine for optimization. Effectiveness of tuning is determined using perceptual Image Quality Assessment (IQA) to quantify a quality measure. Camera parameters are adaptively tuned in real-time based on learned optimal camera parameter settings, state, quality measure, and set of actions, to optimize the analytics accuracy for video analytics tasks.

Adaptive perceptual quality based camera tuning using reinforcement learning

A method for automatically adjusting camera parameters to improve video analytics accuracy during continuously changing environmental conditions is presented. The method includes capturing a video stream from a plurality of cameras, performing video analytics tasks on the video stream, the video analytics tasks defined as analytics units (AUs), applying image processing to the video stream to obtain processed frames, filtering the processed frames through a filter to discard low-quality frames and dynamically fine-tuning parameters of the plurality of cameras. The fine-tuning includes passing the filtered frames to an AU-specific proxy quality evaluator, employing State-Action-Reward-State-Action (SARSA) reinforcement learning (RL) computations to automatically fine-tune the parameters of the plurality of cameras, and based on the reinforcement computations, applying a new policy for an agent to take actions and learn to maximize a reward.

Reinforcement-learning based system for camera parameter tuning to improve analytics

A method for free flow fever screening is presented. The method includes capturing a plurality of frames from thermal data streams and visual data streams related to a same scene to define thermal data frames and visual data frames, detecting and tracking a plurality of individuals moving in a free-flow setting within the visual data frames, and generating a tracking identification for each individual of the plurality of individuals present in a field-of-view of the one or more cameras across several frames of the plurality of frames. The method further includes fusing the thermal data frames and the visual data frames, measuring, by a fever-screener, a temperature of each individual of the plurality of individuals within and across the plurality of frames derived from the thermal data streams and the visual data streams, and generating a notification when a temperature of an individual exceeds a predetermined threshold temperature.

Free flow fever screening

A method for performing resource orchestration for microservices-based 5G applications in a dynamic, heterogenous, multi-tiered compute and network environment is presented. The method includes managing compute requirements and network requirements of a microservices-based application jointly by positioning computing nodes distributed across multiple layers, across edges and at a central cloud, identifying and modeling coupling relationships between compute and network resources for a plurality of microservices, when only application-level requirements are provided, to build coupling functions, solving a multi-objective optimization problem to identify how each of the plurality of microservices are deployed in the dynamic, heterogenous, multi-tiered compute and network environment by employing the coupling functions to jointly optimize resource usage of the compute and network resources across different compute and network slices, and deriving optimal joint network and compute resource allocation and function placement decisions.

Resource orchestration for microservices-based 5G applications

A method for employing a semi-supervised learning approach to improve accuracy of a small model on an edge device is presented. The method includes collecting a plurality of frames from a plurality of video streams generated from a plurality of cameras, each camera associated with a respective small model, each small model deployed in the edge device, sampling the plurality of frames to define sampled frames, performing inference to the sampled frames by using a big model, the big model shared by all of the plurality of cameras and deployed in a cloud or cloud edge, using the big model to generate labels for each of the sampled frames to generate training data, and training each of the small models with the training data to generate updated small models on the edge device.

Dynamic, contextualized AI models

Methods and systems for executing an application include extending a container orchestration system application programming interface (API) to handle objects that specify components of an application. An application representation is executed using the extended container orchestration system API, including the instantiation of one or more services that define a data stream path from a sensor to a device.

Data exchange and transformation in stream computing systems

A method for real-time cross-spectral object association and depth estimation is presented. The method includes synthesizing, by a cross-spectral generative adversarial network (CS-GAN), visual images from different data streams obtained from a plurality of different types of sensors, applying a feature-preserving loss function resulting in real-time pairing of corresponding cross-spectral objects, and applying dual bottleneck residual layers with skip connections to accelerate real-time inference and to accelerate convergence during model training.

Real-time cross-spectral object association and depth estimation

Systems and methods for network bandwidth optimization, including transmitting sensor data from one or more sensors over a wireless network into a generated network slice, submitting a Quality-of-Service (QoS) request for one or more applications by specifying desired network slice characteristics, and predicting network bandwidth needed for granting the QoS request for the one or more applications using a cost function based on magnitude, direction, and frequency of error. Time-varying network bandwidth usage is continuously monitored, and new QoS requests for the one or more applications are periodically requested based on the monitoring. An updated prediction for updated bandwidth needed for the new QoS request is generated using the cost function, and network bandwidth reservations are iteratively adjusted based on the updated prediction for the new QoS request to provide an amount of network resources to the one or more applications to support the new QoS request.

Dynamic self-optimization of application quality-of-service requests to mobile networks

Systems and methods for determining dwell time is provided. The method includes receiving images of an area including one or more people from one or more cameras, and detecting a presence of each of the one or more people in the received images using a worker. The method further includes receiving by the worker digital facial features stored in a watch list from a master controller, and performing facial recognition and monitoring the dwell time of each of the one or more people. The method further includes determining if each of the one or more people is in the watch list or has exceeded a dwell time threshold.

Video analytics system for dwell-time determinations

A method for tracing individuals through physical spaces that includes registering cameras in groupings relating a physical space. The method further includes performing local video monitoring including a video sensor input that outputs frames from inputs from recording with the cameras in the groupings, a face detection application for extracting faces from the output frames, and a face matching application for matching faces extracted from the output frames to a watchlist, and a local movement monitor that assigns tracks to the matched faces. The method further includes performing a global monitor including a biometrics monitor for preparing the watchlist of faces, the watchlist of faces being updated when a new face is detected by the cameras in the groupings, and a global movement monitor that combines the outputs from the assigned tracks to the matched faces to launch a report regarding individual population traveling to the physical spaces.

Tracking within and across facilities

A computer-implemented method for emulating an object recognizer includes receiving testing image data, and emulating, by employing a first object recognizer, a second object recognizer. Emulating the second object recognizer includes using the first object recognizer to perform object recognition on a testing object from the testing image data to generate data, the data including a feature representation for the testing object, and classifying the testing object based on the feature representation and a machine learning model configured to predict whether the testing object would be recognized by a second object recognizer. The method further includes triggering an action to be performed based on the classification.

Object recognizer emulation

A computer-implemented method includes obtaining a usecase specification and a usecase runtime specification corresponding to the usecase. The usecase includes a plurality of applications each being associated with a micro-service providing a corresponding functionality within the usecase for performing a task. The method further includes determining that at least one instance of the at least one of the plurality of applications can be reused during execution of the usecase based on the usecase specification and the usecase runtime specification, and reusing the at least one instance during execution of the usecase.

Usecase specification and runtime execution

Systems and methods to specify and execute real-time streaming applications are provided. The method includes specifying an application topology for an application including spouts, bolts, connections, a global hash table, and a topology manager. Each spout receives input data and each bolt transforms the input data, the global hash table allows in memory communication between each spout and bolt to others of the spouts and the bolts. The topology manager manages the application topology. The method includes compiling the application into a shared or static library for applications, and exporting a special symbol associated with the application. The runtime system can be used to retrieve the application topology from the shared or static library based on the special symbol and execute the application topology on a single node or distribute across multiple nodes.

Specification and execution of real-time streaming applications

A computer-implemented method includes obtaining a usecase specification and a usecase runtime specification corresponding to the usecase. The usecase includes a plurality of applications each being associated with a micro-service providing a corresponding functionality within the usecase for performing a task. The method further includes managing execution of the usecase within a runtime system based on the usecase and usecase runtime specifications to perform the task by serving an on-demand query and dynamically scaling resources based on the on-demand query, including using a batch helper server to employ the usecase specification to load dynamic application instances and connect the dynamic application instances to existing instances, and employ a batch helper configuration to load nodes/machines for execution of the on-demand query.

Usecase specification and runtime execution to serve on-demand queries and dynamically scale resources

A system for planning a trip includes heterogeneous data sources including map data, traffic information, vehicle trace data, weather reports, social media data, commuter feedback data, GIS data, travel time data; a stream analytics engine coupled to the heterogeneous data sources; a batch analytics engine coupled to the heterogeneous data sources; and a multi-modal journey planner coupled to the stream analytics engine and the batch analytics engine, the multi-modal journey planner processing indoor travel information and providing real-time updates while a journey is under progress, the multi-modal journey planner providing a journey time forecast as the journey time reflects indoor travel time.

Growing community of inventors

Dayton, NJ, United States of America

Murugan Sankaradas