Nec Corporation

Nec Laboratories America, Inc.

Iain Melvin

Hans Peter Graf

Eric Cosatto

Asim Kadav

Christopher Malon

Bing Bai

Igor Durdanovic

Masayuki Sakata

Erik Kruus

Alexandru Niculescu-Mizil

Yuki Chiba

David Grangier

Hans-Peter Graf

Koray Kavukcuoglu

Asako Fujii

Pierre-Francois Laquerre

Akshat Aranya

Meera Hahn

Deep Patel

Chris White

Chih-Yao Ma

Systems and methods for optimizing multi-camera multi-entity artificial intelligence tracking systems. Visual and location information of entities from video feeds received from multiple cameras can be obtained by employing an entity detection model and re-identification model. Likelihood scores that entity detections belong to an entity track can be predicted from the visual and location information. The entity detections predicted into entity tracks can be processed by employing combinatorial optimization of the likelihood scores by identifying assumptions from the likelihood scores, entity detections, and the entity tracks, filtering the assumptions with unsatisfiable problems to obtain a filtered assumptions set, and optimizing an answer set by utilizing the filtered assumptions set and the likelihood scores to maximize an overall score and obtain optimized entity tracks. Multiple entities can be monitored by utilizing the optimized entity tracks.

Optimizing multi-camera multi-entity artificial intelligence tracking systems

A classification apparatus according to the present disclosure includes an input unit configured to receive an operation performed by a user, an extraction unit configured to extract moving image data by using a predetermined rule, a display control unit configured to display an icon corresponding to the extracted moving image data on a screen of a display unit, a movement detection unit configured to detect a movement of the icon on the screen caused by the operation performed by the user, and a specifying unit configured to specify a classification of the moving image data corresponding to the icon based on a position of the icon on the screen.

Classification apparatus, classification method and computer readable medium

A computer-implemented method executed by at least one processor for performing mini-batching in deep learning by improving cache utilization is presented. The method includes temporally localizing a candidate clip in a video stream based on a natural language query, encoding a state, via a state processing module, into a joint visual and linguistic representation, feeding the joint visual and linguistic representation into a policy learning module, wherein the policy learning module employs a deep learning network to selectively extract features for select frames for video-text analysis and includes a fully connected linear layer and a long short-term memory (LSTM), outputting a value function from the LSTM, generating an action policy based on the encoded state, wherein the action policy is a probabilistic distribution over a plurality of possible actions given the encoded state, and rewarding policy actions that return clips matching the natural language query.

Efficient and fine-grained video retrieval

A computer-implemented method executed by a processor for training a neural network to recognize driving scenes from sensor data received from vehicle radar is presented. The computer-implemented method includes extracting substructures from the sensor data received from the vehicle radar to define a graph having a plurality of nodes and a plurality of edges, constructing a neural network for each extracted substructure, combining the outputs of each of the constructed neural networks for each of the plurality of edges into a single vector describing a driving scene of a vehicle, and classifying the single vector into a set of one or more dangerous situations involving the vehicle.

Detecting dangerous driving situations by parsing a scene graph of radar detections

Systems and methods for improving video understanding tasks based on higher-order object interactions (HOIs) between object features are provided. A plurality of frames of a video are obtained. A coarse-grained feature representation is generated by generating an image feature for each of for each of a plurality of timesteps respectively corresponding to each of the frames and performing attention based on the image features. A fine-grained feature representation is generated by generating an object feature for each of the plurality of timesteps and generating the HOIs between the object features. The coarse-grained and the fine-grained feature representations are concatenated to generate a concatenated feature representation.

Spatio-temporal interaction network for learning object interactions

A computer-implemented method and system are provided. The system includes an image capture device configured to capture image data relative to an ambient environment of a user. The system further includes a processor configured to detect and localize objects, in a real-world map space, from the image data using a trainable object localization Convolutional Neural Network (CNN). The CNN is trained to detect and localize the objects from image and radar pairs that include the image data and radar data for different scenes of a natural environment. The processor is further configured to perform a user-perceptible action responsive to a detection and a localization of an object in an intended path of the user.

Video to radar

A computer-implemented method and system are provided for driving assistance. The system includes an image capture device configured to capture image data relative to an outward view from a motor vehicle. The system further includes a processor configured to detect and localize objects, in a real-world map space, from the image data using a trainable object localization Convolutional Neural Network (CNN). The CNN is trained to detect and localize the objects from image and radar pairs that include the image data and radar data for different driving scenes of a natural driving environment. The processor is further configured to provide a user-perceptible object detection result to a user of the motor vehicle.

Advanced driver-assistance system

A video device for predicting driving situations while a person drives a car is presented. The video device includes multi-modal sensors and knowledge data for extracting feature maps, a deep neural network trained with training data to recognize real-time traffic scenes (TSs) from a viewpoint of the car, and a user interface (UI) for displaying the real-time TSs. The real-time TSs are compared to predetermined TSs to predict the driving situations. The video device can be a video camera. The video camera can be mounted to a windshield of the car. Alternatively, the video camera can be incorporated into the dashboard or console area of the car. The video camera can calculate speed, velocity, type, and/or position information related to other cars within the real-time TS. The video camera can also include warning indicators, such as light emitting diodes (LEDs) that emit different colors for the different driving situations.

Video capturing device for predicting special driving situations

Semantic indexing methods and systems are disclosed. One such method is directed to training a semantic indexing model by employing an expanded query. The query can be expanded by merging the query with documents that are relevant to the query for purposes of compensating for a lack of training data. In accordance with another exemplary aspect, time difference features can be incorporated into a semantic indexing model to account for changes in query distributions over time.

Query generation and time difference features for supervised semantic indexing

Systems and methods are disclosed for classifying histological tissues or specimens with two phases. In a first phase, the method includes providing off-line training using a processor during which one or more classifiers are trained based on examples, including: finding a split of features into sets of increasing computational cost, assigning a computational cost to each set; training for each set of features a classifier using training examples; training for each classifier, a utility function that scores a usefulness of extracting the next feature set for a given tissue unit using the training examples. In a second phase, the method includes applying the classifiers to an unknown tissue sample with extracting the first set of features for all tissue units; deciding for which tissue unit to extract the next set of features by finding the tissue unit for which a score: S=U&#x2212;h*C is maximized, where U is a utility function, C is a cost of acquiring the feature and h is a weighting parameter; iterating until a stopping criterion is met or no more feature can be computed; and issuing a tissue-level decision based on a current state.

Computationally efficient whole tissue classifier for histology slides

Methods and systems for classifying incomplete data are disclosed. In accordance with one method, pairs of features and values are generated based upon feature measurements on the incomplete data. In addition, a transformation function is applied on the pairs of features and values to generate a set of vectors by mapping each of the pairs to a corresponding vector in an embedding space. Further, a hardware processor applies a prediction function to the set of vectors to generate at least one confidence assessment for at least one class that indicates whether the incomplete data is of the at least one class. The method further includes outputting the at least one confidence assessment.

Feature set embedding for incomplete data

A cell phone having distributed artificial intelligence services is provided. The cell phone includes a neural network for performing a first pass of object recognition on an image to identify objects of interest therein based on one or more criterion. The cell phone also includes a patch generator for deriving patches from the objects of interest. Each of the patches includes a portion of a respective one of the objects of interest. The cell phone additionally includes a transmitter for transmitting the patches to a server for further processing in place of an entirety of the image to reduce network traffic.

Growing community of inventors

Hopewell, NJ, United States of America

Iain Melvin