Hrl Laboratories, LLC

Gm Global Technolgoy Operations, Inc.

The Boeing Company

Theboeing Company

Ryan M Uhlenbrock

Deepak Khosla

Yang Chen

David W Payton

Fredy Monterroza

Leandro G Barajas

Li Yang Ku

Kyungnam Kim

Heiko Hoffmann

Rajan Bhattacharyya

Kang-Yu Ni

Kevin R Martin

Darren J Earl

Zhichao Chen

Tsai-Ching Lu

Swarup S Medasani

Qin Jiang

Hankyu Moon

Jiejun Xu

Anthony Wayne Baker

Vincent De Sapio

Rashmi N Sundareswara

Matthew E Phillips

James Benvenuto

Jason Meltzer

Michael J O'Brien

Brian L Burns

Huapeng Su

Rachel Millin

Eric Martinson

Described is a system and method for accurate image and/or video scene classification. More specifically, described is a system that makes use of a specialized convolutional-neural network (hereafter CNN) based technique for the fusion of bottom-up whole-image features and top-down entity classification. When the two parallel and independent processing paths are fused, the system provides an accurate classification of the scene as depicted in the image or video.

System and method for the fusion of bottom-up whole-image features and top-down enttiy classification for accurate image/video scene classification

Described is a system for activity and behavior detection in a target system. Raw data extracted from various heterogeneous sources of the target system is fused across spatial and temporal scales into a multi-graph representation. Information flows of the multi-graph representation are analyzed using a set of multi-layer information dynamic measures. Based on the set of multi-layer information dynamic measures, at least one of an economic and social indicator of emerging activity of interest in the target system is derived. The indicator is then used for prediction of future activity of interest in the target system.

Multilayer information dynamics for activity and behavior detection

Described is a system for real-time object recognition. The system extracts a candidate target region representing a candidate object from an input image of a scene based on agglomeration of channel features. The candidate target region is classified using a trained convolutional neural network (CNN) classifier, resulting in an initial classified object. A multi-target tracker is used for tracking the classified objects for final classification of each classified object, resulting in a final output, and a device is controlled based on the final output.

Real-time object recognition using cascaded features, deep learning and multi-target tracking

Described is a system for visual activity recognition that includes one or more processors and a memory, the memory being a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions, the one or more processors perform operations including detecting a set of objects of interest in video data and determining an object classification for each object in the set of objects of interest, the set including at least one object of interest. The one or more processors further perform operations including forming a corresponding activity track for each object in the set of objects of interest by tracking each object across frames. The one or more processors further perform operations including, for each object of interest and using a feature extractor, determining a corresponding feature in the video data. The system may provide a report to a user's cell phone or central monitoring facility.

Neuromorphic system for real-time visual activity recognition

Described is a system for converting a convolutional neural network (CNN) designed and trained for color (RGB) images to one that works on infrared (IR) or grayscale images. The converted CNN comprises a series of convolution layers of neurons arranged in a set kernels having corresponding depth slices. The converted CNN is used for performing object detection. A mechanical component of an autonomous device is controlled based on the object detection.

Transfer learning of convolutional neural networks from visible color (RBG) to infrared (IR) domain

Described is a system for visual activity recognition. In operation, the system detects a set of objects of interest (OI) in video data and determines an object classification for each object in the set of OI, the set including at least one OI. A corresponding activity track is formed for each object in the set of OI by tracking each object across frames. Using a feature extractor, the system determines a corresponding feature in the video data for each OI, which is then used to determine a corresponding initial activity classification for each OI. One or more OI are then detected in each activity track via foveation, with the initial object detection and foveated object detection thereafter being appended into a new detected-objects list. Finally, a final classification is provided for each activity track using the new detected-objects list and filtering the initial activity classification results using contextual logic.

System and method for neuromorphic visual activity classification based on foveated detection and contextual filtering

A method includes generating, based on first surface data that includes three-dimensional (3D) point positions corresponding to a first portion of a surface of an object, first hole data that indicates first positions of holes in the first portion of the surface. The method includes generating, based on second surface data corresponding to a second portion of the surface of the object, second hole data that indicates second positions of the holes in the second portion of the surface. The method also includes matching the first positions to the second positions to perform an alignment with respect to the first surface data and the second surface data.

Hole-based 3D point data alignment

A system and method of automated classification of rock types includes: partitioning, by a processing device, an image into partitions; extracting, by the processing device, sub-images from each of the partitions; first-level classifying, by an automated classifier, the sub-images into corresponding first classes; and second-level classifying, by the processing device, the partitions into corresponding second classes by, for each partition of the partitions, selecting a most numerous one of the corresponding first classes of the sub-images extracted from the partition. A method of displaying automated classification results on a display device is provided. The method includes: receiving, by a processing device, an image partitioned into partitions and classified into corresponding classes; and manipulating, by the processing device, the display device to display the image together with visual identification of the partitions and their corresponding classes. The method may include generating a depth-aligned log file of the classifications.

Automated classification of rock types and analyst-centric visualizations&#x2014;front end

Described is an object recognition system. Using an integral channel features (ICF) detector, the system extracts a candidate target region (having an associated original confidence score representing a candidate object) from an input image of a scene surrounding a platform. A modified confidence score is generated based on a location and height of detection of the candidate object. The candidate target regions are classified based on the modified confidence score using a trained convolutional neural network (CNN) classifier, resulting in classified objects. The classified objects are tracked using a multi-target tracker for final classification of each classified object as a target or non-target. If the classified object is a target, a device can be controlled based on the target.

System for real-time object detection and recognition using both image and size features

Described is a system for neural decoding of neural activity. Using at least one neural feature extraction method, neural data that is correlated with a set of behavioral data is transformed into sparse neural representations. Semantic features are extracted from a set of semantic data. Using a combination of distinct classification modes, the set of semantic data is mapped to the sparse neural representations, and new input neural data can be interpreted.

System for mapping extracted Neural activity into Neuroceptual graphs

An apparatus includes a nearest neighbor search engine configured to receive multiple sets of surface data. Each of the multiple sets includes three-dimensional point positions of a corresponding portion of a surface of an object. The nearest neighbor search engine is configured to perform a nearest neighbor search based on adjacency data indicating pairs of overlapping sets of the multiple sets to identify, in each pair of overlapping sets, pairs of closest corresponding points that are within a threshold distance of each other. The apparatus includes a transformation matrix generator configured to determine one or more transformation matrices to reduce a global distance metric that corresponds to distances between the points of each identified pair of closest corresponding points. The apparatus also includes a data set aligner configured to generate updated 3D point positions by applying the one or more transformation matrices to one or more of the multiple sets.

Three-dimensional point data alignment

A method for training an automated classifier of input images includes: receiving, by a processing device, a convolution neural network (CNN) model; receiving, by the processing device, training images and corresponding classes, each of the corresponding classes being associated with several ones of the training images; preparing, by the processing device, the training images, including separating the training images into a training set of the training images and a testing set of the training images; and training, by the processing device, the CNN model utilizing the training set, the testing set, and the corresponding classes to generate the automated classifier.

Automated classification of images using deep learning&#x2014;back end

A system includes a processor configured to generate a registered first 3D point cloud based on a first 3D point cloud and a second 3D point cloud. The processor is configured to generate a registered second 3D point cloud based on the first 3D point cloud and a third 3D point cloud. The processor is configured to generate a combined 3D point cloud based on the registered first 3D point cloud and the registered second 3D point cloud. The processor is configured to compare the combined 3D point cloud with a mesh model of the object. The processor is configured to generate, based on the comparison, output data indicating differences between the object as represented by the combined 3D point cloud and the object as represented by the 3D model. The system includes a display configured to display a graphical display of the differences.

Monitoring object shape and deviation from design

A method for localizing and estimating a pose of a known object in a field of view of a vision system is described, and includes developing a processor-based model of the known object, capturing a bitmap image file including an image of the field of view including the known object, extracting features from the bitmap image file, matching the extracted features with features associated with the model of the known object, localizing an object in the bitmap image file based upon the extracted features, clustering the extracted features of the localized object, merging the clustered extracted features, detecting the known object in the field of view based upon a comparison of the merged clustered extracted features and the processor-based model of the known object, and estimating a pose of the detected known object in the field of view based upon the detecting of the known object.

Growing community of inventors

Calabasas, CA, United States of America

Ryan M Uhlenbrock