Ford Global Technolgoies, LLC

Gaurav Kumar Singh

Gintaras Vincent Puskorius

Dimitar Petrov Filev

Bruno Sielly Jales Costa

Pavithra Madhavan

Vidya Nariyambut Murali

Guy Hotson

Pol Llado

The present invention extends to methods, systems, and computer program products for classifying time series image data. Aspects of the invention include encoding motion information from video frames in an eccentricity map. An eccentricity map is essentially a static image that aggregates apparent motion of objects, surfaces, and edges, from a plurality of video frames. In general, eccentricity reflects how different a data point is from the past readings of the same set of variables. Neural networks can be trained to detect and classify actions in videos from eccentricity maps. Eccentricity maps can be provided to a neural network as input. Output from the neural network can indicate if detected motion in a video is or is not classified as an action, such as, for example, a hand gesture.

Classifying time series image data

A computing system can determine moving objects in a sequence of images based on recursively calculating red-green-blue (RGB) eccentricity 249based on a video data stream. A vehicle can be operated based on the determined moving objects. The video data stream can be acquired by a color video sensor included in the vehicle or a traffic infrastructure system.

Eccentricity maps

Techniques and examples pertaining to objection detection and trajectory prediction for autonomous vehicles are described. A processor receives an input stream of image frames and fuses a spatiotemporal input stream of the image frames and an appearance-based stream of the image frames using a deep neural network (DNN) to generate an augmented stream of the image frames. The processor performs object detection and trajectory prediction of one or more objects in the image frames based on the augmented stream.

Fusion of motion and appearance features for object detection and trajectory prediction

Growing community of inventors

Westland, MI, United States of America

Gaurav Kumar Singh