University of Pennsylvania

Alex Zihao Zhu

Konstantinos Daniilidis

Nikolay Asenov Atanasov

A method for prediction of an indication of motion using input from an event-based camera includes receiving events captured by an event-based camera, wherein each of the events represents a location of a change in pixel intensity, a polarity of the change, and a time. The method further includes discretizing the events into time discretized event volumes, each of which contain events that occur within a specified time range. The method further includes providing the time discretized event volumes as input to an encoder-decoder neural network trained to predict an indication of motion using a loss function that measures quality of image deblurring; generating, using the neural network, a prediction of the indication of motion. The method further includes using the prediction of the indication of motion in a machine vision application.

Methods, systems, and computer readable media for estimation of optical flow, depth, and egomotion using neural network trained using event-based learning

A method for implementing a soft data association modeled with probabilities is provided. The association probabilities are computed in an intertwined expectation maximization (EM) scheme with an optical flow computation that maximizes the expectation (marginalization) over all associations. In addition, longer tracks can be enabled by computing the affine deformation with respect to the initial point and using the resulting residual as a measure of persistence. The computed optical flow enables a varying temporal integration that is different for every feature and sized inversely proportional to the length of the optical flow. The results can be seen in egomotion and very fast vehicle sequences.

Growing community of inventors

Philadelphia, PA, United States of America

Alex Zihao Zhu