Samsung Electronics Co., Ltd.

University of Pennsylvania

Dae Won Lee

Ibrahim Volkan Isler

Daniel Dongyuel Lee

Yuan Chen

Colin Prepscius

Lawrence David Jackel

Camillo Jose Taylor

Xiaoran Fan

Konstantinos Daniilidis

Ziyun Wang

Anthony Robert Bisulco

Fernando Cladera Ojeda

Minghan Wei

Il Memming Park

Kanaka Rajan

Daniel Robert Kepple

Hyunjune Sebastian Seung

Ani Hsieh

Provided is a method for predicting a location of a fast-moving object. The method includes receiving event information from an event camera, the event information corresponding to an event detected by the event camera, generating a Binary Event History Image (BEHI) based on the event information, providing the BEHI as an input to an event-based neural network, obtaining, as an output of the event-based neural network, a first predicted location of the fast-moving object, a normal distribution indicating prediction uncertainty of the predicted location, and a predicted time-to-collision (TTC). The method further includes estimating a second predicted location of the fast-moving object based on the first predicted location, the normal distribution, and the predicted TTC output by the event-based neural network, and actuating a mechanical catching device to be at the second predicted location.

Systems and methods for real-time state estimation of fast-moving objects

A method of predicting occupancy of unseen areas in a region of interest (ROI) includes obtaining a depth image of the ROI, the depth image being captured from a first height; generating an occupancy map based on the obtained depth image, the occupancy map comprising an array of cells corresponding to locations in the ROI; and generating an inpainted map by inputting the occupancy map into a trained inpainting network, the inpainted map comprising an array of cells corresponding to the ROI, and wherein the inpainting network is trained by comparing an output of the inpainting network, based on inputting a training depth image taken from the first height, to a ground truth map, the ground truth map being based on a combination of the training depth image and a depth image taken at a height different than the first height.

Method of predicting occupancy of unseen areas for path planning, associated device, and network training method

A method of collision localization on a robotic device includes obtaining audio signals from a plurality of acoustic sensors spaced apart along the robotic device; identifying, based on a collision being detected, a strongest audio signal; identifying a primary onset time for an acoustic sensor producing the strongest audio signal, the primary onset time being a time at which waves propagating from the collision reach the acoustic sensor producing the strongest audio signal; generating a virtual onset time set, by shifting a calibration manifold, based on the identified primary onset time, the calibration manifold representing relative onset times from evenly spaced marker locations on the robotic device to the plurality of acoustic sensors; determining scores for the marker locations based a standard deviation of elements in the virtual onset time set; and estimating a location of the collision based on a highest score of the determined scores.

Acoustic collision detection and localization for robotic devices

A method may include obtaining a set of events, of a set of pixels of a dynamic vision sensor, associated with an object; determining a set of voltages of the set of pixels, based on the set of events; generating a set of images, based on the set of voltages of the set of pixels; inputting the set of images into a first neural network configured to output a visual motion estimation of the object; inputting the set of images into a second neural network configured to output a confidence score of the visual motion estimation output by the first neural network; obtaining the visual motion estimation of the object and the confidence score of the visual motion estimation of the object, based on inputting the set of images into the first neural network and the second neural network; and providing the visual motion estimation of the object and the confidence score.

Jointly learning visual motion and confidence from local patches in event cameras

A tool control system may include: a tactile sensor configured to, when a tool holds a target object and slides the target object downward across the tool, obtain tactile sensing data from the tool; one or more memories configured to store a target velocity and computer-readable instructions; and one or more processors configured execute the computer-readable instructions to: receive the tactile sensing data from the tactile sensor; estimate a velocity of the target object based on the tactile sensing data, by using one or more neural networks that are trained based on a training image of an sample object captured while the sample object is sliding down; and generate a control parameter of the tool based on the estimated velocity and the target velocity.

Method and apparatus for manipulating a tool to control in-grasp sliding of an object held by the tool

Growing community of inventors

Princeton, NJ, United States of America

Dae Won Lee