Samsung Electronics Co., Ltd.

University of Pennsylvania

Ibrahim Volkan Isler

Daniel Dongyuel Lee

Dae Won Lee

Daewon Lee

Xiaoran Fan

Ziyun Wang

Lawrence David Jackel

Richard Howard

Jinwook Huh

Colin Prepscius

Anthony Robert Bisulco

Fernando Cladera Ojeda

Yuan Chen

Eric Anthony Mitchell

Camillo Jose Taylor

Konstantinos Daniilidis

Pedro Piacenza

Ani Hsieh

Minghan Wei

Hyun Soo Park

Caleb Sebastian Escobedo

Zhijian Yang

Siddharth Rupavatharam

Nikhil Narsingh Chavan Dafle

Sergiy Popovych

Hyunjune Sebastian Seung

Daniel Robert Kepple

Kanaka Rajan

Il Memming Park

Galen Kailun Xing

In some embodiments, an apparatus for performing reactive positioning of a robot gripper includes one or more fingers disposed on an end-effector of the robot, a signal processing circuit, a memory storing instructions, and a processor. Each of the one or more fingers includes a transducer configured to generate vibrational energy based on an input signal, and convert an acoustic reflection of the vibrational energy from an object into a voltage signal. The signal processing circuit is configured to provide the input signal to each transducer, and perform signal processing on the voltage signal of each transducer resulting in reflection data. The processor is configured to execute the instructions to perform pre-touch proximity detection on the reflection data, perform grasp positioning on the reflection data, perform contact detection from the reflection data, and provide, to the robot, results of the pre-touch proximity detection, the grasp positioning, and the contact detection.

Sonicfinger: low-cost, compact, proximity and contact sensor for reactive positioning

A pose of a person is estimated using an image and audio impulse responses. The image represents a 2D scene including the person. The audio impulse responses are obtained with the present absent and present in an environment. The pose is reconstructed based on the image and the one or more audio impulse responses. The pose is a metric scale human pose.

Metric lifting of 3D human pose using sound

Provided is a method for predicting a location of a fast-moving object. The method includes receiving event information from an event camera, the event information corresponding to an event detected by the event camera, generating a Binary Event History Image (BEHI) based on the event information, providing the BEHI as an input to an event-based neural network, obtaining, as an output of the event-based neural network, a first predicted location of the fast-moving object, a normal distribution indicating prediction uncertainty of the predicted location, and a predicted time-to-collision (TTC). The method further includes estimating a second predicted location of the fast-moving object based on the first predicted location, the normal distribution, and the predicted TTC output by the event-based neural network, and actuating a mechanical catching device to be at the second predicted location.

Systems and methods for real-time state estimation of fast-moving objects

A system and a method for generating a trajectory of a target device from a current position to a goal position within an environment is provided. The method may include: inputting physical workspace information associated with the environment in which the target device, to a first neural network to obtain a set of weights representing a cost-to-go function that defines a cost-to-go function relating to a length of a collision-free path from one position to the goal position; configuring a second neural network based on the set of weights; identifying a next position of the target device based on the current position and a motion control input of the target; and inputting the identified next position of the target device and the goal position to the second neural network to identify the trajectory to the goal position, and the motion control input corresponding to the trajectory.

Trajectory generation of a robot using a neural network

A tactile sensing system of a robot may include: a plurality of piezoelectric elements disposed at an object, and including a transmission (TX) piezoelectric element and a reception (RX) piezoelectric element; and at least one processor configured to: control the TX piezoelectric element to generate an acoustic wave having a chirp spread spectrum (CSS) at every preset time interval, along a surface of the object; receive, via the RX piezoelectric element, an acoustic wave signal corresponding to the generated acoustic wave; select frequency bands from a plurality of frequency bands of the acoustic wave signal; and estimate a location of a touch input on the surface of the object by inputting the acoustic wave signal of the selected frequency bands into a neural network configured to provide a touch prediction score for each of a plurality of predetermined locations on the surface of the object.

Method and apparatus for estimating touch locations and touch pressures

A method of estimating an amount of liquid poured from a container includes receiving a tactile signal detected by at least one tactile sensor of an end effector holding the container containing the liquid; receiving a joint signal including an angular position, angular velocity, and torque of a joint connected to the end effector; and estimating, based on the tactile signal and the joint signal, the amount of liquid poured from the container.

Method of estimating an amount of liquid poured from a container, and an apparatus for the same

A method of predicting occupancy of unseen areas in a region of interest (ROI) includes obtaining a depth image of the ROI, the depth image being captured from a first height; generating an occupancy map based on the obtained depth image, the occupancy map comprising an array of cells corresponding to locations in the ROI; and generating an inpainted map by inputting the occupancy map into a trained inpainting network, the inpainted map comprising an array of cells corresponding to the ROI, and wherein the inpainting network is trained by comparing an output of the inpainting network, based on inputting a training depth image taken from the first height, to a ground truth map, the ground truth map being based on a combination of the training depth image and a depth image taken at a height different than the first height.

Method of predicting occupancy of unseen areas for path planning, associated device, and network training method

A detection system includes a filter and a detector. The filter receives input packets from one or more dynamic vision sensors (DVSs) deployed in an area. The filter reduces the bandwidth of the input packets to produce compressed data packets while maintaining good end-to-end detection capability. The compressed data packets are transmitted to a detector in a low bandwidth transmission. The detector may collect compressed data packets from many filters. The detector determines if an object of interest is represented in the information of the input packets. If an object of interest is present, the detector activates an alarm, sends a notification to a person associated with the area and/or stores an indication in a database.

Dynamic vision filtering for event detection

A depth image is used to obtain a three dimensional (3D) geometry of an object as an object mesh. The object mesh is obtained using an object shell representation. The object shell representation is based on a series of depth images denoting the entry and exit points on the object surface that camera rays would pass through. Given a set of entry points in the form of a masked depth image of an object, an object shell (an entry image and an exit image) is generated. Since entry and exit images contain neighborhood information given by pixel adjacency, the entry and exit images provide partial meshes of the object which are stitched together in linear time using the contours of the entry and exit images. A complete object mesh is provided in the camera coordinate frame.

Object mesh based on a depth image

A method of collision localization on a robotic device includes obtaining audio signals from a plurality of acoustic sensors spaced apart along the robotic device; identifying, based on a collision being detected, a strongest audio signal; identifying a primary onset time for an acoustic sensor producing the strongest audio signal, the primary onset time being a time at which waves propagating from the collision reach the acoustic sensor producing the strongest audio signal; generating a virtual onset time set, by shifting a calibration manifold, based on the identified primary onset time, the calibration manifold representing relative onset times from evenly spaced marker locations on the robotic device to the plurality of acoustic sensors; determining scores for the marker locations based a standard deviation of elements in the virtual onset time set; and estimating a location of the collision based on a highest score of the determined scores.

Acoustic collision detection and localization for robotic devices

A method may include obtaining a set of events, of a set of pixels of a dynamic vision sensor, associated with an object; determining a set of voltages of the set of pixels, based on the set of events; generating a set of images, based on the set of voltages of the set of pixels; inputting the set of images into a first neural network configured to output a visual motion estimation of the object; inputting the set of images into a second neural network configured to output a confidence score of the visual motion estimation output by the first neural network; obtaining the visual motion estimation of the object and the confidence score of the visual motion estimation of the object, based on inputting the set of images into the first neural network and the second neural network; and providing the visual motion estimation of the object and the confidence score.

Jointly learning visual motion and confidence from local patches in event cameras

A method for generating a trajectory of a robot from a first configuration to a second configuration within an environment while steering away from obstacles may include obtaining physical workspace information associated with the environment in which the robot is configured to operate; obtaining, using a first neural network, a set of weights of a second neural network that is configured to generate a set of values associated with a set of configurations of the robot with respect to the second configuration; obtaining, by applying the set of weights to the second neural network, the set of values associated with the set of configurations of the robot with respect to the second configuration; and generating the trajectory of the robot from the first configuration to the second configuration within the environment, based on the set of values.

A tool control system may include: a tactile sensor configured to, when a tool holds a target object and slides the target object downward across the tool, obtain tactile sensing data from the tool; one or more memories configured to store a target velocity and computer-readable instructions; and one or more processors configured execute the computer-readable instructions to: receive the tactile sensing data from the tactile sensor; estimate a velocity of the target object based on the tactile sensing data, by using one or more neural networks that are trained based on a training image of an sample object captured while the sample object is sliding down; and generate a control parameter of the tool based on the estimated velocity and the target velocity.

Method and apparatus for manipulating a tool to control in-grasp sliding of an object held by the tool

An apparatus for reconstructing a 3D object, includes a memory storing instructions, and at least one processor configured to execute the instructions to obtain, using a first neural network, mapping function weights of a mapping function of a second neural network, based on an image feature vector corresponding to a 2D image of the 3D object, set the mapping function of the second neural network, using the obtained mapping function weights, and based on sampled points of a canonical sampling domain, obtain, using the second neural network of which the mapping function is set, 3D point coordinates and geodesic lifting coordinates of each of the sampled points in the 3D object corresponding to the 2D image, wherein the 3D point coordinates are first three dimensions of an embedding vector of a respective one of the sampled points, and the geodesic lifting coordinates are remaining dimensions of the embedding vector.

Growing community of inventors

Saint Paul, MN, United States of America

Ibrahim Volkan Isler