Toyota Research Institute, Inc.

Toyota Jidosha Kabushiki Kaisha

Naveen Suresh Kuppuswamy

Alexander Alspach

Avinash Uttamchandani

Takuya Ikeda

Kunimatsu Hashimoto

Erik C Sobel

Russell L Tedrake

Samson F Creasey

Manuel Ludwig Kuehner

Alejandro M Castro

Adrien David Gaidon

Vitor Guizilini

Rares A Ambrus

Hiroshi Yasuda

Andrew M Beaulieu

Katherine Tsui

Duy-Nguyen Ta-Huynh

Abhijeet Agnihotri

Jessica Yin

Deformable sensors having a stereo depth sensor and an infrared sensor and methods of their use are disclosed. In one embodiment, a deformable sensor includes an enclosure having a housing and a deformable membrane, wherein the deformable membrane is transparent to a wavelength band in the infrared spectrum, a stereo depth sensor that is disposed within the enclosure and is configured to view an underside of the deformable membrane, and output a deformation region of the deformable membrane as a result of contact with an object, wherein the deformation region includes depth information, and an infrared sensor that is disposed within the enclosure and is configured to view the object through the deformable membrane when the object contacts the deformable membrane and output a contact patch region corresponding with a region of the deformable membrane that contacts the object.

Deformable sensors having an internal stereo depth sensor and an infrared sensor

Systems and methods for calibrating deformable sensors are disclosed. In one embodiment, a method of calibrating a deformable sensor includes capturing image data of the deformable sensor using an external image sensor, wherein the deformable sensor comprises a deformable membrane defining an enclosure that is configured to be filled with a medium. The method further includes comparing the image data of the deformable sensor to a metric. When the image data does not satisfy the metric, the method includes adjusting a pressure within the enclosure.

Systems and methods for calibrating deformable sensors

A deformable sensor comprises an enclosure comprising a deformable membrane, the enclosure configured to be filled with a medium, and an imaging sensor, disposed within the enclosure, having a field of view configured to be directed toward a bottom surface of the deformable membrane. The imaging sensor is configured to capture an image of the deformable membrane. The deformable sensor is configured to determine depth values for a plurality of points on the deformable membrane based on the image captured by the imaging sensor and a trained neural network.

Robust and perceptive manipulation through soft-bubble grippers and monocular depth

Systems and methods for estimating deformation and field of contact forces are described. A method includes generating a reference configuration including reference points in space. The reference configuration corresponds to an initial shape of a membrane prior to contact with the manipuland. The method further includes receiving raw data from a TOF device. The raw data includes points in space measured by the TOF device and indicating deformation of the membrane due to contact with the manipuland. The method further includes determining deformation of the membrane that best approximates a current configuration in a least squares sense while satisfying a discrete physical model enforced as a linear constraint that corresponds to a linearized physical model of the deformation that is discretized with an FEM, linearizing the relationship, and estimating deformation and field of contact forces by solving a least squares formulation with physical constraints cast as a sparse quadratic program.

Systems and methods for estimating shape, contact forces, and pose of objects manipulated by robots having compliant contact and geometry sensors

Systems and methods for determining a location of a robot are provided. A method includes receiving, by a processor, a signal from a deformable sensor including data with respect to a deformation region in a deformable membrane of the deformable sensor resulting from contact with a first object. The data associated with contact with the first object is compared, by the processor, to details associated with contact with the first object to information associated with a plurality of objects stored in a database. The first object is identified, by the processor, as a first identified object of the plurality of objects stored in the database. The first identified object is an object of the plurality of objects stored in the database that is most similar to the first object. The location of the robot is determined, by the processor, based on a location of the first identified object.

Systems for determining location using robots with deformable sensors

Deformable sensors and methods for magnetically modifying membrane stiffness are provided. A deformable sensor may include a membrane coupled to a housing to form a sensor cavity. The deformable sensor may further include a first magnet located on an inner surface of the membrane in the sensor cavity. The deformable sensor may additionally include a magnetic object located at a base within the sensor cavity. The magnetic object may be configured to modifiably repel the first magnet and modify stiffness of the deformable sensor based upon the modified repulsion.

Deformable sensors and methods using magnetic repulsion for modifying run-time membrane stiffness

Input devices having a deformable membrane and methods of their use are disclosed. In one embodiment, an input device includes a body, a deformable membrane coupled to the body such that the body and the deformable membrane define an enclosure filled with a medium, and an internal sensor disposed within the enclosure, the internal sensor having a field of view configured to be directed through the medium and toward a bottom surface of the deformable membrane. The input device further includes a controller configured to receive an output signal from the internal sensor corresponding to a deformation in the deformable membrane, determine a gesture based on the output signal from the internal, and provide a gesture signal corresponding to the gesture.

Input devices having a deformable membrane and methods of using the same

A robot arm assembly for detecting a pose and force associated with an object is provided. The robot arm assembly includes an end effector having a plurality of fingers, and a deformable sensor provided on each finger. The deformable sensor includes a housing, a deformable membrane coupled to the housing, an enclosure filled with a medium, and an internal sensor disposed within the housing having a field of view directed through the medium and toward an internal surface of the deformable membrane. A processor is configured to receive an output from each internal sensor, the output including a contact region of the deformable membrane as a result of contact with the object. The processor determines an amount of displacement of the contact region based on the output from each internal sensor, and determines the pose and the force associated with the object based on the amount of displacement.

Robot arm assemblies including fingers having deformable sensors

Deformable sensors and methods for modifying membrane stiffness through magnetic attraction are provided. A deformable sensor may include a membrane coupled to a housing to form a sensor cavity. The deformable sensor may further include magnetically-attractable particles located on or within the membrane. The deformable sensor may additionally include a magnetic object located at a base within the sensor cavity. The magnetic object may be configured to modifiably attract the magnetically-attractable particles and modify stiffness of the deformable sensor by modifying air pressure within the sensor cavity, based on modifiable strength of the magnetic object to attract the magnetically-attractable particles.

Deformable sensors and methods for modifying run-time membrane stiffness using magnetic attraction

Devices, systems and methods for calibrating a deformable sensor are disclosed herein. A calibration device for calibrating a deformable sensor includes a frame, at least one actuator supported by the frame, where the at least one actuator comprises a drive mechanism operatively coupled to a probe portion, and an electronic control unit communicatively coupled to the drive mechanism of the at least one actuator. The electronic control unit is configured to cause the drive mechanism to move the probe portion a predetermined position to form a predetermined contact surface defined by an end of the probe portion of the actuator.

Calibration device and calibration procedure for fluid filled sensor

A system for calibrating a deformable sensor is provided. The system includes a deformable sensor including a housing, a deformable membrane coupled to an upper portion of the housing, and an enclosure defined by the housing and the deformable member; an imaging sensor configured to capture an image of the deformable membrane of the deformable sensor; and a controller. The enclosure is configured to be filled with a medium. The controller is configured to: receive the image of the deformable membrane of the deformable sensor; determine whether a contour of the deformable membrane in the image of the deformable membrane of the deformable sensor corresponds to a predetermined contour; and adjust a volume of the medium in the enclosure of the deformable sensor in response to the determination that the contour of the deformable membrane is different from the predetermined contour.

Methods and systems for calibrating deformable sensors using camera

A deformable sensor is provided. The deformable sensor comprises a deformable member defining an enclosure that is configured to be filled with a medium, a mechanical component disposed within the enclosure, and an optical sensor coupled to the mechanical component positioned with the enclosure. In embodiments, the mechanical component is configured to rotate at least from a first position to a second position, and the optical sensor is configured to capture first portion data associated with a first portion of the deformable member at the first position and second portion data associated with a second portion of the deformable member at the second position.

Growing community of inventors

Arlington, MA, United States of America

Naveen Suresh Kuppuswamy