Mitsubishi Electric Research Laboratories, Inc.

Mitsubishi Electric Corporation

Alan Sullivan

Tim Marks

Anoop Cherian

Ronald N Perry

William Yerazunis

Sarah F Frisken

Jonathan Le Roux

Chiori Hori

Huseyin Erdim

Siheng Chen

Anthony Vetro

Zafer Sahinoglu

Scott Bortoff

Jay E Thornton

Mouhacine Benosman

Jeroen Van Baar

Takaaki Hori

Abraham Goldsmith

Bret Harsham

John C Barnwell, Iii

Vijay Shilpiekandula

Teng-Yok Lee

Radu Ioan Corcodel

Tyler W Garaas

Siddarth Jain

Jun Wang

Liana Manukyan

Arun R Yoganandan

Goncalo José Dias Pais

Dmytro Konobrytskyi

Xiaolong Li

Kuan-Chuan Peng

An imaging controller is provided for segmenting instances from depth images including objects to be manipulated by a robot. The imaging controller includes an input interface configured to receive a depth image that includes objects, a memory configured to store instructions and a neural network trained to segment instances from the objects in the depth image, and a processor, coupled with the memory, configured to perform the instructions to segment a pickable instance using the trained neural network. The instructions include steps of selecting a tallest point in the depth image, defining a region using a shape such that the region surrounds the tallest point, sampling points in the region of the depth image, computing depth-geodesics between the tallest point and the sampled points, submitting the depth-geodesics to the neural network to segment the pickable instance among instances of the objects in the depth image, and an output interface configured to output a geometrical feature of the pickable instance to a manipulator controller of the robot.

Discriminative 3D shape modeling for few-shot instance segmentation

The present disclosure provides a system and a method for motion prediction for autonomous driving. The system disclosed herein provides an efficient deep-neural-network-based system to jointly perform perception and motion prediction from 3D point clouds. This system is able to take a pair of LiDAR sweeps as input and outputs for each point in the second sweep, both a classification of the point into one of a set of semantic classes, and a motion vector indicating the motion of the point within the world coordinate system. The system includes a spatiotemporal pyramid network, which extracts deep spatial and temporal features in a hierarchical fashion. The training of this system is regularized with spatial and temporal consistency losses. Thus providing an improved motion planner for autonomous driving applications.

System and method for motion prediction in autonomous driving

Systems, methods and devices for real-time contactless elevator service operation of an elevator includes a trained neural network (TNN) model. The TNN model is trained using a training processor with augmented datasets as a synthetic training dataset, to later perform elevator identifier recognition. The augmented data sets are generated from synthetic text images, the synthetic text images are augmented with different geometrical parameters and visual parameters to a predetermined number of variations in appearance to a set of training elevator identifiers. A camera captures a user image. A text image portion from the user image is extracted using the TNN model, and detects an elevator identifier in the extracted text image portion using the extracted text image portion and the TNN model. The detected elevator identifier is displayed for user confirmation or user cancellation, and upon user confirmation, generates a control command based the detected elevator identifier associated with an elevator service.

Contactless elevator service for an elevator based on augmented datasets

A system for parameter tuning for robotic manipulators is provided. The system includes an interface configured to receive a task specification, a plurality of physical parameters, and a plurality of control parameters, wherein the interface is configured to communicate with a real-world robot via a robot controller. The system further includes a memory to store computer-executable programs including a robot simulation module, a robot controller, and an auto-tuning module a processor, in connection with the memory. In this case, the processor is configured to acquire, in communication with the real-world robot, state values of the real-world robot, state values of the robot simulation module, simultaneously update, by use of a predetermined optimization algorithm with the auto-tuning module, an estimate of one or more of the physical, and said control parameters, and store the updated parameters.

Simulation-in-the-loop tuning of robot parameters for system modeling and control

System and method for generating verisimilar images from real depth images. Train a generative adversarial neural network (GAN) by accessing test depth images having identical instances as instances of a real depth image. Input the test depth images in the generator to generate estimated depth images representing an implicit three-dimensional model of the object. Input, each estimated depth image into a discriminator to obtain a loss and into a pose encoder to obtain a matching loss. Iteratively repeat processes until the losses are minimized to a threshold, to end training. Identify the instances in the real image using the trained GAN pose encoder, to produce a pose transformation matrix for each instance in the real image. Identify pixels in the depth images corresponding to the instances of the real image and merge the pixels for the depth images to form an instance segmentation map for the real depth image.

InSeGAN: a generative approach to instance segmentation in depth images

A navigation system for providing driving instructions to a driver of a vehicle traveling on a route is provided. The driving instructions are generated by executing a multimodal fusion method that comprises extracting features from sensor measurements, annotating the features with directions for the vehicle to follow the route with respect to objects sensed by the sensors, and encoding the annotated features with a multimodal attention neural network to produce encodings. The encodings are transformed into a common latent space, and the transformed encodings are fused using an attention mechanism producing an encoded representation of the scene. The method further comprises decoding the encoded representation with a sentence generation neural network to generate a driving instruction and submitting the driving instruction to an output device.

Method and system for scene-aware interaction

Embodiments of the present disclosure discloses a scene-aware video encoder system. The scene-aware encoder system transforms a sequence of video frames of a video of a scene into a spatio-temporal scene graph. The spatio-temporal scene graph includes nodes representing one or multiple static and dynamic objects in the scene. Each node of the spatio-temporal scene graph describes an appearance, a location, and/or a motion of each of the objects (static and dynamic objects) at different time instances. The nodes of the spatio-temporal scene graph are embedded into a latent space using a spatio-temporal transformer encoding different combinations of different nodes of the spatio-temporal scene graph corresponding to different spatio-temporal volumes of the scene. Each node of the different nodes encoded in each of the combinations is weighted with an attention score determined as a function of similarities of spatio-temporal locations of the different nodes in the combination.

Scene-aware video encoder system and method

A distributed machining simulation partitions a model of the workpiece to produce a set of sub-workpieces and modifies the machining instructions for each sub-workpiece to produce a set of groups of modified machining instructions different from the machining instructions. Each group of the modified machining instructions is associated with a sub-workpiece and includes effective machining instructions that move the machining tool through the sub-workpiece and at least one fake machining instruction that moves the machining tool through the model of the workpiece outside of the sub-workpiece. The distributed machining simulation remote hosts to simulate the machining of the workpiece using at least one group of the modified machining instructions to produce geometries of a machining simulation of the workpiece, each geometry is associated with a corresponding sub-workpiece and reconstructs a representation of the machined workpiece using a portion of each geometry within the volume of the corresponding sub-workpiece.

System and method for distributed machining simulation

A method determines a feedrate of a tool machining a workpiece according to a path. The method partitions the path into a set of segments, such that within each segment a function of engagement of the tool and the workpiece is substantially constant. Next, the method determines a feedrate for each segment in the set.

System and method for determining feedrates of machining tools

A method calibrates a laser processing machine by commanding a scan head to direct a laser beam to a desired position, then senses an actual position of the laser beam directly using a position sensitive detector (PSD) after the scan head is positioned. A relative position between the scan head and PSD is altered using one or more linear actuators. A position feedback loop is closed around the linear actuators so that the relative position of the laser beam on the PSD is reduced to zero. The actual position of the laser spot is then measured indirectly by encoders attached to linear axes of the laser processing machine. The actual position is stored in a memory. An error is determined as a difference between the desired position and the actual position. Compensation coefficients are determined from the error and stored for later use during operation of the laser processing machine.

System and method for calibrating laser processing machines

A method and system renders a 3D model of a 3D object as an image including image samples. The 3D model is a distance field, and the distance field includes surface cells representing portions of a surface of the 3D object. A set of image samples in the image is determined in an object-order phase for each surface cell. Each surface cell is then processed independent of other surface cells. The processing casts a ray, in an image order phase from each image sample in the set of image samples, through the surface cell to determine a contribution of the surface cell to the image sample, and then the image is rendered.

Method and system for rendering 3D distance fields

A method associates a cell in the composite adaptive distance field (cADF) with a machining instruction that changes a type of the cell or a type of a distance field in the cell in forming a composite surface of the workpiece to produce at least part of an association, and associates the distance field in the cell with the machining instruction changing the type of the distance field in forming the composite surface of the workpiece to produce at least part of the association. In response to receiving a command to undo a simulation of the machining to the intermediate machining instruction, a subset of cells and a subset of distance fields forming the composite surface of the workpiece at a time of the simulation by the intermediate machining instruction are identified using the association and the intermediate representation of the workpiece is determined using the subset of cells and the subset of distance fields.

System and method for performing undo and redo operations during machining simulation

A computer program product for processing a model of an object according to a set of instructions includes a non-transitory computer-readable memory storing a model of an object represented by a hybrid adaptively sampled distance field (ADF), wherein the model includes a hierarchy of cells, wherein at least one cell includes a set of distance functions forming at least part of a boundary of the object and a set of distance samples of at least some of the distance functions, such that a processor executing the set of instructions processes the model of the object.

Hybrid adaptively sampled distance fields

A method simulates a machining of a workpiece by a tool. The method populates a swept volume representing a motion of the tool in relation to the workpiece with a set of rays, and determines a volume removed from the workpiece by the swept volume based on intersections of at least some rays with the workpiece. A model of the workpiece is modified based on the removed volume.

Analyzing volume removed during machining simulation

A computer simulation of machining of an object by a motion of a tool is described. The object is represented by a boundary representation (BP) object, wherein the BP object includes a boundary representation of a surface of the object, wherein the motion is represented by a set of swept volumes, wherein the set of swept volumes includes a first set of implicit functions defining a surface of the set of swept volumes. The simulation includes determining a proxy object having a second set of implicit functions defining a surface of the proxy object; simulating the machining of the proxy object with the set of swept volumes to produce a machined proxy (MP) object having a third set of implicit functions defining a surface of the MP object; rendering an image of a Boolean intersection between the MP object and the BP object.

Growing community of inventors

Middleton, MA, United States of America

Alan Sullivan