Palo Alto Research Center Inc.

Xerox Corporation

Palo Alto Research Center Incorporate

Robert Roy Price

Matthew Adam Shreve

Rong Zhou

Johan De Kleer

Juan Liu

Kurt Edward Partridge

Lukas Daniel Kuhn

Minh Binh Do

Raja Bala

Daniel H Greene

John Hanley

Nicolas B Ducheneaut

James M A Begole

Bryan Thomas Preas

Sricharan Kallur Palli Kumar

Christoforos Somarakis

Tim C Schmidt

Svyatoslav Korneev

Hoda M A Eldardiry

Adrian Lew

Haitham A S Hindi

Francisco Eduardo Torres

Eric Scott Hamby

Ajay Raghavan

Pei Li

Gaurang R Gavai

Jin Sun

Palghat Srinivas Ramesh

Rui Zhang

Florent C Perronnin

Robert Michael Lofthus

Kristine A German

Claudio Cifarelli

Evgeniy Bart

Matt Patterson

Eli Wilson

Brendan Mcnamara

Hsuan-Ting Wu

Anirban Roy

Kent J Evans

Eric Saund

Ed H Chi

Lester D Nelson

Maurice Kyojin Chu

Christian Fritz

Michael Roberts

Rajinderjeet Singh Minhas

Haitham Hindi

Victoria M E Bellotti

Hoda Eldardiry

Shiwali Mohan

Jeyasri Subramanian

James D Glasnapp

Shekhar Gupta

David Garcia

Johan Dekleer

Hoda Eldadiry

Edo Collins

Nicholas B Ducheneaut

Sumeet Menon

Lukas Kuhn

A system captures, by a recording device, a scene with physical objects, the scene displayed as a three-dimensional (3D) mesh. The system marks 3D annotations for a physical object and identifies a mask. The mask indicates background pixels corresponding to a region behind the physical object. Each background pixel is associated with a value. The system captures a plurality of images of the scene with varying features, wherein a respective image includes: two-dimensional (2D) projections corresponding to the marked 3D annotations for the physical object; and the mask based on the associated value for each background pixel. The system updates the value of each background pixel with a new value. The system trains a machine model using the respective image as generated labeled data, thereby obtaining the generated labeled data in an automated manner based on a minimal amount of marked annotations.

Method and system for facilitating generation of background replacement masks for improved labeled image dataset collection

One embodiment can provide a system for augmenting perceptions of a machine sensing system. During operation, one or more sensors of the system can obtain sensory information associated with a physical system. The system can determine a state of a first component of the physical system based on the obtained sensory information; select, from a rule database, one or more logical rules associated with at least the first component; and augment a perception of the machine sensing system toward the physical system based on both the determined state of the first component and the selected one or more logical rules, thereby facilitating the machine sensing system to make a decision associated with the physical system.

Method and system for rule-based augmentation of perceptions

A method of labeling training data includes inputting a plurality of unlabeled input data samples into each of a plurality of pre-trained neural networks and extracting a set of feature embeddings from multiple layer depths of each of the plurality of pre-trained neural networks. The method also includes generating a plurality of clusterings from the set of feature embeddings. The method also includes analyzing, by a processing device, the plurality of clusterings to identify a subset of the plurality of unlabeled input data samples that belong to a same unknown class. The method also includes assigning pseudo-labels to the subset of the plurality of unlabeled input data samples.

Using multiple trained models to reduce data labeling efforts

Image processing techniques for determining print quality for a 3D printer are disclosed. An example method includes obtaining an image of material jetted from a nozzle of the 3D printer. The method also includes binarizing the image to distinguish background features from foreground features contained in the image. The method also includes determining, by a processing device, a jetting quality based on the binarized image.

Techniques for binarizing images to determine jetting quality in additive manufacturing

A system determines an input video and a first annotated image from the input video which identifies an object of interest. The system initiates a tracker based on the first annotated image and the input video. The tracker generates, based on the first annotated image and the input video, information including: a sliding window for false positives; a first set of unlabeled images from the input video; and at least two images with corresponding labeled states. A semi-supervised classifier classifies, based on the information, the first set of unlabeled images from the input video. If a first unlabeled image is classified as a false positive, the system reinitiates the tracker based on a second annotated image occurring in a frame prior to a frame with the false positive. The system generates an output video comprising the input video displayed with tracking on the object of interest.

Method and system for semi-supervised state transition detection for object tracking

A system and method provide extractions of regions of interest from images hand annotated by reviewers by lifting annotations from images, filtering out text labels, reconstructing continuous closed boundaries, and marking the contained region.

Automatic extraction of regions of interest from annotated images with occluding labels

Techniques for determining print quality for a 3D printer are disclosed. An example method includes obtaining an image of a stream of material jetted from a nozzle of the 3D printer, and binarizing the image to distinguish background features from foreground features contained in the image. The method also includes identifying elements of jetted material in the foreground features, and computing statistical data characterizing the identified elements. The method also includes generating a quality score of jetting quality based on the statistical data and controlling the 3D printer based on the quality score. The quality score indicates a degree to which the elements of jetted material form droplets of a same size, shape, alignment, and jetting frequency.

Image processing techniques for jetting quality classification in additive manufacturing

A method includes defining a model for a liquid while the liquid is positioned at least partially within a nozzle of a printer. The method also includes synthesizing video frames of the liquid using the model to produce synthetic video frames. The method also includes generating a labeled dataset that includes the synthetic video frames and corresponding model values. The method also includes receiving real video frames of the liquid while the liquid is positioned at least partially within the nozzle of the printer. The method also includes generating an inverse mapping from the real video frames to predicted model values using the labeled dataset. The method also includes reconstructing the liquid in the real video frames based at least partially upon the predicted model values.

Characterizing liquid reflective surfaces in 3D liquid metal printing

A method of labeling a dataset includes inputting a testing set comprising a plurality of input data samples into a plurality of pre-trained machine learning models to generate a set of embeddings output by the plurality of pre-trained machine learning models. The method further includes performing an iterative cluster labeling algorithm that includes generating a plurality of clusterings from the set of embeddings, analyzing the plurality of clusterings to identify a target embedding with a highest duster quality, analyzing the target embedding to determine a compactness for each of the plurality of clusterings of the target embedding, and identifying a target cluster among the plurality of clusterings of the target embedding based on the compactness. The method further includes assigning pseudo-labels to the subset of the plurality of input data samples that are members of the target duster.

One embodiment provides a system that facilitates efficient collection of training data. During operation, the system obtains, by a recording device, a first image of a physical object in a scene which is associated with a three-dimensional (3D) world coordinate frame. The system marks, on the first image, a plurality of vertices associated with the physical object, wherein a vertex has 3D coordinates based on the 3D world coordinate frame. The system obtains a plurality of second images of the physical object in the scene while changing one or more characteristics of the scene. The system projects the marked vertices on to a respective second image to indicate a two-dimensional (2D) bounding area associated with the physical object.

System and method using augmented reality for efficient collection of training data for machine learning

A three-dimensional (3D) printer includes a nozzle and a camera configured to capture a real image or a real video of a liquid metal while the liquid metal is positioned at least partially within the nozzle. The 3D printer also includes a computing system configured to perform operations. The operations include generating a model of the liquid metal positioned at least partially within the nozzle. The operations also include generating a simulated image or a simulated video of the liquid metal positioned at least partially within the nozzle based at least partially upon the model. The operations also include generating a labeled dataset that comprises the simulated image or the simulated video and a first set of parameters. The operations also include reconstructing the liquid metal in the real image or the real video based at least partially upon the labeled dataset.

A system is provided which obtains images of a physical object captured by an AR recording device in a 3D scene. The system measures a level of diversity of the obtained images, for a respective image, based on at least: a distance and angle; a lighting condition; and a percentage of occlusion. The system generates, based on the level of diversity, a first visualization of additional images to be captured by projecting, on a display of the recording device, first instructions for capturing the additional images using the AR recording device. The system trains a model based on the collected data. The system performs an error analysis on the collected data to estimate an error rate for each image of the collected data. The system generates, based on the error analysis, a second visualization of further images to be captured. The model is further trained based on the collected data.

System and method for interactive feedback in data collection for machine learning in computer vision tasks using augmented reality

Embodiments described herein provide a system for facilitating dynamic assistance to a user in an augmented reality (AR) environment of an AR device. During operation, the system detects a first element of an object using an object detector, wherein the object is associated with a task and the first element is associated with a step of the task. The system then determines an orientation and an alignment of the first element in the physical world of the user, and an overlay for the first element. The overlay can distinctly highlight one or more regions of the first element and indicate how the first element fits in the object. The system then applies the overlay to the one or more regions of the first element at the determined orientation in the AR environment.

Alignment- and orientation-based task assistance in an AR environment

One embodiment provides a robotic system comprising: a robot, the robot further comprising a moveable robotic arm that moves within the robot's reference space; a depth-sensing camera, the camera having a reference frame that is in substantial view of the robot's reference space; a controller, the controller further comprising a processor and a computer readable memory that comprises instructions such that, when read by the controller, the controller inputs image data from the camera and sends signals to the moveable robotic arm, the instructions further comprising the steps of: calibrating the camera to the robot by instructing the robot to engage in a number of robot poses; extracting the location of the robot poses to obtain the robot poses in the camera reference frame; and creating a transformation that transforms robot points afterwards to camera points.

Target-free RGBD camera alignment to robots

A method of labeling a dataset of input samples for a machine learning task includes selecting a plurality of pre-trained machine learning models that are related to a machine learning task. The method further includes processing a plurality of input data samples through each of the pre-trained models to generate a set of embeddings. The method further includes generating a plurality of clusterings from the set of embeddings. The method further includes analyzing, by a processing device, the plurality of clusterings to extract superclusters. The method further includes assigning pseudo-labels to the input samples based on analysis.

Growing community of inventors

Palo Alto, CA, United States of America

Robert Roy Price