Adobe, Inc.

Bentley Systems, Inc.

Universitè Laval

Marc-André Gardner

Kalyan Krishna Sunkavalli

Xiaohui Shen

Mehmet Yumer

Emiliano Gambaretto

Karl-Alexandre Jahjah

Simon Savary

Yannick Hold-Geoffroy

Jonathan Eisenmann

Jean-Francois Lalonde

Marc-André Lapointe

Evan Rausch-Larouche

Andrew Smith

Raphaël Melancon

André Villemaire

Olivier Bloch

Christian Gagne

Lucas Flett

In example embodiments, a hybrid classification/anomaly machine learning architecture is provided that combines a classification model and an anomaly model to perform an engineering task. The classification model and the anomaly model may be used in parallel, and their inference results compared, with their consistency used to improve confidence, and their inconsistency used to detect when additional training or other improvement is required and to capture data useful in such additional training/improvement.

Integrating machine learning classification models and machine learning anomaly models

In example embodiments, techniques are provided for using a combination of multiple ML models and signal processing to extract links and connectivity from a schematic diagram in an image-only format. A first ML model (i.e. link segmenter) may produce a first set of predictions about the positions of link segments in the schematic diagram (e.g., in the form of a segmentation map). A second ML model (i.e. keypoint detector) may produce a second set of predictions about starting and stopping points of link segments in the schematic diagram (e.g., in the form of one or more heatmaps). A signal processing module may combine the first set of predictions and the second set of predictions to produce a description of links and connectivity they provide (e.g., combining the segmentation map with data from the one or more heatmaps). The results of the combining may be saved as a graph connectivity matrix.

Techniques for extracting links and connectivity from schematic diagrams

In example embodiments, techniques are provided for using machine learning to extract machine-readable labels for text boxes and symbols in P&IDs in image-only formats. A P&ID data extraction application uses an optical character recognition (OCR) algorithm to predict labels for text boxes in a P&ID. The P&ID data extraction application uses a first machine learning algorithm to detect symbols in the P&ID and return a predicted bounding box and predicted class of equipment for each symbol. One or more of the predicted bounding boxes may be decimate by non-maximum suppression to avoid overlapping detections. The P&ID data extraction application uses a second machine learning algorithm to infer properties for each detected symbol having a remaining predicted bounding box. The P&ID data extraction application stores the predicted bounding box and a label including the predicted class of equipment and inferred properties in a machine-readable format.

Techniques for extracting machine-readable information from P and IDs

In example embodiments, techniques are provided for using machine learning to predict railroad track geometry exceedances to enable proactive maintenance. A machine learning model of a rail operational analytics application may be trained to directly output a probability of future railroad track geometry exceedances for each portion of track of a railroad. Training may be performed using all available railroad track data, and the task of selecting which data is relevant to predicting probability of railroad track geometry exceedances may be devolved to the machine learning model. Further, assumptions about the specific railroad and data characteristics may be avoided, providing the machine learning model flexibility, and allowing for dynamic changes in the problem formulation.

Techniques for predicting railroad track geometry exceedances

This disclosure relates to methods, non-transitory computer readable media, and systems that can render a virtual object in a digital image by using a source-specific-lighting-estimation-neural network to generate three-dimensional ('3D') lighting parameters specific to a light source illuminating the digital image. To generate such source-specific-lighting parameters, for instance, the disclosed systems utilize a compact source-specific-lighting-estimation-neural network comprising both common network layers and network layers specific to different lighting parameters. In some embodiments, the disclosed systems further train such a source-specific-lighting-estimation-neural network to accurately estimate spatially varying lighting in a digital image based on comparisons of predicted environment maps from a differentiable-projection layer with ground-truth-environment maps.

Generating light-source-specific parameters for digital images using a neural network

In example embodiments, techniques are provided for efficiently labeling, reviewing and correcting predictions for P&IDs in image-only formats. To label text boxes in the P&ID, the labeling application executes an OCR algorithm to predict a bounding box around, and machine-readable text within, each text box, and displays these predictions in its user interface. The labeling application provides functionality to receive a user confirmation or correction for each predicted bounding box and predicted machine-readable text. To label symbols in the P&ID, the labeling application receives user input to draw bounding boxes around symbols and assign symbols to classes of equipment. Where there are multiple occurrences of specific symbols, the labeling application provides functionality to duplicate and automatically detect and assign bounding boxes and classes. To label connections in the P&ID, the labeling application receives user input to define connection points at corresponding symbols and creates connections between the connection points.

Techniques for labeling, reviewing and correcting label predictions for PandIDS

Dynamically estimating light-source-specific parameters for digital images using a neural network

Systems and techniques for estimating illumination from a single image are provided. An example system may include a neural network. The neural network may include an encoder that is configured to encode an input image into an intermediate representation. The neural network may also include an intensity decoder that is configured to decode the intermediate representation into an output light intensity map. An example intensity decoder is generated by a multi-phase training process that includes a first phase to train a light mask decoder using a set of low dynamic range images and a second phase to adjust parameters of the light mask decoder using a set of high dynamic range image to generate the intensity decoder.

High dynamic range illumination estimation

Methods and systems are provided for using a single image of an indoor scene to estimate illumination of an environment that includes the portion captured in the image. A neural network system may be trained to estimate illumination by generating recovery light masks indicating a probability of each pixel within the larger environment being a light source. Additionally, low-frequency RGB images may be generated that indicating low-frequency information for the environment. The neural network system may be trained using training input images that are extracted from known panoramic images. Once trained, the neural network system infers plausible illumination information from a single image to realistically illumination images and objects being manipulated in graphics applications, such as with image compositing, modeling, and reconstruction.

Growing community of inventors

CA

Marc-André Gardner