Robert Boschgmbh

Robert Bosch Gmbh

Lincan Zou

Liu Ren

Liang Gou

Zeng Dai

Axel Wendt

Nanxiang Li

Huan Song

Cheng Zhang

Wenbin He

Fuliang Weng

Zhe Feng

Zhongnan Shen

Baoshi Yan

Yen-Lin Chen

Bilal Alsallakh

Zi'ang Ding

Shen Yan

A visual analytics workflow and system are disclosed for assessing, understanding, and improving deep neural networks. The visual analytics workflow advantageously enables interpretation and improvement of the performance of a neural network model, for example an image-based objection detection and classification model, with minimal human-in-the-loop interaction. A data representation component extracts semantic features of input image data, such as colors, brightness, background, rotation, etc. of the images or objects in the images. The input image data are passed through the neural network to obtain prediction results, such as object detection and classification results. An interactive visualization component transforms the prediction results and semantic features into interactive and human-friendly visualizations, in which graphical elements encoding the prediction results are visually arranged depending on the extracted semantic features of input image data.

Visual analytics system to assess, understand, and improve deep neural networks

Methods and systems for performing function testing for moveable objects. One system includes an electronic processor configured to access a driving scene including a moveable object. The electronic processor is also configured to perform spatial representation learning on the driving scene. The electronic processor is also configured to generate an adversarial example based on the learned spatial representation. The electronic processor is also configured to retrain the deep learning model using the adversarial example and the driving scene.

Function testing for movable objects in safety critical applications with spatial representation learning and adversarial generation

Visual analytics tool for updating object detection models in autonomous driving applications. In one embodiment, an object detection model analysis system including a computer and an interface device. The interface device includes a display device. The computer includes an electronic processor that is configured to extract object information from image data with a first object detection model, extract characteristics of objects from metadata associated with image data, generate a summary of the object information and the characteristics, generate coordinated visualizations based on the summary and the characteristics, generate a recommendation graphical user interface element based on the coordinated visualizations and a first one or more user inputs, and update the first object detection model based at least in part on a classification of one or more individual objects as an actual weakness in the first object detection model to generate a second object detection model for autonomous driving.

Visual analytics platform for updating object detection models in autonomous driving applications

A system and method for domain adaptation involves a first domain and a second domain. A machine learning system is trained with first sensor data and first label data of the first domain. Second sensor data of a second domain is obtained. Second label data is generated via the machine learning system based on the second sensor data. Inter-domain sensor data is generated by interpolating the first sensor data of the first domain with respect to the second sensor data of the second domain. Inter-domain label data is generated by interpolating first label data of the first domain with respect to second label data of the second domain. The machine learning system is operable to generate inter-domain output data in response to the inter-domain sensor data. Inter-domain loss data is generated based on the inter-domain output data with respect to the inter-domain label data. Parameters of the machine learning system are updated upon optimizing final loss data that includes at least the inter-domain loss data. After domain adaptation, the machine learning system, which is operable in the first domain, is adapted to generate current label data that identifies current sensor data of the second domain.

System and method for unsupervised domain adaptation with mixup training

A method for generating graphics of a three-dimensional (3D) virtual environment includes: receiving, with a processor, a first camera position in the 3D virtual environment and a first viewing direction in the 3D virtual environment; receiving, with the processor, weather data including first precipitation information corresponding to a first geographic region corresponding to the first camera position in the 3D virtual environment; defining, with the processor, a bounding geometry at first position that is a first distance from the first camera position in the first viewing direction, the bounding geometry being dimensioned so as to cover a field of view from the first camera position in the first viewing direction; and rendering, with the processor, a 3D particle system in the 3D virtual environment depicting precipitation only within the bounding geometry, the 3D particle system having features depending on the first precipitation information.

Method and system for efficient rendering of 3D particle systems for weather effects

A system includes a camera configured to obtain image information from objects. The system also includes a processor in communication with the camera and programmed to receive an input data including the image information, encode the input via an encoder, obtain a latent variable defining an attribute of the input data, generate a sequential reconstruction of the input data utilizing at least the latent variable and an adversarial noise, obtain a residual between the input data and the sequential reconstruction utilizing a comparison of at least the input and the reconstruction to learn a mean shift in latent space, and output a mean shift indicating a test result of the input compared to the adversarial noise based on the comparison.

Semantic adversarial generation based function testing method in autonomous driving

Abnormal motions are detected in sensor data collected with respect to performance of repetitive human activities. An autoencoder network model is trained based on a set of standard activity. Repetitive activity is extracted from sensor data. A first score is generated indicative of distance of a repetition of the repetitive activity from the standard activity. The repetitive activity is used to retrain the autoencoder network model, using weights of the autoencoder network model as initial values, the weights being based on the training of the autoencoder network model using the set of standard activity. A second score is generated indicative of whether the repetition is an outlier as compared to other repetitions of the repetitive activity. A final score is generated based on a weighting of the first score and the second score.

Repetitive human activities abnormal motion detection

Motion windows are generated from a query activity sequence. For each of the motion windows in the query activity sequence, a corresponding motion window in the reference activity sequence is found. One or more difference calculations are performed between the motion windows of the query activity sequence and the corresponding motion windows in the reference activity sequence based on at least one criterion associated with physical meaning. Abnormality of the motion windows is determined based on the one or more difference calculations. A standardized evaluation result of the query activity sequence is output based on the detected abnormal motion windows in the query activity sequence.

System and method for standardized evaluation of activity sequences

Using a global optimization, a cycle within a frame buffer including frames corresponding to one or more cycles of query activity sequences is detected. The detection includes creating a plurality of cycle segmentations by recursively iterating through the frame buffer to identify candidate cycles corresponding to cycles of a reference activity sequence until the frame buffer lacks sufficient frames to create additional cycles, computing segmentation errors for each of the plurality of cycle segmentations, and identifying the detected cycle as the one of the plurality of cycle segmentations having a lowest segmentation error. Cycle duration data for the detected cycle is generated. Frames belonging to the detected cycle are removed from the frame buffer. The cycle duration data is output.

System and method for cycle duration measurement in repeated activity sequences

Weaknesses may be exposed in image object detectors. An image object is overlaid onto a background image at each of a plurality of locations, the background image including a scene in which the image objects can be present. A detector model is used to attempt detection of the image object as overlaid onto the background image, the detector model being trained to identify the image object in background images, the detection resulting in background scene detection scores indicative of likelihood of the image object being detected at each of the plurality of locations. A detectability map is displayed overlaid on the background image, the detectability map including, for each of the plurality of locations, a bounding box of the image object illustrated according to the respective detection score.

Visual analytics exposure of image object detector weakness

A method for generating graphics of a three-dimensional (3D) virtual environment includes: receiving, with a processor, weather data corresponding to a geographic region, the weather data including a sequence of precipitation intensity values, each precipitation intensity value being associated with a respective timestamp of a chronological sequence of timestamps; calculating, with the processor, a first precipitation accumulation value based on the sequence of precipitation intensity values, the first precipitation accumulation value corresponding to a first time; and rendering, with the processor, a depiction of accumulated precipitation in the 3D virtual environment, the depiction of accumulated precipitation depending on the first precipitation accumulation value.

Method and system for efficient rendering of accumulated precipitation for weather effects

A system for determining a quantitative accuracy of a test movement relative to a reference movement includes a display output device, a memory, and a processor operatively connected to the display output device and the memory. The memory stores motion capture data and programming instructions. The processor executes the programming instructions to determine a quantitative accuracy of the test movement relative to the reference movement. A method, executable by the processor, for determining the quantitative accuracy includes receiving, with the processor, motion capture data that includes the reference movement and the test movement. The motion data is split into individual movements, and the test movement is aligned with the reference movement. The processor computes a quantitative accuracy of the test movement relative to the reference movement, and generates, with the display output device, a visualization representative of the test movement. The computed accuracy is encoded into the visualization.

Method and system for automatic repetitive step and cycle detection for manual assembly line operations

A method and device for determining a footprint of a 3D structure are disclosed. The method includes: receiving mesh data for the 3D structure that includes vertices and edges that form polygons of the 3D structure; determining a connection graph including candidate nodes and candidate lines by (i) identifying all edges having a vertex that is less than threshold height value, and (ii) mapping the identified edges and vertices thereof onto a 2D plane; determining an adjacency list that indicates, for each candidate node, which other candidate nodes are connected to the candidate node by a candidate line; and generating a footprint of the 3D structure based on the connection graph and the adjacency list, the footprint including vertices corresponding to a selection of the candidate nodes and including edges corresponding to a selection of the candidate lines.

Method and device for efficient building footprint determination

A method for displaying graphics of clouds in a three-dimensional (3D) virtual environment includes generating a filtered texture based on a threshold filter applied to a cloud texture where the filter threshold corresponds to cloud coverage information in weather data of a geographic region. The method further includes mapping the filtered texture to a geometric surface corresponding to a sky dome in the 3D virtual environment, coloring a plurality of texels in the mapped filtered texture on the geometric surface stored in the memory based on an isotropic single-scatter color model, and generating a graphical depiction of the 3D virtual environment including at least a portion of the geometric surface corresponding to the sky dome with clouds based on the plurality of texels of the filtered texture that are colored and mapped to the geometric surface.

Method and system for efficient rendering of cloud weather effect graphics in three-dimensional maps

Methods for generating ground shadow and lighting effects for three-dimensional models include identifying polygon data for a three-dimensional model, generating a convex polygon around a base of the model, generating hard and soft shadow meshes in and around the base of the model, and rendering the model with the shadow meshes with a display device. Methods for generating wall shadows and lighting effects for the three dimensional models further include identifying an orientation and height of a polygon in the model that extends from a ground surface in a virtual environment, and rendering the model with a lighting texture applied to either the full polygon if the polygon height is less than a threshold height or to only a portion of the polygon below the threshold height if the polygon exceeds the threshold height.

Growing community of inventors

San Jose, CA, United States of America

Lincan Zou