Gm Global Technolgoy Operations, Inc.

Wei Tong

Shuqing Zeng

Shige Wang

Upali Priyantha Mudalige

Donald K Grimm

Alaa M Khamis

Roman L Millett

Wende Zhang

Mohannad F Murad

Jinsong Wang

Yao Hu

Xiaofeng Frank Song

Wen-Chiao Lin

Siddhartha Gupta

Bakhtiar Brian Litkouhi

Qingrong Zhao

David R Petrucci

Gregg R Kittinger

Jacob Alan Bond

Lawrence A Bush

Fahim Javid

Yasen Hu

Hojjat Izadi

Sourav Dey

Praveen Choudhury

Ming Yang

Fan Bai

Paul Edward Krajewski

David T Proefke

Yiran Hu

Sai Vishnu Aluru

Steven E Muldoon

Richard Gordon

Unmesh Dutta Bordoloi

Jiang L Du

Sravan Daruri

Farui Peng

Robert D Burns

Jiang-Ling Du

Zhuoning Yuan

Zachariah E Tyree

Stephen N McKinnie

Chengqi Bian

Gopichand Movva

Radhika Ciripanga

A system for determining opportunity and capability for identified adversarial behavior within an autonomous vehicle, comprising a pedestrian detection system, a vehicle controller and an adversarial intent algorithm adapted to determine a risk level of adversarial behavior from at least one pedestrian within proximity of the autonomous vehicle, and, when the adversarial intent algorithm determines that there is a risk of adversarial behavior, to determine a risk level of the adversarial behavior based on opportunity and capability.

Vulnerable road user's adversarial behavior opportunity and capability

A system for responding to adversarial behavior within an autonomous vehicle includes a pedestrian detection system in communication with a vehicle controller and an adversarial intent algorithm adapted to determine a risk level of adversarial behavior from at least one pedestrian within proximity of the autonomous vehicle, the vehicle controller adapted to provide proactive and reactive actions to be performed by the autonomous vehicle in response to the adversarial behavior based on the risk level of the adversarial behavior.

Responses to vulnerable road user's adversarial behavior

A dynamic side blind zone method includes determining that a host vehicle is approaching a first lane that is nonparallel to a second lane. The host vehicle is moving in the second lane. The method further includes activating an adaptive side blind zone alert system of the host vehicle in response to determining that the host vehicle is approaching the first lane that is nonparallel to the second lane, determining a warning zone in response to activating the adaptive side blind zone alert system of the host vehicle, and detecting a remote vehicle inside the warning zone after determining the warning zone. The remote vehicle is moving in the first lane. The method further includes providing an alert to a vehicle user of the host vehicle in response to detecting that the remote vehicle is inside the warning zone.

Adaptive side blind zone alert system and method

A perception processing system includes a memory and a main controller. The main controller includes modules and implements a data processing pipeline including algorithm stages, which are executed in parallel relative to sets of data and are executed sequentially relative to each of the sets of data. The algorithm stages share resources of the modules and the memory to process the sets of data and generate perception information. One of the modules executes global and local controllers. The global controller sets a processing rate for the local controllers. The local controllers monitor current processing rates of the algorithm stages. When one of the current processing rates is less than the set processing rate, the corresponding one of the local controllers sends a first signal to the global controller and in response the global controller sends a broadcast signal to the local controllers to adjust the current processing rates.

Perception processing with multi-level adaptive data processing flow rate control

A system for identifying adversarial behavior directed to an autonomous vehicle includes a pedestrian detection system in communication with a vehicle controller adapted to identify a pedestrian within proximity of the autonomous vehicle, track the pedestrian, determine if the pedestrian is intending to cross in front of the autonomous vehicle, and if the pedestrian tracking system determines that the pedestrian is not intending to cross in front of the autonomous vehicle, the pedestrian tracking system is further adapted to continue tracking the pedestrian, and if the pedestrian tracking system determines that the pedestrian is intending to cross in front of the autonomous vehicle, the pedestrian tracking system is further adapted to determine, with an adversarial intent algorithm, if the pedestrian is exhibiting any adversarial behavior.

Vulnerable road user's adversarial behavior recognition

A system comprises a computer including a processor and a memory. The memory includes instructions such that the processor is programmed to: receive intermediate concept constraints at a neural network and train the neural network with training data, training labels, and the at least one of the data constraint, the feature constraint, or the intermediate concept constraint.

Robust neural network learning system

A system comprises a computer including a processor and a memory. The memory includes instructions such that the processor is programmed to determine a pairwise region of interest feature similarity based on features extracted from a first cropped image portion and corresponding point cloud data and features extracted from a second cropped image portion and corresponding point cloud data. The processor is also programmed to determine a loss using a loss function based on the pairwise region of interest feature similarity, wherein the loss function corresponds to at least one a first deep neural network or a second deep neural network. The processor is also programmed to update at least one weight of the at least one of the first deep neural network or the second deep neural network based on the loss.

Computationally efficient unsupervised DNN pretraining

A method for validating an autonomous vehicle performance using nearby traffic patterns includes receiving remote vehicle data. The remote vehicle data includes at least one remote-vehicle motion parameter about a movement of a plurality of remote vehicles during a predetermined time interval. The method further includes determining a traffic pattern of the plurality of remote vehicles using the at least one remote-vehicle motion parameter. The method includes determining a similarity between the traffic pattern of the plurality of remote vehicles and movements of the host vehicle. Further, the method includes determining whether the similarity between the traffic pattern of the plurality of remote vehicles and movements of the host vehicle is less than a predetermined threshold. Also, the method includes commanding the host vehicle to adjust the movements thereof to match the traffic pattern of the plurality of remote vehicles.

Method and system for validating autonomous vehicle performance using nearby traffic patterns

An in-vehicle violence detection system includes a speech and non-speech audio recognition module capturing occupant threat words and non-speech audio events. In-vehicle accelerometers generate data analyzed in a shaking movement recognition module. A heart rate and breathing rate detection module measures physiological changes in occupant heart rates and breathing rhythms. An in-vehicle semantic scene recognition module analyzes occupant non-verbal interactions. Occupant threat indicators including an audible threat indicator are generated by the speech and non-speech audio event recognition module. Visual threat indicators are generated by the in-vehicle semantic scene recognition module. Physiological threat indicators are generated by the heart rate and breathing rate detection module. Vibration-based threat indicators are generated by the shaking movement recognition module. The four threat indicators are fused to estimate a threat level, consolidated by incorporating contextual information including a vehicle location and time. Different actions are taken based on the consolidated threat level.

In-vehicle multimodal violence detection for shared vehicles and self-driving shuttles

An on-demand, feature focused data collection system of an automobile vehicle includes at least one data collection device provided with an automobile vehicle. A data collection unit receives data from the at least one data collection device in response to a data collection request submitted by a user. A data scene group combines portions of the data received by the data collection unit and stored in the data collection unit of the automobile vehicle or at a remote server. A control logic device receives the data collection request and activates collection of the data. The control logic device includes: a data collection strategy which differs if different types of the data are available; a reference algorithm loaded or deployed; a storage capacity allocated for the data; and a storage policy employed to save the data.

System and apparatus for on-demand, feature focused data collection in autonomous vehicles and autonomous driving systems

Presented are embedded control systems with logic for computation and data sharing, methods for making/using such systems, and vehicles with distributed sensors and embedded processing hardware for provisioning automated driving functionality. A method for operating embedded controllers connected with distributed sensors includes receiving a first data stream from a first sensor via a first embedded controller, and storing the first data stream with a first timestamp and data lifespan via a shared data buffer in a memory device. A second data stream is received from a second sensor via a second embedded controller. A timing impact of the second data stream is calculated based on the corresponding timestamp and data lifespan. Upon determining that the timing impact does not violate a timing constraint, the first data stream is purged from memory and the second data stream is stored with a second timestamp and data lifespan in the memory device.

Intelligent vehicles with distributed sensor architectures and embedded processing with computation and data sharing

An automated driving system includes an object detection system. A neural network image encoder generates image embeddings associated with an image including an object. A neural network text encoder generates concept embeddings associated with each of a plurality of concepts. Each of the plurality of concepts is associated with one of at least two object classes. A confidence score module generates a confidence score for each of the plurality of concepts based on the image embeddings and the concept embeddings associated with the concept. An object class prediction module generates a predicted object class of the object based on an association between a set of concepts of the plurality of concepts having at least two of the highest values of the generated confidence scores and the one of the at least two object classes associated with a majority of the set of concepts.

Systems and methods for classifying detected objects in an image at an automated driving system

An automobile vehicle continuous validation system includes a backend collecting data from a vehicle fleet and wirelessly communicating with the vehicle fleet. The backend is in wireless communication with at least one client. A vehicle module is provided on-board individual ones of multiple automobile vehicles of the vehicle fleet and performing an on-board vehicle validation analysis. A fleet-based validation module provided either at the backend or cloud based manages data defining a configuration of and a capability of the multiple automobile vehicles of the vehicle fleet. A validation manager generates validation tasks based on a user's definition or a desired production of the validation tasks of the validation analysis and a fleet vehicle availability. A client-side module remote from the multiple automobile vehicles of the vehicle fleet has interface items applied by the at least one client seeking to perform the validation analysis.

System and method enabling validation-as-a-service for continuous validation of advanced driver assistance systems and autonomous vehicles

A system for an attention-based perception includes a camera device configured to provide an image of an operating environment of a vehicle. The system further includes a computerized device monitoring the image, analyzing sensor data to identify a feature in the image as corresponding to an object in the operating environment and assign a score for the feature based upon an identification, a location, or a behavior of the object. The computerized device is further operable to define candidate regions of interest upon the image, correlate the score for the feature to the candidate regions of interest to accrue a total region score, select some of the candidate regions for analysis based upon the total region scores, and analyze the portion of the candidate regions to generate a path of travel output. The system further includes a device controlling the vehicle based upon the output.

System and method for region of interest window generation for attention based perception

A perception system includes a perception module configured to capture first sensor data that includes data from at least one of an external sensor and a camera captured in a first period, a prediction module configured to receive the first sensor data, generate, based on the first sensor data, predicted sensor data for a second period subsequent to the first period, receive second sensor data for the second period, and output results of a comparison between the predicted sensor data and the second sensor data, and a diagnostic module configured to selectively identify a fault in the perception system based on the results of the comparison.

Perception system diagnosis using predicted sensor data and perception results

Growing community of inventors

Troy, MI, United States of America

Wei Tong