Gm Global Technolgoy Operations, Inc.

Carnegie Mellon University

Jinsong Wang

Wende Zhang

Bakhtiar Brian Litkouhi

Qingrong Zhao

Ryan M Frakes

Kent S Lybecker

Jeffrey S Piasecki

Qi Zhang

Shuqing Zeng

Travis S Hester

Xiujie Gao

Wei Tong

Esther Anderson

Donald K Grimm

Shengbing Jiang

SeyedAlireza Kasaiezadeh Mahabadi

Wen-Chiao Lin

Guangyu J Zou

Mohsen Khalili

Adrienne M Williamson

Jingfu Jin

Nancy L Johnson

Paul Edward Krajewski

Xiaosong Huang

Upali Priyantha Mudalige

Xinyu Du

Charles A Green

Allan K Lewis

Shige Wang

Kausalya Singuru

Robert D Sims, Iii

Vijayakumar Bhagavatula

Jeffrey A Bozeman

Michael T Chaney, Jr

Mutasim A Salman

Unmesh Dutta Bordoloi

Brunno L Moretti

Stephen G Lusko

Timothy J Canner

Veronique C Prinet

Yang Zou

Xinhua Xiao

Jie Yin

Lee Jongho

David Wettergreen

James Clem

Tianyu Wang

A system for analyzing images includes a processing device including a receiving module configured to receive an image associated with a target domain, and a domain adaptation module configured to characterize one or more features represented in the received image based on a domain adaptation model. The domain adaptation model is generated using a machine learning algorithm to train the domain adaptation model, and the machine learning algorithm is configured to train the domain adaptation model based on one or more source domain images associated with a source domain, one or more previously acquired images associated with the target domain, and acquired characterization data associated with the target domain. The system also includes an output module configured to output the received image with characterization data identifying one or more features characterized by the domain adaptation module.

Domain adaptation for analysis of images

Examples of techniques for controlling a vehicle based on trailer position are disclosed. In one example implementation according to aspects of the present disclosure, a computer-implemented method includes extracting, by a processing device, a feature point on a trailer from an image captured by a camera associated with a vehicle, the trailer being coupled to the vehicle. The method further includes determining, by the processing device, a distance between the feature point on the trailer and a virtual boundary. The method further includes, responsive to determining that the distance between the feature point on the trailer and the virtual boundary is less than a threshold, controlling, by the processing device, the vehicle to cause the distance between the feature point on the trailer and the virtual boundary to increase.

Controlling a vehicle based on trailer position

Examples of techniques for controlling a vehicle based on trailer sway are disclosed. In one example implementation according to aspects of the present disclosure, a computer-implemented method includes estimating, by a processing device, an estimated articulation angle between a vehicle and a trailer coupled to the vehicle. The method further includes calculating, by the processing device, an expected articulation angle between the vehicle and the trailer. The method further includes comparing, by the processing device, the estimated articulation angle and the expected articulation angle to determine whether the trailer is experiencing trailer sway. The method further includes, responsive to determining that the trailer is experiencing sway, controlling, by the processing device, the vehicle to reduce the trailer sway.

Controlling a vehicle based on trailer sway

A vehicle subsystem includes an on-vehicle camera that is disposed to monitor a field of view (FOV) that includes a travel surface for the vehicle. A controller captures, via the on-vehicle camera, an image file associated with the FOV and segments the image file into a first set of regions associated with the travel surface and a second set of regions associated with an above-horizon portion. Image features on each of the first set of regions and the second set of regions are extracted and classified. A surface condition for the travel surface for the vehicle is identified based upon the classified extracted image features from each of the first set of regions and the second set of regions. Operation of the vehicle is controlled based upon the identified surface condition.

Method and apparatus for evaluating a vehicle travel surface

Examples of techniques for dynamically selecting a batch size used in vehicle camera image processing are disclosed. In one example implementation, a method includes generating, by a processing device, a batch table and a mode table. The method further includes determining, by the processing device, image processing performance requirements for a current mode of a vehicle using the mode table, the vehicle comprising a plurality of cameras configured to capture a plurality of images. The method further includes selecting, by the processing device, a batch size and a processing frequency based at least in part on the image processing performance requirements for the current mode of the vehicle. The method further includes processing, by an accelerator, at least a subset of the plurality of images based at least in part on the batch size and processing frequency.

Dynamic batch size selection for vehicle camera image processing

Systems, Methods and Apparatuses are provided for detecting surface conditions, which includes: an image scene captured by a camera wherein the image scene includes: a set of a plurality of regions of interest (ROIs); and a processor configured to receive the image scene to: extract at least a first and a second ROI from the set of the plurality of ROIs of the image scene; associate the first ROI with an above-horizon region and associate the second ROI with a surface region; analyze the first ROI and the second ROI in parallel for a condition related to an ambient lighting in the first ROI and for an effect related to the ambient lighting in the second ROI; and extract from the first ROI features of the condition of the ambient lighting and extract from the second ROI features of the effect of the ambient lighting on a surface region.

Systems, methods and apparatuses are provided for enhanced surface condition detection based on image scene and ambient light analysis

An autonomic vehicle control system is described, and includes a vehicle spatial monitoring system including a subject spatial sensor that is disposed to monitor a spatial environment proximal to the autonomous vehicle. A controller is in communication with the subject spatial sensor, and the controller includes a processor and a memory device including an instruction set. The instruction set is executable to evaluate the subject spatial sensor, which includes determining first, second, third, fourth and fifth SOH (state of health) parameters associated with the subject spatial sensor, and determining an integrated SOH parameter for the subject spatial sensor based thereupon.

Method and apparatus for monitoring of an autonomous vehicle

A vehicle includes a plurality of on-vehicle cameras, and a controller executes a method to evaluate a travel surface by capturing images for fields of view of the respective cameras. Corresponding regions of interest for the images are identified, wherein each of the regions of interest is associated with the portion of the field of view of the respective camera that includes the travel surface. Portions of the images are extracted, wherein each extracted portion is associated with the region of interest in the portion of the field of view of the respective camera that includes the travel surface and wherein one extracted portion of the respective image includes the sky. The extracted portions of the images are compiled into a composite image datafile, and an image analysis of the composite image datafile is executed to determine a travel surface state. The travel surface state is communicated to another controller.

A method is used to evaluate a camera-related subsystem in a digital network, e.g., aboard a vehicle or fleet, by receiving, via a camera diagnostic module (CDM), sensor reports from the subsystem and possibly from a door sensor, rain/weather sensor, or other sensor. The CDM includes data tables corresponding to subsystem-specific fault modes. The method includes evaluating performance of the camera-related subsystem by comparing potential fault indicators in the received sensor reports to one of the data tables, and determining a pattern of fault indicators in the reports. The pattern is indicative of a health characteristic of the camera-related subsystem. A control action is executed with respect to the digital network in response to the health characteristic, including recording a diagnostic or prognostic code indicative of the health characteristic. The digital network and vehicle are also disclosed.

Camera subsystem evaluation using sensor report integration

A vehicle guidance system facilitates maneuvering of an autonomous vehicle with respect to an object in a scene. The vehicle includes a steering apparatus having a range of angular positions and a multitude of actuators for controlling the dynamics of the vehicle. The system includes a steering angle sensor, a camera device, and a video processing module. The sensor is configured to monitor the angular position of the wheel. The device is configured to capture an original image of a scene having the object. The module is configured to receive and process the original image from the device, detect the object in the original image, receive and process the angular position from the sensor, generate a dynamic trajectory based on at least the angular position, orientate the dynamic trajectory with regard to the object, and operate at least one of the actuators to guide the vehicle along the dynamic trajectory.

Vehicle guidance system

A vehicle guidance system assists a driver in maneuvering a vehicle with respect to an object in a scene. The system includes a steering angle sensor, a camera device, a video processing module (VPM), and a human-machine interface (HMI). The sensor is configured to monitor the angular position of a vehicle wheel. The device is configured to capture an original image of a scene having the object. The VPM is configured to receive and process the original image from the device, detect the object in the original image, receive and process the angular position from the sensor, generate a vehicle trajectory based on the angular position, and orientate the trajectory with regard to the object. The HMI is configured to display a processed image associated with the original image and a trajectory overlay associated with the trajectory from the VPM. Together, the object is displayed in relation to the overlay.

Techniques for road scene primitive detection using a vehicle camera system are disclosed. In one example implementation, a computer-implemented method includes receiving, by a processing device having at least two parallel processing cores, at least one image from a camera associated with a vehicle on a road. The processing device generates a plurality of views from the at least one image that include a feature primitive. The feature primitive is indicative of a vehicle or other road scene entities of interest. Using each of the parallel processing cores, a set of primitives are identified from one or more of the plurality of views. The feature primitives are identified using one or more of machine learning and classic computer vision techniques. The processing device outputs, based on the plurality of views, result primitives based on the plurality of identified primitives from multiple views based on the plurality of identified entities.

Parallel scene primitive detection using a surround camera system

A method for determining wetness on a path of travel. A surface of the path of travel is captured by at least one image capture device. A plurality of wet surface detection techniques is applied to the at least one image. An analysis for each wet surface detection technique is determined in real-time of whether the surface of the path of travel is wet. Each analysis independently determines whether the path of travel is wet. Each analysis by each wet surface detection technique is input to a fusion and decision making module. Each analysis determined by each wet surface detection technique is weighted within the fusion and decision making module by comprehensive analysis of weather information, geology information, and vehicle motions. A wet surface detection signal is provided to a control device. The control device applies the wet surface detection signal to mitigate the wet surface condition.

Fusion-based wet road surface detection

A vehicle includes at least one imaging device configured to generate image data indicative of a vicinity of the vehicle. The vehicle also includes a user interface display configured to display image data from the at least one imaging device. A vehicle controller is programmed to monitor image data for the presence of moving external objects within the vicinity, and to activate the user interface display to display image data in response to detecting a moving external object in the vicinity while the vehicle is at a rest condition. The controller is also programmed to assign a threat assessment value based on conditions in the vicinity of the vehicle, and upload image data to an off-board server in response to the threat assessment value being greater than a first threshold.

Vehicle environment imaging systems and methods

A system and method are provided for detecting and identifying elongated objects relative to a host vehicle. The method includes detecting objects relative to the host vehicle using a plurality of object detection devices, identifying patterns in detection data that correspond to an elongated object, wherein the detection data includes data fused from at least two of the plurality of object detection devices, determining initial object parameter estimates for the elongated object using each of the plurality of object detection devices, calculating object parameter estimates for the elongated object by fusing the initial object parameter estimates from each of the plurality of object detection devices, and determining an object type classification for the elongated object by fusing the initial object parameter estimates from each of the plurality of object detection devices.

Detecting long objects by sensor fusion

Growing community of inventors

Troy, MI, United States of America

Jinsong Wang