Gm Global Technolgoy Operations, Inc.

Mohsen Khalili

Bakhtiar Brian Litkouhi

Qingrong Zhao

Jinsong Wang

Hao Yu

Wende Zhang

Xinyu Du

Yao Hu

Andrew W Averhart

Yousef A Omar

Ismail Hamieh

Jingfu Jin

Xiaobo Guo

A vision system for a vehicle includes a camera configured to generate one of distorted and undistorted images. An alignment system is configured to generate a region of interest (ROI) for the camera, generate a bird's eye view (BEV) homography to convert undistorted points in the ROI to BEV points, generate a BEV image based on the one of distorted and undistorted images and one of the BEV homography and an updated BEV homography, adaptively update the BEV homography to generate the updated BEV homography, detect current locations of features in the BEV image, predict future locations of the features for a next BEV image, compare the current locations of the features in the BEV image with prior predicted locations, and convert the BEV points in the BEV image to an undistorted points in a perspective undistorted image using an inverse of the updated BEV homography.

Vehicle vision system including camera to ground alignment using bird's eye view image

Methods and systems are provided for controlling a vehicle action based on a condition of a road on which a vehicle is travelling, including: obtaining first sensor data as to a surface of the road from one or more first sensors onboard the vehicle; obtaining second sensor data from one or more second sensors onboard the vehicle as to a measured parameter pertaining to operation of the vehicle or conditions pertaining thereto; generating a plurality of road surface channel images from the first sensor data, wherein each road surface channel image captures one of a plurality of facets of properties of the first sensor data; classifying, via a processor using a neural network model, the condition of the road on which the vehicle is travelling, based on the measured parameter and the plurality of road surface channel images; and controlling a vehicle action based on the classification of the condition of the road.

Apparatus and methodology of road condition classification using sensor data

A calibration system for a camera and an inertial sensor, respectively rigidly affixed to a vehicle, includes a controller having a processor and tangible, non-transitory memory on which instructions are recorded. The camera is adapted to capture one or more images and the inertial sensor is adapted to obtain motion data, including an angular velocity vector of the vehicle. The controller is configured to receive respective data from the camera and the inertial sensor. The controller is configured to perform adaptive feature selection on the one or more images, including extracting one or more regions of interest respectively from the images and calculating a respective optimal quality level for each of the regions of interest. The camera is aligned to the inertial sensor based in part on the adaptive feature selection and the respective data from the inertial sensor.

Targetless calibration of camera and inertial sensor in a vehicle

A camera of a vehicle has a field of view. A perception module of the vehicle is configured to determine a location of an object relative to the vehicle based on a model for the camera and a camera ground truth orientation of the camera. A calibration module is configured to: wirelessly receive images from the camera; determine a vehicle camera target based on the images; when the vehicle camera target is aligned with a calibration module target on the calibration module, determine a present orientation of the calibration module measured using a gyroscope of the calibration module; when at least one component of the present orientation of the calibration module is different than the at least one component of an expected orientation of the calibration module, determine an updated camera ground truth orientation of the camera; and wirelessly transmit the updated ground truth orientation to the vehicle.

Systems and methods for calibrating vehicle cameras using external smart sensor

A vehicle subsystem includes an on-vehicle camera that is disposed to monitor a field of view (FOV) that includes a travel surface for the vehicle. A controller captures, via the on-vehicle camera, an image file associated with the FOV and segments the image file into a first set of regions associated with the travel surface and a second set of regions associated with an above-horizon portion. Image features on each of the first set of regions and the second set of regions are extracted and classified. A surface condition for the travel surface for the vehicle is identified based upon the classified extracted image features from each of the first set of regions and the second set of regions. Operation of the vehicle is controlled based upon the identified surface condition.

Method and apparatus for evaluating a vehicle travel surface

Systems, Methods and Apparatuses are provided for detecting surface conditions, which includes: an image scene captured by a camera wherein the image scene includes: a set of a plurality of regions of interest (ROIs); and a processor configured to receive the image scene to: extract at least a first and a second ROI from the set of the plurality of ROIs of the image scene; associate the first ROI with an above-horizon region and associate the second ROI with a surface region; analyze the first ROI and the second ROI in parallel for a condition related to an ambient lighting in the first ROI and for an effect related to the ambient lighting in the second ROI; and extract from the first ROI features of the condition of the ambient lighting and extract from the second ROI features of the effect of the ambient lighting on a surface region.

Growing community of inventors

Sterling Heights, MI, United States of America

Mohsen Khalili