Gm Global Technolgoy Operations, Inc.

Jacob Alan Bond

Joseph F Szczerba

Manoj Sharma

Hannah Elizabeth Wagner

Kai-Han Chang

Wei Tong

John P Weiss

Siddhartha Gupta

Thomas A Seder

Sayyed Rouhollah Jafari Tafti

Ki Hyun Ahn

Akilesh Rajavenkatanarayanan

Zhuoning Yuan

Eduard Shpati

Alan D Hejl

An occupant output system includes at least one output device adapted to provide output directed to a single passenger, an occupant monitoring system to track location and movement of the passenger within the vehicle, at least one processor configured to execute program instructions, wherein the program instructions are configured to cause the at least one processor to receive data from the occupant monitoring system, analyze the data from the occupant monitoring system to determine if the passenger is vision-impaired, analyze the data from the occupant monitoring system to determine a location of the passenger within the vehicle and an orientation of a head of the passenger, and, when the passenger is determined to be vision impaired, generate an output for the passenger on the at least one output device, based on the location of the passenger within the vehicle and the orientation of the head of the passenger.

Systems and methods for communicating with vision-impaired vehicle occupants

A door handle locator system for a vehicle includes one or more modalities that provide sensory feedback to a user indicating a location of a door handle that is part of a door of the vehicle. The door handle locator system also includes one or more controllers in electronic communication with the one or more modalities. The one or more controllers execute instructions to receive a wireless signal indicating a location of the user with respect to the vehicle. The one or more controllers determine the user is approaching the vehicle based on the location of the user with respect to the vehicle. In response to determining the user is approaching the vehicle, the one or more controllers instruct the one or more modalities to provide the sensory feedback to the user indicating the location of the door handle.

Door handle locator system for a vehicle

A system for displaying information to an occupant of a vehicle includes a microphone, a camera for capturing images of an environment surrounding the vehicle, a display, and a controller in electrical communication with the microphone, the camera, and the display. The controller is programmed to receive a voice command from the occupant using the microphone, determine at least one characteristic of the requested object based on the voice command, capture an image of the environment using the camera, where the environment includes a relevant object, and to identify a location of the relevant object in the environment based at least in part on the image and the at least one characteristic of the requested object. The controller is further programmed to display a graphic based at least in part on the location of the relevant object in the environment using the display.

Identification of objects by color with ARHUD

A system comprises a computer including a processor and a memory. The memory includes instructions such that the processor is programmed to: receive intermediate concept constraints at a neural network and train the neural network with training data, training labels, and the at least one of the data constraint, the feature constraint, or the intermediate concept constraint.

Robust neural network learning system

A system for detecting a health condition of an occupant of a vehicle includes a plurality of vehicle sensors and a controller in electrical communication with the plurality of vehicle sensors. The controller is programmed to perform a plurality of measurements using the plurality of vehicle sensors determine a health risk probability based at least in part on the plurality of measurements. The health risk probability is a probability that the occupant of the vehicle has the health condition. The controller is further programmed to notify the occupant of the vehicle in response to determining that the health risk probability is greater than or equal to a predetermined health risk threshold.

Detection of health conditions in vehicle occupants

A method for mitigating an adversarial attack includes receiving input data. The input data includes sensor data from a plurality of sensors and map data. The method further includes monitoring, in real time, an environment around an autonomous vehicle to identify a region that is possibly subject to an adversarial attack and determining a probability of the adversarial attack in the region. The method further includes determining whether the probability of the adversarial attack in the region that is possibly subject to the adversarial attack is greater than a predetermined threshold and, in response, planning a motion of the autonomous vehicle by taking into account the adversarial attack to generate a planned motion. The method further includes controlling a host vehicle to move in accordance with the planned motion.

Algorithm to detect and mitigate real-time perceptual adversial attacks on autonomous vehicles

A system for displaying information for a vehicle includes a plurality of vehicle sensors, a display, and a controller in electrical communication with the plurality of vehicle sensors and the display. The controller is programmed to detect a relevant traffic signal in an environment surrounding the vehicle using the plurality of vehicle sensors, receive information about the relevant traffic signal using the plurality of vehicle sensors, determine a status of the relevant traffic signal based on the received information, and display a graphic based on the status of the relevant traffic signal using the display.

Augmented reality display for traffic signal awareness

A speech recognition method includes receiving audible data and user data. The audible data includes information about an utterance by the user. The user data includes information about movements by the user. The method further includes fusing the audible data and the user data to obtain fused data and determining at least one spoken word of the utterance based on the fused data.

System and method for integrating auditory and non-auditory inputs for adaptable speech recognition

A system for providing information to an occupant of a vehicle includes at least one vehicle sensor for determining information about an environment surrounding the vehicle, a display for providing information to the occupant, and a controller in electrical communication with the at least one vehicle sensor and the display. The controller is programmed to identify at least one driving rule associated with at least one portion of the environment surrounding the vehicle using the at least one vehicle sensor, determine an applicability state of the at least one driving rule, where the applicability state includes an applicable state and a non-applicable state, and display a graphic using the display in response to determining that the applicability state of the at least one driving rule is the applicable state.

Augmenting roadway markings and traffic signs for occupant awareness

A system for communicating fleet-specific features to one or more personal electronic devices includes one or more controllers in wireless communication with a centralized computer system including one or more databases for storing the fleet-specific features. The one or more controllers execute instructions to undergo a passive wireless interaction with the one or more personal electronic devices. The passive wireless interaction involves determining the one or more personal electronic devices are located within a predefined proximity around a vehicle without human interaction. In response to undergoing the passive wireless interaction, the one or more controllers transmit one or more fleet-specific features to the one or more personal electronic devices, wherein the one or more fleet-specific features are shared by a fleet of vehicles, where the fleet of vehicles include an immediate vehicle and a group of vehicles that share one or more common attributes.

System for communicating fleet-specific features of an immediate vehicle to a personal electronic device

A system and method for selectively augmenting information in a vehicle includes receiving a scene information from at least one sensor. identifying a region of interest from the scene information, analyzing the scene information to determine whether the region of interest is occluded, and augmenting the region of interest using a display system when the region of interest is occluded. The region of interest includes an object identified within the scene information that is classified as related to vehicle safety.

System and method for augmented decision making by detecting occlusions

A system for notifying an occupant of a vehicle of unauthorized interaction with his or her possessions includes one or more cameras that collect image data of a surrounding environment of the occupant. The surrounding environment of the occupant is an interior cabin of the vehicle. The system also includes one or more controllers in electronic communication with the one or more cameras, the one or more controllers execute instructions to monitor, by the one or more cameras, the image data of the surrounding environment of the occupant. In response to detecting the unauthorized interaction with the one or more possessions associated with the occupant, the one or more controllers generate a notification informing the occupant of the unauthorized interaction.

System for notifying an occupant of a vehicle of unauthorized access to his or her possessions

An automated driving system includes an object detection system. A neural network image encoder generates image embeddings associated with an image including an object. A neural network text encoder generates concept embeddings associated with each of a plurality of concepts. Each of the plurality of concepts is associated with one of at least two object classes. A confidence score module generates a confidence score for each of the plurality of concepts based on the image embeddings and the concept embeddings associated with the concept. An object class prediction module generates a predicted object class of the object based on an association between a set of concepts of the plurality of concepts having at least two of the highest values of the generated confidence scores and the one of the at least two object classes associated with a majority of the set of concepts.

Systems and methods for classifying detected objects in an image at an automated driving system

A system for determining when additional information is needed by a passenger with color deficient vision within a vehicle includes a plurality of perception sensors within the vehicle adapted to collect data related to a physical identity of a primary object and to communicate the data related to the physical identity of the primary object, via a communication bus, to a data processor within the vehicle, the data processor adapted to determine that the physical identity of the primary object may be unclear to a passenger with color deficient vision based on the data related to the physical identity of the primary object, and that assistance should be provided to the passenger with color deficient vision to aid the passenger with color deficient vision in identifying the primary object.

Identification of confusing objects for color deficient vision

A system for communicating vehicle-specific features to one or more personal electronic devices includes one or more controllers in wireless communication with a centralized computer system including one or more database for storing the vehicle-specific features. The one or more controllers execute instructions to undergo a passive wireless interaction with the one or more personal electronic devices. The passive wireless interaction involves determining the one or more personal electronic devices are located within a predefined proximity around a vehicle without human interaction. In response to undergoing the passive wireless interaction, the one or more controllers transmit one or more vehicle-specific features to the one or more personal electronic devices, where the one or more vehicle-specific features are unique to the vehicle.

Growing community of inventors

Rochester Hills, MI, United States of America

Jacob Alan Bond