Nissan North America, Inc.

United States of America As Represented by the Administrator of Nasa

Florida Institute for Human and Machine Cognition, Inc.

Nio Usa, Inc.

Mauro Della Penna

Maarten Sierhuis

Liam Pedersen

Hans Utz

Mark Allan

Eric Schafer

Maria Bualat

Terrence Fong

Ali Mortazavi

Naoki Kojo

Matthew Johnson

Jeffrey M Bradshaw

Lawrence K Bunch

Michael Vignati

Daisuke Saito

Siddharth Thakur

Atsuhide Kobashi

Armelle Guerin

Richard Lui

John Deniston

David Lees

Dawn Wheeler

Sannidhi Jalukar

Jen Hamon

Shane Hogan

Ben Christel

Stephen Wu

Richard Salloum

Andrea Angquist

Richard Enlow

Various embodiments relate to recording event data to identify and resolve anomalies associated with control of driverless vehicles. Some examples include computing vehicular drive parameters to facilitate driverless transit, monitoring control signals, detecting an event, triggering storage of event data, determining transmission control criteria, and transmitting the event data based on the transmission control criteria. Other examples include receiving event data via a communications network from an autonomous vehicle, identifying a computed vehicular drive parameter, extracting sensor data associated with the event, detecting application of control signals, analyzing the control signals, the sensor data, and the subset of computed vehicular drive parameters to identify a type of event, and generating update executable instructions responsive to the type of event.

Event data recordation to identify and resolve anomalies associated with control of driverless vehicles

Methods and systems for remote support of autonomous operation of vehicles have been disclosed. State indicators are generated by a first state display based on state data from a portion of vehicles assigned to a respective first level control station. A second state display is generated for a second control station and displays state indicators for the state data of the vehicles. A remote support interface including the first state display and image data received from a first vehicle of the vehicles is generated. Instruction data to the first vehicle is transmitted using the remote support interface and based on an indication that the first vehicle needs remote support, the instruction data modifying the autonomous operation of the first vehicle. A workload between the first level control stations is allocated by assigning the vehicles using the state indicators of the second state display.

Autonomous vehicle remote support mapping interface

A remote system for an autonomous vehicle, includes a receiver, a controller, and a display device. The receiver is configured to receive road information. The controller is programmed to receive input related to the road information and create a supervision zone when the road information impacts road drivability. The display device is disposed at a control center area and configured to display a visual indication on a map of the supervision zone.

Remote system for an autonomous vehicle

Methods and systems for providing remote support and negotiating problem situations of autonomous operation of vehicles based on signal states and vehicle information are described. A system comprises a memory and a processor configured to execute instructions stored in the memory to: assign vehicles to support queues based on state data, generate a map display including locations of the vehicles, and generate a state display including the support queues, vehicle manager indicators corresponding to the support queues and state indicators corresponding to the state data.

Tele-operation of autonomous cars to negotiate problem situations

An autonomous vehicle service system having a display device, a receiver, and a controller. The receiver is configured to receive transmitted data from an autonomous vehicle related to status of the autonomous vehicle and information from a third party related to road conditions. The controller is programmed to monitor the transmitted data related to the status of the autonomous vehicle and the road conditions, determine when the autonomous vehicle requires assistance based on the transmitted data, and, when the autonomous vehicle requires assistance, cause information related to the autonomous vehicle to be displayed on the display device.

Autonomous vehicle service system

Methods, apparatuses, systems, and non-transitory computer readable storage media for generating solution data for autonomous vehicles to negotiate problem situations have been disclosed. The disclosed technology generates state data associated with a vehicle using sensor data received from the vehicle and from external objects within a vicinity of the vehicle. The state data includes any of a location of the vehicle, a destination of the vehicle, an operational status of the vehicle, and information associated with a vehicle environment. In response to determining that the state data satisfies a state criterion, a determination of solution profile data that matches the state data is made on the basis of a comparison of the state data to the solution profile data. Solution data is generated using the matching solution profile data to transmit the solution data to the vehicle for execution.

Generation of solution data for autonomous vehicles to negotiate problem situations

Methods and systems for generating a solution path overlay interface to transmit a solution path are described. The disclosed technology includes receiving vehicle data and external data from a vehicle. The vehicle data includes a vehicle location and a vehicle destination, and the external data includes a location and a movement path for each of a plurality of external objects. A solution path is determined between the vehicle location and the vehicle destination, wherein the solution path does not intersect with the plurality of external objects. A solution path overlay interface is generated that includes the vehicle traveling the solution path and at least some of the plurality of external objects. The solution path overlay interface is outputted for display that is configured to receive a command from an operator which results in an updated solution path that is transmitted to the vehicle for execution.

Solution path overlay interfaces for autonomous vehicles

Methods, apparatuses, systems, and non-transitory computer readable storage media for monitoring vehicles including autonomous vehicles are described. The disclosed technology includes a vehicle monitoring system that receives vehicle data and external data associated with a vehicle and a corresponding predetermined area. The vehicle data includes a vehicle state of the vehicle and the external data includes external states of external objects. An issue type of the vehicle is determined based on the vehicle state and at least one of the external states. An indication of the issue type is generated for display on an interface.

Autonomous vehicle monitoring using generated interfaces

Methods and systems for providing remote support and negotiating problem situations of autonomous operation of vehicles based on signal states and vehicle information are described. The disclosed technology receives state data for the vehicles by an apparatus such as a remote vehicle support apparatus. The state data indicates a respective current state for the vehicles. The vehicles are each assigned to respective remote vehicle support queues based on the respective state data. An indication that one of the vehicles is requesting remote support is received by the remote vehicle support apparatus. In response to a determination that a change in the state data indicates that autonomous operation of the one of the vehicles is operating outside of defined parameter values, the remote support is provided to the one of the vehicles through a communications link by transmitting instruction data to modify the autonomous operation of the one of the vehicles.

Methods, apparatuses, systems, and non-transitory computer readable storage media for providing bandwidth constrained image processing are described. The disclosed technology determines a data transfer rate of at least one signal received from a vehicle, the at least one signal including state/status data of the vehicle. In response to determining that the data transfer rate satisfies a data transfer rate criterion, a location of the vehicle and a location of at least one of a plurality of objects that obstruct the at least one signal is determined using the state data and external data associated with the vehicle. The disclosed technology generates optimized state data using the state data and the external data.

Bandwidth constrained image processing for autonomous vehicles

A parking lot mapping system includes a server. The server is remote from a plurality of vehicles, and includes a receiver, an electronic controller and a storage device. The receiver is configured to receive data transmitted from the vehicles, the data including a route traveled and a stopping position of each of the vehicles. The electronic controller is configured to determine that each vehicle has entered a parking lot, and form a map of the parking lot including parking lot routes and parking spots based on accumulation of the data including the stopping position of each vehicle and the route traveled by each of the vehicles after determination that each vehicle vehicles has entered the parking lot. The storage device is configured to store the map of the parking lot.

Parking lot mapping system

Methods, apparatuses, systems, and non-transitory computer readable storage media for generating risk indicators are described. The disclosed technology includes determining a vehicle route of a vehicle and external object routes of external objects. The vehicle route is determined using vehicle route data including a vehicle location and a vehicle destination. The external object routes are determined using external object route data including external object locations and external object destinations. Based on a comparison of the vehicle route data and the external object route data, external object routes that satisfy a proximity criterion are determined. Risk data for the vehicle is generated based on a vehicle state of the vehicle and external object states of the external objects corresponding to the external object routes that satisfy the proximity criterion. In response to determining that the risk data satisfies a risk criterion, at least one risk indicator is generated.

Advanced threat warning for autonomous vehicles

Methods, apparatuses, and non-transitory computer readable storage media for optimizing driving time based on traffic signal states are described. The disclosed technology includes a vehicle that is able to determine, based on route data, a plurality of distances that correspond to paths between a vehicle location and a destination location for the vehicle. The route data can include a map of a predetermined area that includes the vehicle location and the destination location. The vehicle can receive traffic signal data that includes traffic signal states for a corresponding traffic signals on the paths. The vehicle can determine travel times corresponding to a predetermined portion of the paths based on the distances and the traffic signal states. The vehicle can determine an optimized path between the vehicle location and the destination location based on the path that is determined to have the shortest travel time.

Growing community of inventors

San Francisco, CA, United States of America

Mauro Della Penna