Nissan North America, Inc.

United States of America As Represented by the Administrator of Nasa

Florida Institute for Human and Machine Cognition, Inc.

Hans Utz

Liam Pedersen

Maarten Sierhuis

Mauro Della Penna

Mark Allan

Eric Schafer

Terrence Fong

Maria Bualat

Jeffrey M Bradshaw

Matthew Johnson

Lawrence K Bunch

Michael Vignati

Methods and systems for remote support of autonomous operation of vehicles have been disclosed. State indicators are generated by a first state display based on state data from a portion of vehicles assigned to a respective first level control station. A second state display is generated for a second control station and displays state indicators for the state data of the vehicles. A remote support interface including the first state display and image data received from a first vehicle of the vehicles is generated. Instruction data to the first vehicle is transmitted using the remote support interface and based on an indication that the first vehicle needs remote support, the instruction data modifying the autonomous operation of the first vehicle. A workload between the first level control stations is allocated by assigning the vehicles using the state indicators of the second state display.

Autonomous vehicle remote support mapping interface

Methods and systems for providing remote support and negotiating problem situations of autonomous operation of vehicles based on signal states and vehicle information are described. A system comprises a memory and a processor configured to execute instructions stored in the memory to: assign vehicles to support queues based on state data, generate a map display including locations of the vehicles, and generate a state display including the support queues, vehicle manager indicators corresponding to the support queues and state indicators corresponding to the state data.

Tele-operation of autonomous cars to negotiate problem situations

Methods, apparatuses, systems, and non-transitory computer readable storage media for generating solution data for autonomous vehicles to negotiate problem situations have been disclosed. The disclosed technology generates state data associated with a vehicle using sensor data received from the vehicle and from external objects within a vicinity of the vehicle. The state data includes any of a location of the vehicle, a destination of the vehicle, an operational status of the vehicle, and information associated with a vehicle environment. In response to determining that the state data satisfies a state criterion, a determination of solution profile data that matches the state data is made on the basis of a comparison of the state data to the solution profile data. Solution data is generated using the matching solution profile data to transmit the solution data to the vehicle for execution.

Generation of solution data for autonomous vehicles to negotiate problem situations

Methods and systems for generating a solution path overlay interface to transmit a solution path are described. The disclosed technology includes receiving vehicle data and external data from a vehicle. The vehicle data includes a vehicle location and a vehicle destination, and the external data includes a location and a movement path for each of a plurality of external objects. A solution path is determined between the vehicle location and the vehicle destination, wherein the solution path does not intersect with the plurality of external objects. A solution path overlay interface is generated that includes the vehicle traveling the solution path and at least some of the plurality of external objects. The solution path overlay interface is outputted for display that is configured to receive a command from an operator which results in an updated solution path that is transmitted to the vehicle for execution.

Solution path overlay interfaces for autonomous vehicles

Methods, apparatuses, systems, and non-transitory computer readable storage media for monitoring vehicles including autonomous vehicles are described. The disclosed technology includes a vehicle monitoring system that receives vehicle data and external data associated with a vehicle and a corresponding predetermined area. The vehicle data includes a vehicle state of the vehicle and the external data includes external states of external objects. An issue type of the vehicle is determined based on the vehicle state and at least one of the external states. An indication of the issue type is generated for display on an interface.

Autonomous vehicle monitoring using generated interfaces

Methods and systems for providing remote support and negotiating problem situations of autonomous operation of vehicles based on signal states and vehicle information are described. The disclosed technology receives state data for the vehicles by an apparatus such as a remote vehicle support apparatus. The state data indicates a respective current state for the vehicles. The vehicles are each assigned to respective remote vehicle support queues based on the respective state data. An indication that one of the vehicles is requesting remote support is received by the remote vehicle support apparatus. In response to a determination that a change in the state data indicates that autonomous operation of the one of the vehicles is operating outside of defined parameter values, the remote support is provided to the one of the vehicles through a communications link by transmitting instruction data to modify the autonomous operation of the one of the vehicles.

Methods, apparatuses, systems, and non-transitory computer readable storage media for providing bandwidth constrained image processing are described. The disclosed technology determines a data transfer rate of at least one signal received from a vehicle, the at least one signal including state/status data of the vehicle. In response to determining that the data transfer rate satisfies a data transfer rate criterion, a location of the vehicle and a location of at least one of a plurality of objects that obstruct the at least one signal is determined using the state data and external data associated with the vehicle. The disclosed technology generates optimized state data using the state data and the external data.

Bandwidth constrained image processing for autonomous vehicles

Methods, apparatuses, systems, and non-transitory computer readable storage media for generating risk indicators are described. The disclosed technology includes determining a vehicle route of a vehicle and external object routes of external objects. The vehicle route is determined using vehicle route data including a vehicle location and a vehicle destination. The external object routes are determined using external object route data including external object locations and external object destinations. Based on a comparison of the vehicle route data and the external object route data, external object routes that satisfy a proximity criterion are determined. Risk data for the vehicle is generated based on a vehicle state of the vehicle and external object states of the external objects corresponding to the external object routes that satisfy the proximity criterion. In response to determining that the risk data satisfies a risk criterion, at least one risk indicator is generated.

Growing community of inventors

Campbell, CA, United States of America

Hans Utz