Florida Institute for Human and Machine Cognition, Inc.

Nissan North America, Inc.

Matthew Johnson

Jeffrey M Bradshaw

Lawrence K Bunch

Liam Pedersen

Maarten Sierhuis

Mauro Della Penna

Hans Utz

Thomas Eskridge

Michael Vignati

Chua Wei Liang Kenny

Marco Mattos Carvalho

Paul J Feltovich

Methods and systems for remote support of autonomous operation of vehicles have been disclosed. State indicators are generated by a first state display based on state data from a portion of vehicles assigned to a respective first level control station. A second state display is generated for a second control station and displays state indicators for the state data of the vehicles. A remote support interface including the first state display and image data received from a first vehicle of the vehicles is generated. Instruction data to the first vehicle is transmitted using the remote support interface and based on an indication that the first vehicle needs remote support, the instruction data modifying the autonomous operation of the first vehicle. A workload between the first level control stations is allocated by assigning the vehicles using the state indicators of the second state display.

Autonomous vehicle remote support mapping interface

Methods, apparatuses, systems, and non-transitory computer readable storage media for monitoring vehicles including autonomous vehicles are described. The disclosed technology includes a vehicle monitoring system that receives vehicle data and external data associated with a vehicle and a corresponding predetermined area. The vehicle data includes a vehicle state of the vehicle and the external data includes external states of external objects. An issue type of the vehicle is determined based on the vehicle state and at least one of the external states. An indication of the issue type is generated for display on an interface.

Autonomous vehicle monitoring using generated interfaces

A visual display for use by a user for navigation and obstacle avoidance. A typical user employs the invention in operating a vehicle. The display may include a conventional video feed. A visual arch metaphor is also provided. If used in conjunction with a video feed, the arch metaphor preferably extends from the left side of the video, over the top of the video, and on to the right side of the video. The ranging data is then correlated to a predefined color scale. The ranging data is also correlated to a position on the arch metaphor.

User display providing obstacle avoidance

A visual display for use by a user for navigation and obstacle avoidance. A typical user employs the invention in operating a vehicle. The display may include a conventional video feed. A visual arch metaphor is also provided. A ranging device mounted on the vehicle collects ranging data around the vehicle. As an example, the ranging device might collect 180 degrees of ranging data extending from the vehicle's left side, across the vehicle's front, and over to the vehicle's right side. The ranging data is correlated to a predefined color scale. The ranging data is also correlated to a position on the arch metaphor.

A method for visually depicting complex events. Software agents are preferably employed to assist the human operator by collecting, enriching, selecting, aggregating, and analyzing data so that patterns of interest can be visually flagged or otherwise emphasized in the visual display. Events are depicted as an 'event flow' from a source surface to a destination surface. Intervening surfaces may also be defined. The point of origin on the source surface reveals some information about the event flow, as does the point of impact on the destination surface. The event flow may be mapped to one or more intervening surfaces in order to visually depict other characteristics of the event. The entire depiction is rendered in a simulated three-dimensional view. The user is preferably given the ability to pan, zoom, and reorient the vantage point from which the user 'views' the depiction on the computer display.

Growing community of inventors

Pensacola, FL, United States of America

Matthew Johnson