The Boeing Company

Hrl Laboratories, LLC

Sean Soleyman

Deepak Khosla

Joshua G Fadaie

Yang Chen

Fan Hin Hung

Charles Richard Tullock

Kevin R Martin

Nigel D Stepp

Samuel D Johnson

Michael A Warren

Navid Naderializadeh

Richard Hanes

Ignacio M Soriano

Fan H Hung

Shawn Moffit

Calvin Chung

Chun Kit Chung

A vehicle management system comprising a computer system and an agent. The agent comprises a machine learning model and a rule system. The machine learning model system is trained to receive observations for the vehicle system and select a behavior for the vehicle system in response to receiving the observations. The rule system is configured to select a set of actions to execute the behavior for the vehicle system in response to a selection of the behavior by the machine learning model system.

Behavior management of multiple vehicles

Training adversarial aircraft controllers is provided. The method comprises inputting current observed states of a number of aircraft into a world model encoder, wherein each current state represents a state of a different aircraft, and wherein each current state comprises a missing parameter value. A number of adversarial control actions for the aircraft are input into the world model encoder concurrently with the current observed state, wherein the adversarial control actions are generated by competing neural network controllers. The world model encoder generates a learned observation from the current observed states and adversarial control actions, wherein the learned observation represents the missing parameter value from the current observed states. The learned observation and current observed states are input into the competing neural network controllers, wherein each current observed state is fed into a respective controller. The competing neural network controllers then generate next adversarial control actions.

Predictive modeling of aircraft dynamics

Described is a learning system for multi-agent applications. In operation, the system initializes a plurality of learning agents. The learning agents include both tactical agents and strategic agents. The strategic agents take an observation from an environment and select one or more of the tactical agents to produce an action that is used to control a platform's actuators or simulated movements in the environment to complete a task. Alternatively, the tactical agents produce the action corresponding to a learned low-level behavior to control the platform's actuators or simulated movements in the environment to complete the task.

Robust, scalable and generalizable machine learning paradigm for multi-agent applications

Described is a system for performing a simulated vehicle control task based on adversarial decision analysis. Empirical game theoretic analyses are performed between an evolving and an adversary population of neural network strategies. Each empirical game theoretic analysis includes using a neuroevolution procedure to perform a fitness-based selection of a strategy in the evolving population that out-performs the adversary population. Using an empirical game theory procedure, the neuroevolution procedure is iteratively run and the selected strategy is added to the adversary population with each iteration, resulting in monotonic strategy improvement with each iteration. Following the empirical game theoretic analyses, a final strategy is selected for the evolving population and the adversary population using a tournament selection procedure. The final strategy is used to train a neural network which is used to perform a simulated vehicle control task.

Empirical game theoretic system and method for adversarial decision analysis

Described is a system for controlling multiple autonomous platforms. A training process is performed to produce a trained learning agent in a simulation environment. In each episode, each controlled platform is assigned to one target platform that produces an observation. A learning agent processes the observation using a deep learning network and produces an action corresponding to each controlled platform until an action has been produced for each controlled platform. A reward value is obtained corresponding to the episode. The trained learning agent is executed to control each autonomous platform, where the trained agent receives one or more observations from one or more platform sensors and produces an action based on the one or more observations. The action is then used to control one or more platform actuators.

Multi-agent planning and autonomy

A method includes receiving, by machine-learning logic, observations indicative of a states associated with a first and second group of vehicles arranged within an engagement zone during a first interval of an engagement between the first and the second group of vehicles. The machine-learning logic determines actions based on the observations that, when taken simultaneously by the first group of vehicles during the first interval, are predicted by the machine-learning logic to result in removal of one or more vehicles of the second group of vehicles from the engagement zone during the engagement. The machine-learning logic is trained using a reinforcement learning technique and on simulated engagements between the first and second group of vehicles to determine sequences of actions that are predicted to result in one or more vehicles of the second group being removed from the engagement zone. The machine-learning logic communicates the plurality of actions to the first group of vehicles.

Method and system for vehicle engagement control

A method includes obtaining multiple sets of trajectory data, each descriptive of trajectories of two or more objects (e.g., first and second objects). The method also includes generating transformed trajectory data based on the trajectory data. Each set of transformed trajectory data is descriptive of the trajectories of the two or more objects in a normalized reference frame in which a movement path of the first object is constrained. The method further includes generating feature data, performing a clustering operation based on the feature data to generate a set of trajectory clusters, and generating training data based on the set of trajectory clusters. The method further includes using the training data to train a machine learning classifier to classify particular trajectory patterns.

Trajectory classification and response

Apparatus and methods for training a machine learning algorithm (MLA) to control a first aircraft in an environment that comprises the first aircraft and a second aircraft are described. Training of the MLA can include: the MLA determining a first-aircraft action for the first aircraft to take within the environment; sending the first-aircraft action from the MLA; after sending the first-aircraft action, receiving an observation of the environment and a reward signal at the MLA, the observation including information about the environment after the first aircraft has taken the first-aircraft action and the second aircraft has taken a second-aircraft action, the reward signal indicating a score of performance of the first-aircraft action based on dynamic and kinematic properties of the second aircraft; and updating the MLA based on the observation of the environment and the reward signal.

Growing community of inventors

Calabasas, CA, United States of America

Sean Soleyman