Hrl Laboratories, LLC

The Boeing Company

Raytheon Company

Xlabs Holdings, LLC

Hrl Labortories, LLC

Theboeing Company

Deepak Khosla

Yang Chen

David J Huber

Kyungnam Kim

Lei Zhang

Ryan M Uhlenbrock

Shinko Yuanhsien Cheng

Changsoo S Jeong

Alexander L Honda

Kevin R Martin

Sean Soleyman

Fredy Monterroza

Fan Hin Hung

Leon Nguyen

Jiejun Xu

Rajan Bhattacharyya

Yongqiang Cao

Paul Alex Dow

Yuri Owechko

Narayan Srinivasa

Swarup S Medasani

Anthony Wayne Baker

Brandon M Courter

Joshua G Fadaie

Haden Harrison Smith

Darrel J VanBuer

James Guillochon

Michael J Daily

Suhas E Chelian

Shane Edward Arthur

Matthew S Keegan

Zhichao Chen

David L Ii

William K Leach

Charles Richard Tullock

Tsai-Ching Lu

David L Allen

Qin Jiang

Nick Shadbeh Evans

Aleksey Nogin

Nigel D Stepp

Justin Cleve Hatcher

Terrell Nathan Mundhenk

Ronald Tadao Azuma

Luke Charles Ingram

Samuel D Johnson

Eric Huang

Howard C Choe

Michael A Warren

Paul S Idell

Navid Naderializadeh

Yifan Yang

James L Clayton

Thomas S Nichols

Michelle D Warren

Jeremy Daniel Howe

Ignacio M Soriano

David A Fuciarelli

Scott Rad

Richard Hanes

Chris A Cantrell

Taraneh Sadjadpour

Aaron Feldman

Jacob Moorman

Benjamin Nuernberger

William Kentarou Kaneko Leach

Anders Eirik Heussy

Fan H Hung

Shawn Moffit

Calvin Chung

Penn Tasinga

Christopher Kanan

Alex Dow

Huapeng Su

Chun Kit Chung

A method of monitoring an assembly process is disclosed herein. The method includes obtaining an event model for each of a plurality of objects in the assembly process with the event model for each of the plurality of objects including a predetermined time frame for a change in presence to occur. The method includes collecting an image sequence of the assembly process for monitoring and identifying if a change in presence for each of the plurality of objects occurred with a detector model. The method further includes reviewing the event model for each of the plurality of objects to determine if the predetermined time frame lapsed without the change in presence of a corresponding one of the plurality of objects being identified and issuing an alert if the predetermined time frame lapsed without the presence of a corresponding one of the plurality of objects being identified.

Method and system for process monitoring

A vehicle management system comprising a computer system and an agent. The agent comprises a machine learning model and a rule system. The machine learning model system is trained to receive observations for the vehicle system and select a behavior for the vehicle system in response to receiving the observations. The rule system is configured to select a set of actions to execute the behavior for the vehicle system in response to a selection of the behavior by the machine learning model system.

Behavior management of multiple vehicles

Training adversarial aircraft controllers is provided. The method comprises inputting current observed states of a number of aircraft into a world model encoder, wherein each current state represents a state of a different aircraft, and wherein each current state comprises a missing parameter value. A number of adversarial control actions for the aircraft are input into the world model encoder concurrently with the current observed state, wherein the adversarial control actions are generated by competing neural network controllers. The world model encoder generates a learned observation from the current observed states and adversarial control actions, wherein the learned observation represents the missing parameter value from the current observed states. The learned observation and current observed states are input into the competing neural network controllers, wherein each current observed state is fed into a respective controller. The competing neural network controllers then generate next adversarial control actions.

Predictive modeling of aircraft dynamics

Described is a learning system for multi-agent applications. In operation, the system initializes a plurality of learning agents. The learning agents include both tactical agents and strategic agents. The strategic agents take an observation from an environment and select one or more of the tactical agents to produce an action that is used to control a platform's actuators or simulated movements in the environment to complete a task. Alternatively, the tactical agents produce the action corresponding to a learned low-level behavior to control the platform's actuators or simulated movements in the environment to complete the task.

Robust, scalable and generalizable machine learning paradigm for multi-agent applications

A system for refining a six degrees of freedom pose estimate of a target object based on a one-dimensional measurement includes a camera and a range-sensing device. The range-sensing device is configured to determine an actual distance measured between the range-sensing device and an actual point of intersection. The range-sensing device projects a line-of-sight that intersects with the target object at the actual point of intersection. The system also includes one or more processors in electronic communication with the camera and the range-sensing device and a memory coupled to the processors. The memory stores data into one or more databases and program code that, when executed by the processors, causes the system to predict the six degrees of freedom pose estimate of the target object. The system also determines a revised six degrees of freedom pose estimate of the target object based on at least an absolute error.

System for refining a six degrees of freedom pose estimate of a target object

Described is a system for performing a simulated vehicle control task based on adversarial decision analysis. Empirical game theoretic analyses are performed between an evolving and an adversary population of neural network strategies. Each empirical game theoretic analysis includes using a neuroevolution procedure to perform a fitness-based selection of a strategy in the evolving population that out-performs the adversary population. Using an empirical game theory procedure, the neuroevolution procedure is iteratively run and the selected strategy is added to the adversary population with each iteration, resulting in monotonic strategy improvement with each iteration. Following the empirical game theoretic analyses, a final strategy is selected for the evolving population and the adversary population using a tournament selection procedure. The final strategy is used to train a neural network which is used to perform a simulated vehicle control task.

Empirical game theoretic system and method for adversarial decision analysis

Aspects of the disclosure provide fuel receptacle position estimation for aerial refueling (derived from aircraft position estimation). A video stream comprising a plurality of video frames each showing an aircraft to be refueled, is received from a single camera. An initial position estimate is determined for the aircraft for the plurality of video frames, generating an estimated flight history for the aircraft. The estimated flight history for the aircraft is used to determine a temporally consistent refined position estimate, based on known aircraft flight path trajectories in an aerial refueling setting. The position of a fuel receptacle on the aircraft is determined, based on the refined position estimate for the aircraft, and an aerial refueling boom may be controlled to engage the fuel receptacle. Examples may use a deep learning neural network (NN) or optimization (e.g., bundle adjustment) to determine the refined position estimate from the estimated flight history.

Temporally consistent position estimation refinement for aerial refueling

Aspects of the disclosure provide fuel receptacle and boom tip position and pose estimation for aerial refueling. A video frame is received and within the video frame, aircraft keypoints for an aircraft to be refueled are determined. Based on at least the aircraft keypoints, a position and pose of a fuel receptacle on the aircraft is determined. Within the video frame, a boom tip keypoint for a boom tip of an aerial refueling boom is also determined. Based on at least the boom tip keypoint, a position and pose of the boom tip is determined. Based on at least the position and pose of the fuel receptacle and the position and pose of the boom tip, the aerial refueling boom is controlled to engage the fuel receptacle. Some examples overlay projections of an aircraft model on displayed video for a human observer.

Fuel receptacle and boom tip position and pose estimation for aerial refueling

Apparatuses, systems, and methods dynamically model intrinsic parameters of a camera. Methods include: collecting, using a camera having a focus motor, calibration data at a series of discrete focus motor positions; generating, from the calibration data, a set of constant point intrinsic parameters; determining, from the set of constant point intrinsic parameters, a subset of intrinsic parameters to model dynamically; performing, for each intrinsic parameter of the subset of intrinsic parameters, a fit of the point intrinsic parameter values against focus motor positions; generating a model of the intrinsic parameters for the camera based, at least in part, on the fit of the point intrinsic parameter values against the focus motor positions; and determining a position of a fiducial marker within a field of view of the camera based, at least in part, on the model of the intrinsic parameters for the camera.

Method and apparatus for modeling dynamic intrinsic parameters of a camera

Aspects of the disclosure provide fuel receptacle position/pose estimation for aerial refueling (derived from aircraft position and pose estimation). A video frame, showing an aircraft to be refueled, is received from a single camera. An initial position/pose estimate is determined for the aircraft, which is used to generating an initial rendering of an aircraft model. The video frame and the initial rendering are used to determining refinement parameters (e.g., a translation refinement and a rotational refinement) for the initial position/pose estimate, providing a refined position/pose estimate for the aircraft. The position/pose of a fuel receptacle on the aircraft is determined, based on the refined position/pose estimate for the aircraft, and an aerial refueling boom may be controlled to engage the fuel receptacle. Examples extract features from the aircraft in the video frame and the aircraft model rendering, and use a deep learning neural network (NN) to determine the refinement parameters.

Pose estimation refinement for aerial refueling

Apparatuses and methods train a model and then use the trained model to determine a global three dimensional (3D) position and orientation of a fiduciary marker. In the context of an apparatus for training a model, a wider field-of-view sensor is configured to acquire a static image of a space in which the fiducial marker is disposed and a narrower field-of-view sensor is configured to acquire a plurality of images of at least a portion of the fiducial marker. The apparatus also includes a pan-tilt unit configured to controllably alter pan and tilt angles of the narrower field-of-view sensor during image acquisition. The apparatus further includes a control system configured to determine a transformation of position and orientation information determined from the images acquired by the narrower field-of-view sensor to a coordinate system for the space for which the static image is acquired by the wider field-of-view sensor.

Method and apparatus for determining a three-dimensional position and pose of a fiducial marker

A method, apparatus and computer program product are provided to generate a model of one or more objects relative to a vehicle. In the context of a method, radar information is received in the form of in-phase quadrature (IQ) data and the IQ data is converted to one or more first range-doppler maps. The method further includes evaluating the one or more first range-doppler maps with a machine learning model to generate the model that captures the detection of the one or more objects relative to the vehicle. A corresponding apparatus and computer program product are also provided.

Radar-based method and apparatus for generating a model of an object relative to a vehicle

Described is a system for controlling multiple autonomous platforms. A training process is performed to produce a trained learning agent in a simulation environment. In each episode, each controlled platform is assigned to one target platform that produces an observation. A learning agent processes the observation using a deep learning network and produces an action corresponding to each controlled platform until an action has been produced for each controlled platform. A reward value is obtained corresponding to the episode. The trained learning agent is executed to control each autonomous platform, where the trained agent receives one or more observations from one or more platform sensors and produces an action based on the one or more observations. The action is then used to control one or more platform actuators.

Multi-agent planning and autonomy

Apparatuses and methods determine the three-dimensional position and orientation of a fiducial marker and tracking the three-dimensional position and orientation across different fields-of-view. Methods include: capturing an image of a first space in which the fiducial marker is disposed with a first sensor having a first field-of-view; determining the three-dimensional location and orientation of the fiducial marker within the first space based on the image of the first space in which the fiducial marker is disposed; capturing an image of a second space in which the fiducial marker is disposed with a second sensor having a second field-of-view; calculating pan and tilt information for the second sensor to move the second field-of-view of the second sensor to acquire an image of the fiducial marker; and determining the three-dimensional location and orientation of the fiducial marker within the second space based on the image of the second space.

Method and apparatus for hand-off and tracking for pose estimation of a fiducial marker

Described is a system for onboard, real-time, activity detection and classification. During operation, the system detects one or more objects in a scene using a mobile platform and tracks each of the objects as the objects move in the scene to generate tracks for each object. The tracks are transformed using a fuzzy-logic based mapping to semantics that define group activities of the one or more objects in the scene. Finally, a state machine is used to determine whether the defined group activities are normal or abnormal phases of a predetermined group operation.

System and method for onboard, real-time mission analytics with anomaly detection and explanation

Growing community of inventors

Camarillo, CA, United States of America

Deepak Khosla