The Boeing Company

Theboeing Company

Tyler Charles Staudinger

Nick Shabdeh Evans

Eric Raymond Muir

Zachary D Jorgensen

Martin Edward Lozano

Marianne E Mata

Mark W Henley

Huafeng Yu

John H Blumer

Kevin Scott Callahan

Nick Shadbeh Evans

Stephen G Dame

Dragos Dorin Margineantu

Daniel S ReMine

Joshua David Kalin

Jai Joon Choi

Matthew A Moser

Brian Kenyon Rupnik

Brian T Whitehead

Gregory R Day

Isaac Chang

Jan Wei Pan

Michelle D Warren

Cullen W Billhartz

Mary Karroqe

Jacob D Delaney

In an example, a method is described. The method includes causing one or more sensors arranged on an aircraft to acquire, over a window of time, first data associated with a first object that is within an environment of the aircraft, where the one or more sensors include one or more of a light detection and ranging (LIDAR) sensor, a radar sensor, or a camera, causing an array of microphones arranged on the aircraft to acquire, over approximately the same window of time as the first data is acquired, first acoustic data associated with the first object, and training a machine learning model by using the first acoustic data as an input value to the machine learning model and by using an azimuth, a range, an elevation, and a type of the first object identified from the first data as ground truth output labels for the machine learning model.

Methods and systems for acoustic machine perception for an aircraft

In one or more embodiments, a disclosed method for cooling a vehicle involves routing, by a bypass line, boil-off of a cryogenic fuel of the vehicle past a cold plate to cool a subsystem mounted proximate the cold plate. The method further involves purging the boil-off in a direction away from the vehicle, after the boil-off passes the cold plate. In one or more embodiments, the cryogenic fuel involves liquid hydrogen (LH2), liquid oxygen (LO2), and/or liquid methane (LCH4). In at least one embodiment, the boil-off of the cryogenic fuel involves gaseous hydrogen (GH2), gaseous oxygen (GO2), and/or gaseous methane (GCH4). The vehicle is a space vehicle, an airborne vehicle, a terrestrial vehicle, or a marine vehicle. In at least one embodiment, the vehicle is a fuel cell vehicle (FCV).

Advanced cooling for cryogenic powered vehicles

Solutions are provided for auto-labeling sensor data for machine learning (ML). An example includes: determining a platform's own position; recording, from a sensor aboard the platform, sensor data comprising a sensor image; receiving position data for at least one intruder object (e.g., a nearby airborne object); based at least on the position data for the intruder object and the platform's position, determining a relative position and a relative velocity of the intruder object; based at least on the relative position and a relative velocity of the intruder object and a field of view of the sensor, determining an expected position of the intruder object in the sensor image; labeling the sensor image, wherein the labeling comprises annotating the sensor image with a region of interest and an object identification; and training an artificial intelligence (AI) model using the labeled sensor image.

Auto-labeling sensor data for machine learning

A method of supporting an aircraft approaching a runway is provided. Example implementations involve receiving a sequence of images, captured by a camera onboard the aircraft. Example implementations involve applying a received image to a machine learning model trained to detect the runway or a runway marking in the image, and to produce a mask that includes a segment of pixels of the image assigned to an object class for the runway or marking. Example implementations may also involve applying the mask to a corner detector to detect interest points on the mask and match the interest points to corresponding points on the runway or the marking that have known runway-framed local coordinates. Example implementations may also involve performing a perspective-n-point estimation to determine a current pose estimate of the aircraft relative to the runway or the marking and outputting the current pose estimate for use during final approach.

Perception-based autonomous landing for aircraft

A method of machine learning model development includes building an autoencoder including an encoder trained to map an input into a latent representation, and a decoder trained to map the latent representation to a reconstruction of the input. The method includes building an artificial neural network classifier including the encoder, and a classification layer partially trained to perform a classification in which a class to which the input belongs is predicted based on the latent representation. Neural network inversion is applied to the classification layer to find inverted latent representations within a decision boundary between classes in which a result of the classification is ambiguous, and inverted inputs are obtained from the inverted latent representations. Each inverted input is labeled with a class that is its ground truth, and thereby producing added training data for the classification, and the classification layer is further trained using the added training data.

Machine learning model development

A fuel cell-based power system comprises a fuel cell configured for continuously receiving a first reactant and a second reactant to produce chemical reactions that generate electrical power, water, and heat, a coolant subsystem configured for circulating a primary coolant in association with the fuel cell, thereby absorbing the generated heat, a tank configured for storing a reactant, and a reactant distribution subsystem configured for conveying the reactant from the tank to an independent system, the fuel cell as the first reactant, and the coolant subsystem as a secondary coolant to remove the absorbed heat from the primary coolant and/or a water accumulator. The secondary coolant may be conveyed to a gas thruster as a gas after the absorbed heat has been removed from the secondary coolant. The reactant may boil off of a cryogenic liquid or vapor or gas transformed from a cryogenic liquid via a heater.

Integration of fuel cell with cryogenic source for cooling and reactant

A method and apparatus for performing automated supervision and inspection of an assembly process. The method is implemented using a computer system. Sensor data is generated at an assembly site using a sensor system positioned relative to the assembly site. A current stage of an assembly process for building an assembly at the assembly site is identified using the sensor data. A context for the current stage is identified. A quality report for the assembly is generated based on the sensor data and the context for the current stage.

Automated supervision and inspection of assembly process

A set of 3D user-designed images is used to create a high volume of realistic scenes or images which can be used for training and testing deep learning machines. The system creates a high volume of scenes having a wide variety of environmental, weather-related factors as well as scenes that take into account camera noise, distortion, angle of view, and the like. A generative modeling process is used to vary objects contained in an image so that more images, each one distinct, can be used to train the deep learning model without the inefficiencies of creating videos of actual, real life scenes. Object label data can be generated for each distinct image. This and other methods can be used to artificially create new scenes that do not have to be recorded in real-life conditions and that do not require costly and time-consuming, manual labelling or tagging of objects.

Generative image synthesis for training deep learning machines

An apparatus for building an artificial-intelligence system is provided. The apparatus accesses images of a real-world scene and generates an image of a simulated object corresponding to a real-world object using a first generative adversarial network (GAN). The apparatus inserts the image of the simulated object into the images of the real-world scene to produce images of the real-world scene including the simulated object. The apparatus applies the images of the real-world scene including the simulated object to a second GAN to remove visual artifacts thereby producing a training set of images of the real-world scene including the simulated object. The apparatus trains an artificial-intelligence algorithm using the training set of images to build the artificial-intelligence system to detect the real-world object in further images of the real-world scene and outputs the artificial-intelligence system for deployment on an autonomous vehicle.

Building an artificial-intelligence system for an autonomous vehicle

Systems, methods, and computer-readable media storing instructions for determining cross-track error of an aircraft on a taxiway are disclosed herein. The disclosed techniques capture electronic images of a portion of the taxiway using cameras or other electronic imaging devices mounted on the aircraft, pre-process the electronic images to generate regularized image data, apply a trained multichannel neural network model to the regularized image data to generate a preliminary estimate of cross-track error relative to the centerline of the taxiway, and post-process the preliminary estimate to generate an estimate of cross-track error of the aircraft. Further embodiments adjust a GPS-based location estimate of the aircraft using the estimate of cross-track error or adjust the heading of the aircraft based upon the estimate of cross-track error.

Aircraft positioning on a taxiway

A method of machine learning model development includes receiving a plurality of images of a scene, and performing an unsupervised image selection. This includes applying the images to a pre-trained model to extract and embed the images with respective feature vectors, and performing a cluster analysis to group the images in a clusters based on correlations among the respective feature vectors. The unsupervised image selection also includes selecting at least some but not all images in each of the clusters, and any images considered outliers that belong to none of the clusters, for a subset of the images that includes fewer than all of the images. And the method includes receiving user input to label or labeling objects depicted in the subset of the images to produce a training set of images, and building a machine learning model for object detection using the training set of images.

Machine learning model development with unsupervised image selection

A computer-implemented method for generating a training image set includes retrieving, from at least one memory device, model data corresponding to a three-dimensional (3-D) model of a target object, and creating a plurality of two-dimensional (2-D) synthetic images from the model data. The 2-D synthetic images include a plurality of views of the 3-D model. The method also includes creating a plurality of semantic segmentation images by identifying a plurality of pixels that define the target object in the 2-D synthetic image, and assigning a semantic segmentation label to the identified pixels of the target object. The method further includes generating linking data associating each of the semantic segmentation labels with a corresponding one of the 2-D synthetic images, and storing the training image set including the 2-D synthetic images, the semantic segmentation labels, and the linking data.

Systems and methods for automatically generating training image sets for an object

Example implementations relate to autonomous airport runway navigation. An example system includes a first sensor and a second sensor coupled to an aircraft at a first location and a second location, respectively, and a computing system configured to receive sensor data from one or both of the first sensor and the second sensor to detect airport markings positioned proximate a runway. The computing system is further configured to identify a centerline of the runway based on the airport markings and receive sensor data from both of the first sensor and the second sensor to determine a lateral displacement that represents a distance between a reference point of the aircraft and the centerline of the runway. The computing system is further configured to control instructions that indicate adjustments for aligning the reference point of the aircraft with the centerline of the runway during subsequent navigation of the aircraft.

System, method, and computer readable medium for autonomous airport runway navigation

A fuel cell-based power system comprises a fuel cell configured for continuously receiving a first reactant and a second reactant to produce chemical reactions that generate electrical power, water, and heat, a coolant subsystem configured for circulating a primary coolant in association with the fuel cell, thereby absorbing the generated heat, a tank configured for storing a reactant, and a reactant distribution subsystem configured for conveying the reactant from the tank to an independent system, the fuel cell as the first reactant, and the coolant subsystem as a secondary coolant to remove the absorbed heat from the primary coolant and/or a water accumulator. The secondary coolant may be conveyed to a gas thruster as a gas after the absorbed heat has been removed from the secondary coolant. The reactant may be boil off of a cryogenic liquid or vapor or gas transformed from a cryogenic liquid via a heater.

Growing community of inventors

Huntsville, AL, United States of America

Tyler Charles Staudinger