Foresight Ai Inc.

Nantstudios, LLC

Matheen Siddiqui

Chang Yuan

Shili Xu

Cheng-Yi Lin

A computer system may identify a first position of a physical camera corresponding to a first time period. The computer system may render a first virtual scene for the first time period. The system may project the first scene onto a display surface to determine a first rendered image for the first time period. The computer system may receive a first camera image of the display surface from the camera during the first time period. The system may determine a first corrected position of the camera by comparing the first rendered image to the first camera image. The system may predict a second position of the camera corresponding to a second time period. The computer system may render a second virtual scene for the second time period. The system may project the second virtual scene onto the display surface to determine a second rendered image for the second time period.

Camera tracking via dynamic perspectives

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a driving scenario machine learning network and providing a simulated driving environment. One of the operations is performed by receiving video data that includes multiple video frames depicting an aerial view of vehicles moving about an area. The video data is processed and driving scenario data is generated which includes information about the dynamic objects identified in the video. A machine learning network is trained using the generated driving scenario data. A 3-dimensional simulated environment is provided which is configured to allow an autonomous vehicle to interact with one or more of the dynamic objects.

Driving scenario machine learning network and driving environment simulation

Methods that identify local motions in point cloud data that are generated by one or more rounds of LIDAR scans are disclosed. The point cloud data describes an environment with points with each point having a scanned time and scanned coordinates. From the point cloud data, a subset of points is selected. A surface is reconstructed at a common reference time using the subset of points. The reconstructed surface includes points that are moved from the scanned coordinates of the points in the subset. The moved points are derived from a projected movement under a projected motion parameter in duration between the scanned time and the common reference time. The surface quality of the reconstructed surface is determined. If the surface has a high quality, the projected motion parameter is determined to be the final motion parameter that is used as an indication whether an object is moving.

Identification of local motions in point cloud data

The present invention generally relates to generating a three-dimensional representation of a physical environment, which includes dynamic scenarios. An exemplary device comprises one or more processors; a memory; and one or more programs that includes instructions for: obtaining a set of three-dimensional data of the physical environment, wherein the three-dimensional data is associated with a first sensing device; based on the set of three-dimensional data, generating a three-dimensional representation of the physical environment; obtaining a set of two-dimensional data of the physical environment, wherein the set of two-dimensional data is associated with a second sensing device and wherein the set of two-dimensional data comprises information of a dynamic object in the physical environment; generating an alignment between the three-dimensional representation of the physical environment and the set of two-dimensional data; and based on the alignment, obtaining a set of three-dimensional information associated with the dynamic object.

Intelligent capturing of a dynamic physical environment

A method of using a drone that is equipped with a camera and an inertial measurement unit (IMU) to survey an environment to reconstruct a 3D map is described. A key frame location is first identified. A first image of the environment is captured by the camera from the key frame location. The drone is then moved away from the key frame location to another location. A second image of the environment is captured from the other location. The drone then returns to the key frame location. The drone may perform additional rounds of scans and returns to the key frame location between each round. By constantly requiring the drone to return to the key frame location, the precise location of the drone may be determined by the acceleration data of the IMU because the location information may be recalibrated each time at the key frame location.

Growing community of inventors

Long Beach, CA, United States of America

Matheen Siddiqui