Oxford University Innovation Ltd

Bae Systems, Inc.

Isis Innovation Limited

The Chancellor, Masters and Scholars of the University of Oxford

Paul Michael Newman

Winston Samuel Churchill

Alexander Douglas Stewart

William Paul Maddern

Ian Baldwin

Ashley Alan Napier

Christopher James Linegar

Benjamin Charles Davis

Ian Alan Baldwin

Mark Christopher Sheehan

Alastair Robin Harrison

Colin Alexander McManus

Colin McManus

William Maddern

Geoffrey Michael Pascoe

A method of localizing portable apparatus () obtains: a stored experience data set comprising a set of connected nodes; captured location representation data provided by at least one sensor associated with the portable apparatus, and a current pose estimate of the portable apparatus within the environment. The pose estimate is used to select a candidate set of the nodes that contain a potential match for the captured location representation data. The pose estimate is used to obtain a set of paths from path memory data, each said path comprising a set of said nodes previously traversed in the environment under similar environmental/visual conditions. The set of paths is used to refine the candidate set. The captured location representation data and the refined candidate set of nodes is compared in order to identify a current pose of the portable apparatus within the environment.

Localising portable apparatus

A method of localizing portable apparatus () in an environment, the method comprising obtaining captured image data representing an image captured by an imaging device () associated with the portable apparatus, and obtaining mesh data representing a 3-dimensional textured mesh of at least part of the environment. The mesh data is processed to generate a plurality of synthetic images, each synthetic image being associated with a pose within the environment and being a simulation of an image that would be captured by the imaging device from that associated pose. The plurality of synthetic images is analyzed to find a said synthetic image similar to the captured image data, and an indication is provided of a pose of the portable apparatus within the environment based on the associated pose of the similar synthetic image.

A method of localizing a transportable apparatus () within an environment around the apparatus is disclosed. The method comprises obtaining a sequence of images () of the environment and generating one or more sequences of transformed images () from the sequence of images wherein an image from the sequence of images has undergone a transformation to provide a transformed image within the sequence of transformed images. Processing circuitry () is then used to compare one or more images from the sequence of transformed images () and one or more images from at least one of the sequence of images () and a further sequence of transformed images against a representation of the environment. The comparison determines corresponding features within the images () and/or transformed images () and the representation. The transportable apparatus () is then localized according to a position of the one or more corresponding features.

Vehicle localization

Generating a 3D reconstruction of an environment around a monitoring-unit as that monitoring-unit is moved through the environment: a) providing at least a camera and a LIDAR sensor, each being controlled by independent clocks; b) using the camera to determine the trajectory of the monitoring-unit and determining a first time series using the clock of the camera, where the first time series details when the monitoring-unit was at predetermined points of the trajectory; c) recording the returns from the LIDAR sensor and determining a second time series using the clock of the LIDAR sensor, where the second time series details when each scan from the LIDAR was taken; d) using a timer to relate the first and second series in order to match the return from the LIDAR sensor to the point on the trajectory at which the return was received; and e) creating the 3D reconstruction based upon the LIDAR returns using information from the two time series.

Generation of 3D models of an environment

Method and apparatus for localizing transportable apparatus () within an environment () includes obtaining () point cloud data () representing a 3D point cloud with appearance information of at least part of the environment. The method further obtains () first frame data () representing an image produced by a sensor () onboard transportable apparatus at a first time and location within the environment and second frame data () representing an image produced by the sensor at a second time and location within the environment. The method harmonizes () information about the first frame data, the second frame data and an overlapping subset of the point cloud data in order to determine a location within the point cloud data where at least one of the first frame and the second frame was produced, thereby localizing () the transportable apparatus within the environment.

Localising transportable apparatus

A method of localizing transportable apparatus () within an environment includes receiving () data obtained from a first ranging sensor device () that is configured to collect information relating to a 2D representation of an environment () through which the transportable device is moving. Further data is received (), that data being obtained from a second ranging sensor device () of the transportable apparatus configured to collect information relating to at least a surface () over which the transportable apparatus is moving. The ranging sensor device data is used () to estimate linear and rotational velocities of the transportable apparatus and the estimates are used () to generate a new 3D point cloud () of the environment. The method seeks to match () the new 3D point cloud with, or within, existing 3D point cloud () in order to localize the transportable apparatus with respect to the existing point cloud.

Method for localizing a vehicle equipped with two lidar systems

A method and system for determining extrinsic calibration parameters for at least one pair of sensing devices mounted on transportable apparatus obtains () image data () representing images captured by an image generating sensing device () of the pair at a series of poses during motion through an environment () by transportable apparatus (). The method obtains () data () representing a 3D point cloud based on data produced by a 2D LIDAR sensing device () of the pair. The method selects () an image captured by the image at a particular pose. The method generates () a laser reflectance image based on a portion of the point cloud corresponding to the pose. The method computes () a metric measuring alignment between the selected image and the corresponding laser reflectance image and uses () the metric to determine extrinsic calibration parameters for at least one of the sensing devices.

Extrinsic calibration of imaging sensing devices and 2D LIDARs mounted on transportable apparatus

A method of detecting the structural elements within a scene sensed by at least one sensor within a locale, the method comprising: a) capturing data from the sensor, which data provides a first representation of the sensed scene at the current time; b) generating a second representation of the sensed scene where the second representation is generated from a prior model of the locale; and c) comparing the first and second representations with one another to determine which parts of the first representation represent structural elements of the locale.

Method of detecting structural parts of a scene

Navigation data is generated by receiving () a new experience data set () relating to a new experience capture. At least one stored experience data set () relating to at least one previous experience capture is also received (). An experience data set includes a set of nodes, with each node comprising a series of visual image frames taken over a series of time frames. A candidate set of said nodes is obtained () from the stored experience data set that potentially matches a said node in the new experience data set, and then a check () if performed to see if a node in the candidate set matches the node in the new experience data set. If the result of the checking is positive then data relating to the matched nodes is added () to a place data set useable for navigation, the place data set indicating that said nodes in different said experience data sets relate to a same place.

Generating navigation data

A method of localizing a vehicle hosting a sensor comprises capturing data from the sensor providing a sensed scene around the vehicle at a current time, processing the sensed scene to extract a set of features from the sensed scene and determine a position of the vehicle from the sensed scene in relation to a previous position of the sensor, comparing the extracted set of features to one or more stored experiences, wherein each stored experience comprises a plurality of sets of features, where each set of features has been determined from a previously sensed scene, to ascertain whether the sensed scene can be recognized within any of the stored experiences, and if the extracted set of features of the sensed scene are recognized then that stored experience is used to produce an estimate of the position of the vehicle at the current time with respect to the stored experience.

Method of locating a sensor and related apparatus

A method and apparatus are disclosed for localizing a vehicle along a route. The method comprises initially surveying the route to be traversed by the vehicle by creating a plurality of orthographic images of the route, at a plurality of locations along the route and subsequently passing the vehicle along the route while obtaining a current view of the route. The method further comprises resolving a pose of the vehicle relative to a survey trajectory by maximizing a relation between the orthographic images and the current view.

Localising a vehicle along a route

A method of determining extrinsic calibration parameters for at least one sensor () mounted on transportable apparatus (). The method includes receiving () data representing pose history of the transportable apparatus and receiving () sensor data from at least one sensor mounted on transportable apparatus. The method generates () at least one point cloud data using the sensor data received from the at least one sensor, each point in a said point cloud having a point covariance derived from the pose history data. The method then maximizes () a value of a quality function for the at least one point cloud, and uses () the maximized quality function to determine extrinsic calibration parameters for the at least one sensor.

Determining extrinsic calibration parameters for a sensor

A method of localizing transportable apparatus () within an environment including receiving () data obtained from a ranging sensor device () of the transportable apparatus configured to collect information relating to at least a surface () over which the transportable apparatus is moving in an environment, and using () the ranging sensor device data to generate a new 3D point cloud () of the environment. The method further obtains () data representing an existing 3D point cloud () of at least part of the environment, and seeks to match () the new 3D point cloud with, or within, the existing 3D point cloud in order to localize the transportable apparatus with respect to the existing point cloud.

Growing community of inventors

Oxford, United Kingdom

Paul Michael Newman