Magik Eye Inc.

Olympus Optical Company, Ltd.

B-core Inc.

Akiteru Kimura

Takashi Inoue

Miki Matsuzaki

Tatsuya Suzuki

Hisashi Goto

Akira Watanabe

Atsushi Maruyama

Azuma Miyazawa

Moriya Katagiri

Kazuyasu Motoyama

Hiroshi Takase

Akira Inoue

Hiroaki Miyazaki

Makoto Kondo

Tokuo Shimizu

Shinichi Harada

Masaaki Daigaku

Goro Kitajima

Hidenori Sakurai

Takeo Miyazawa

Yoshio Tomizawa

Hiroo Hosono

Masatoshi Inoue

Masayuki Matsuda

Kunio Kando

Atsuhiko Tanaka

Hajime Osada

Tomaso Poggio

Jan Heller

An example method includes acquiring a three-dimensional coordinate system associated with a three-dimensional camera and a first reference point associated with the three-dimensional camera, acquiring a two-dimensional coordinate system associated with a two-dimensional camera and a second reference point associated with the two-dimensional camera, aligning the three-dimensional coordinate system with the two-dimensional coordinate system, based on a fixed and known positional relationship between the first reference point and the second reference point, to obtain a fixed positional relationship between the three-dimensional coordinate system and the two-dimensional coordinate system, and transferring the three-dimensional coordinate system to a datum point of the two-dimensional coordinate system, using the fixed positional relationship between the three-dimensional coordinate system and the two-dimensional coordinate system.

Transferring the coordinate system of a three-dimensional camera to the incident point of a two-dimensional camera

An example method includes causing a light projecting system of a distance sensor to project a three-dimensional pattern of light onto an object, wherein the three-dimensional pattern of light comprises a plurality of points of light that collectively forms the pattern, causing a light receiving system of the distance sensor to acquire an image of the three-dimensional pattern of light projected onto the object, causing the light receiving system to acquire a two-dimensional image of the object, detecting a feature point in the two-dimensional image of the object, identifying an interpolation area for the feature point, and computing three-dimensional coordinates for the feature point by interpolating using three-dimensional coordinates of two points of the plurality of points that are within the interpolation area.

Associating three-dimensional coordinates with two-dimensional feature points

An example apparatus includes a light projecting system, a light receiving system, and a controller. The light projecting system projects beams of light of a wavelength that is invisible. The beams form a pattern on a surface when the beams are incident upon the surface. The light receiving system acquires an image of the pattern on the surface. The controller calculates a distance to the surface based on the image. The light receiving system includes a lens, an imaging sensor, and a bandpass filter. The imaging sensor includes a first subset of photodetectors sensitive to wavelengths of light that are visible and a second subset of photodetectors sensitive to the wavelength of light that is invisible. The bandpass filter includes a first passband whose range corresponds to the wavelengths of light that are visible and a second passband whose range corresponds to the wavelength of light that is invisible.

Three-dimensional sensor including bandpass filter having multiple passbands

In one example, a distance sensor includes a camera to capture images of a field of view, a plurality of light sources arranged around a lens of the camera, wherein each light source of the plurality of light sources is configured to project a plurality of beams of light into the field of view, and wherein the plurality of beams of light creates a pattern of projection artifacts in the field of view that is visible to a detector of the camera, a baffle attached to a first light source of the plurality of light sources, wherein the baffle is positioned to limit a fan angle of a plurality of beams of light that is projected by the first light source, and a processing system to calculate a distance from the distance sensor to an object in the field of view, based on an analysis of the images.

Baffles for three-dimensional sensors having spherical fields of view

An example method includes causing a light projecting system of a distance sensor to project a pattern of light onto an object. The pattern of light includes a plurality projection artifacts arranged in a grid. Rows and columns of the grid are arranged in a staggered manner. Spacing between the rows and spacing between the columns is set so that a pattern density parameter of the pattern of light increases with a length of a trajectory of the projection artifacts. The pattern density parameter is defined as a ratio between a maximum diameter of the projection artifacts and the length of the trajectory. The method further includes causing a light receiving system of the distance sensor to acquire an image of the pattern of light projected onto the object, and computing a distance from the object to the distance sensor based on locations of the projection artifacts in the image.

Distance measurement using high density projection patterns

In one example, a method includes instructing, by a processing system of a distance sensor, a pattern projector of the distance sensor to project a pattern of light onto an object, wherein the pattern comprises a plurality projection artifacts, detecting, by the processing system, a location of a first projection artifact of the plurality of artifacts in an image of the object that includes the pattern of light, wherein the detecting comprises identifying an area of peak light intensity within a window corresponding to a trajectory of the first projection artifact, wherein the trajectory represents a range of potential movement of the first projection artifact on an imaging sensor of the distance sensor, and calculating, by the processing system, a distance from the distance sensor to the object based in part on the location of the first projection artifact.

An example method includes setting an exposure time of a camera of a distance sensor to a first value, instructing the camera to acquire a first image of an object in a field of view of the camera, where the first image is acquired while the exposure time is set to the first value, instructing a pattern projector of the distance sensor to project a pattern of light onto the object, setting the exposure time of the camera to a second value that is different than the first value, and instructing the camera to acquire a second image of the object, where the second image includes the pattern of light, and where the second image is acquired while the exposure time is set to the second value.

Adjusting camera exposure for three-dimensional depth sensing and two-dimensional imaging

An example method includes causing a distance sensor to project a pattern of light onto an object, causing the distance sensor to acquire an image of the pattern of light on the object, detecting an image position of a first point of light of the pattern on an imaging sensor, calculating a first estimated distance and a second estimated distance between the distance sensor and the object, based on the image position, causing the distance sensor to emit a pulse of light toward the object, causing the distance sensor to acquire an image of a reflected portion of the pulse of light, calculating a third estimated distance between the distance sensor and the object, based on a time of flight of the pulse, and calculating a fourth estimated distance between the distance sensor and the object, based on the first estimated distance, second estimated distance, and third estimated distance.

Enhancing triangulation-based three-dimensional distance measurements with time of flight information

A projection pattern is projected onto a surface of an object from a projection point of a distance by projecting a plurality of beams of light from the projection point. The plurality of beams of light creates a plurality of projection artifacts that is arranged in a grid on the surface of the object. A center projection artifact lies at an intersection of a grid longitude line and a grid latitude line. A projection of the plurality of beams is adjusted so that the longitude line and/or the latitude line is rotated by a predetermined amount from an original position to a new position, resulting in an adjusted projection pattern on the surface of the object. An image of the object, including at least a portion of the adjusted projection pattern, is captured. A distance from the distance sensor to the object is calculated using information from the image.

Distance measurement using a longitudinal grid pattern

In one example, a method includes instructing a pattern projector of a distance sensor to project a pattern of light onto the object, wherein the pattern comprise a plurality of parallel rows of projection artifacts, and wherein a spatial density of the projection artifacts in a first row of the plurality of parallel rows is different from a spatial density of the projection artifacts in a second row of the plurality of parallel rows, instructing a camera of the distance sensor to acquire an image of the object, where the image includes the pattern of light, and calculating a distance from the distance sensor to the object based on an analysis of the image.

Distance measurement using projection patterns of varying densities

A 3D distance sensor projects a plurality of points of light onto a target. A 2D camera having a fixed position relative to the 3D distance sensor acquires a first image of the points on the target. A light receiving system of the 3D distance sensor acquires a second image of the points on the target. A set of 2D coordinates of a first point is associated with a distance coordinate of the first point to form a single set of coordinates for the first point. The set of 2D coordinates is obtained from the first image, and the distance coordinate is obtained from the second image. A relational expression is derived from the single set of coordinates and describes a relationship between a distance of the light receiving system to the target and a position of the first point on an image sensor of the 2D camera.

Mapping three-dimensional depth map data onto two-dimensional images

In one embodiment, a method for calculating a distance to an object includes simultaneously activating a first projection point and a second projection point of a distance sensor to collectively project a reference pattern into a field of view, activating a third projection point of the distance sensor to project a measurement pattern into the field of view, capturing an image of the field of view, wherein the object, the reference pattern, and the measurement pattern are visible in the image, calculating a distance from the distance sensor to the object based on an appearance of the measurement pattern in the image, detecting a movement of a lens of the distance sensor based on an appearance of the reference pattern in the image, and adjusting the distance as calculated based on the movement as detected.

Distance sensor including adjustable focus imaging sensor

In an example, an apparatus includes a first sensor, a second sensor, and an integrated management system. The first sensor is for capturing a first set images of a calibration target that is placed in a monitored site, wherein the first sensor has a first position in the monitored site, and wherein a physical appearance of the calibration target varies when viewed from different positions within the monitored site. The second sensor is for capturing a second set of images of the calibration target, wherein the second sensor has a second position in the monitored site that is different from the first position. The integrated management system is for determining a positional relationship of the first sensor and the second sensor based on the first set of images, the second set of images, and knowledge of the physical appearance of the calibration target.

Calibrating a sensor system including multiple movable sensors

A projection pattern is projected onto an object from a projection point of a distance sensor. The projection pattern is created by a plurality of beams of light projected from the projection point. The plurality of beams creates a plurality of projection artifacts that is arranged on the surface of the object. A layout of the plurality of projection artifacts depends on a positional relationship between the projection point and an image capturing device of the distance sensor. At least one parameter that defines the positional relationship between the projection point and an image capturing device of the distance sensor is optimized, prior to projecting the pattern, to minimize overlap of trajectories associated with the projection artifacts. An image of the object, including at least a portion of the adjusted projection pattern, is captured. A distance from the distance sensor to the object is calculated using information from the image.

Growing community of inventors

Hachioji, Japan

Akiteru Kimura