Ams Sensors Singapore Pte. Ltd.

Heptagon Micro Optics Pte. Ltd.

Deere & Company

Florin Cutu

Hartmut Rudmann

Chi Zhang

Alireza Yasan

Markus Rossi

Jens Geiger

Peter Roentgen

Miguel Bruno Vaello Paños

Mario Cesana

Jukka Alasirnio

Moshe Doron

Bassam Hallal

Ohad Meitav

Tobias Senn

Daniel Pérez Calero

Michael Kiy

Xin Liu

Christian Tang-Jespersen

James Patrick Long

James Eilertsen

A drone is deployed from a vehicle, e.g., an autonomous or semi-autonomous vehicle, to assist in vehicle control, e.g. in situations in which the vehicle's embedded sensors may not provide sufficient information to perform a desired operation safely, e.g. backing up, parking in a tight environment, traversing a very narrow road, navigating a sharp corner, or bypassing an obstruction, etc. The deployed drone includes sensors, e.g. cameras, radars, LIDARs, etc, which capture sensor data from a position offset from the vehicle. Captured sensor data is communicated from the drone to a vehicle control system in the vehicle and/or to a remote control system, e.g., including an operator who can make decisions. Based on the captured sensor data, which supplements sensor data collected by the vehicle's embedded sensors, vehicle movement is controlled.

Methods and apparatus for vehicle control

Image sensor modules include primary high-resolution imagers and secondary imagers. For example, an image sensor module may include a semiconductor chip including photosensitive regions defining, respectively, a primary camera and a secondary camera. The image sensor module may include an optical assembly that does not substantially obstruct the field-of-view of the secondary camera. Some modules include multiple secondary cameras that have a field-of-view at least as large as the field-of-view of the primary camera. Various features are described to facilitate acquisition of signals that can be used to calculate depth information.

Image sensor modules including primary high-resolution imagers and secondary imagers

Presenting a merged, fused three-dimensional point cloud includes acquiring multiple sets of images of a scene from different vantage points, each set of images including respective stereo matched images and a color image. For each respective set of images, a disparity map based on the plurality of stereo images is obtained, data from the color image is fused onto the disparity map so as to generate a fused disparity map, and a three-dimensional fused point cloud is created from the fused disparity map. The respective three-dimensional fused point clouds is merged together so as to obtain a merged, fused three-dimensional point cloud. The techniques can be advantageous even under the constraints of sparseness and low-depth resolution, and are suitable, in some cases, for real-time or near real-time applications in which computing time needs to be reduced.

Generating a merged, fused three-dimensional point cloud based on captured images of a scene

This disclosure describes systems and techniques for generating a disparity map having reduced over-smoothing. To achieve the reduction in over-smoothing that might otherwise occur, a third image is captured in addition to stereo reference and search images. Information from the third image, which may be of the same or different type as the stereo images (e.g., RGB, grey-scale, infra-red (IR)) and which may have the same or different resolution as the stereo images, can help better define the object edges.

Generating a disparity map having reduced over-smoothing

The present disclosure describes imaging techniques and devices having improved autofocus capabilities. The imaging techniques can include actively illuminating a scene and determining distances over the entire scene and so that a respective distance to each object or point in the scene can be determined. Thus, distances to all objects in a scene (within a particular range) at any given instant can be stored. A preview of the image can be displayed so as to allow a user to select a region of the scene of interest. In response to the user's selection, the imager's optical assembly can be adjusted automatically, for example, to a position that corresponds to optimal image capture of objects at the particular distance of the selected region of the scene.

Imaging devices having autofocus control in response to the user touching the display screen

Techniques are described for generating a distance map (e.g., a map of disparity, depth or other distance values) for image elements (e.g., pixels) of an image capture device. The distance map is generated based on an initial distance map (obtained, e.g., using a block or code matching algorithm) and a segmentation map (obtained using a segmentation algorithm). In some instances, the resulting distance map can be less sparse than the initial distance map, can contain more accurate distance values, and can be sufficiently fast for real-time or near real-time applications. The resulting distance map can be converted, for example, to a color-coded distance map of a scene that is presented on a display device.

Generating a distance map based on captured images of a scene

The present disclosure describes projecting a structured light pattern onto a surface and detecting and responding to interactions with the same. A method includes-acquiring an image based on light reflected from a vicinity of the projection surface, and identifying regions of the acquired image that correspond to a feature that is within a specified distance of the projection surface by identifying regions of the acquired image for which intensity data differs relative to other regions of the acquired image and which fit a specified homographic relationship with respect to corresponding regions of a reference image.

Projecting a structured light pattern onto a surface and detecting and responding to interactions with the same

The present disclosure describes projecting a structured light pattern projected onto a surface and detecting and responding to interactions with the same. The techniques described here can, in some cases, facilitate recognizing that an object such as a user's hand is adjacent the plane of a projection surface and can distinguish the object from the projection surface itself. Movement of the object then can be interpreted, for example, as a specified type of gesture that can trigger a specified type of operation to occur.

Imaging devices having autofocus control

Generating a super-resolved reconstructed image includes acquiring a multitude of monochromatic images of a scene and extracting high-frequency band luma components from the acquired images. A high-resolution luma image is generated using the high-frequency components and motion data for the acquired images. The high-resolution luma image is combined with an up-sampled color image, generated from the acquired images, to generate a super-resolved reconstructed color image of the scene.

Super-resolution image reconstruction using high-frequency band extraction

The disclosure describes optoelectronic devices for collecting three-dimensional data without determining disparity. The approach permits significant flexibility relative to state-of-the-art approaches for collecting three-dimensional data. For example, the illuminations (e.g., patterns) generated by the disclosed devices need not exhibit the same degree of complexity or randomness of illuminations generally required by structured-light approaches. Further, the devices described in the present disclosure exhibit, in some instances, optimal resolution over a wide distance range. An optoelectronic device described herein is operable to generate an illumination comprising a plurality of high-intensity features that exhibit a distance-dependent alteration when imaged by an imager incorporated into the optoelectronic device.

Growing community of inventors

San Jose, CA, United States of America

Florin Cutu