Texas Instruments Corporation

Rivian IP Holdings, LLC

Vikram Vijayanbabu Appia

Shashank Dabral

Sujith Shivalingappa

Aziz Umit Batur

Do-Kyoung Kwon

Hemant Hariyani

Ibrahim Ethem Pekkucuksen

Mihir Narendra Mody

Niraj Nandan

Gang Hua

Hetul Sanghvi

Rajasekhar Reddy Allu

Brian Okchon Chae

Martin Fritz Mueller

Sanjeev Jagannatha Koppal

Janice Shuay-Ann Pan

Mayank Mangla

Brijesh Rameshbhai Jadav

Hemant Vijay Kumar Hariyani

Lucas Carl Weaver

Anand Yalgurdrao Kulkarni

Umit Batur

Qian Song

Muhammad Zubair Ikram

Dan Wang

Yingmao Li

Buyue Zhang

Rajesh Narasimha

Pedro Ramon Gelabert

Tarek Aziz Lahlou

Ziguo Zhong

Yucheng Liu

Utkarsh Deshmukh

Akshay Arvind Velankar

Vishwas Venkatachalapathy

Andrei Polzounov

Ravi Kumar Satzoda

Methods and systems are provided to detect an instance of a line in a two-dimensional image captured by a vehicle and to determine whether the instance of the line is a lane boundary for a lane that will be used by the vehicle to traverse a route. An instance of a line in a two-dimensional image captured by a vehicle is detected using processing circuitry. The processing circuitry is used to determine that the instance of the line is a lane boundary for a lane associated with the vehicle. A curve fit for the lane boundary based on the instance of the line is determined using the processing circuitry. The processing circuitry is also used to determine a sinuosity of the lane based on the curve fit. Execution of a vehicle action is facilitated using the processing circuitry based on the determined sinuosity.

System and method for deep learning based lane curvature detection from 2D images

Apparatuses, systems, and methods for determining a field in a path of travel of a vehicle for detection of objects in the path of travel. An apparatus includes a path identifier configured to identify a path of travel for a vehicle. At least one camera is disposed on the vehicle and is configured to capture image data of an area including the path of travel. A region of interest selector is configured to select a region of interest within the image data. A horizon identifier is configured to identify a horizon in the image data. A field determiner is configured to project the horizon onto the region of interest to isolate a field specified by the path of travel and the horizon, the field being evaluatable for the presence of objects in the path of travel of the vehicle.

Field determination for obstacle detection

Systems and methods for determining the drivable space of a road, for applications such as autonomous navigation. To determine the non-drivable space under another vehicle, systems and methods of embodiments of the disclosure generate 3D bounding boxes from 2D bounding boxes of objects in captured roadway images, and from various geometric constraints. Image portions may be labeled as drivable or non-drivable according to projections of these 3D bounding boxes onto their road surfaces. These labeled images, along with accompanying semantic information, may be compiled to form training datasets for a machine learning model such as a CNN. The training datasets may train the CNN to classify input image portions into drivable and non-drivable space, for applications such as autonomous navigation.

Systems and methods for determining drivable space

A system for determining blockage of a vehicle camera is configured to apply a partition grid having a plurality of locations to a sequence of images to form a plurality of regions. The system determines at least one spatial feature corresponding to the partition grid and at least one temporal feature corresponding to the partition grid. The system generates a sequence of classifications for each location of the plurality of locations based on the at least one spatial feature, the at least one temporal feature, and reference information. The system applies a smoothing technique to determine a subset of regions that are blocked among the sequence of classifications, and generates an output signal based on the subset of regions. The output may be provided to an output system to wash the camera lens, notify the user or the vehicle of the blockage, or modify image processing.

Systems and methods for determining camera blockage

A system on a chip (SoC) includes a digital signal processor (DSP) and a graphics processing unit (GPU) coupled to the DSP. The DSP is configured to receive a stream of received depth measurements and generate a virtual bowl surface based on the stream of received depth measurements. The DSP is also configured to generate a bowl to physical camera mapping based on the virtual bowl surface. The GPU is configured to receive a first texture and receive a second texture. The GPU is also configured to perform physical camera to virtual camera transformation on the first texture and on the second texture, based on the bowl to physical camera mapping, to generate an output image.

Surround view

A method for computing a depth map of a scene in a structured light imaging system including a time-of-flight (TOF) sensor and a projector is provided that includes capturing a plurality of high frequency phase-shifted structured light images of the scene using a camera in the structured light imaging system, generating, concurrently with the capturing of the plurality of high frequency phase-shifted structured light images, a time-of-flight (TOF) depth image of the scene using the TOF sensor, and computing the depth map from the plurality of high frequency phase-shifted structured light images wherein the TOF depth image is used for phase unwrapping.

Time-of-flight (TOF) assisted structured light imaging

A method for automatic generation of calibration parameters for a surround view (SV) camera system is provided that includes capturing a video stream from each camera comprised in the SV camera system, wherein each video stream captures two calibration charts in a field of view of the camera generating the video stream; displaying the video streams in a calibration screen on a display device coupled to the SV camera system, wherein a bounding box is overlaid on each calibration chart, detecting feature points of the calibration charts, displaying the video streams in the calibration screen with the bounding box overlaid on each calibration chart and detected features points overlaid on respective calibration charts, computing calibration parameters based on the feature points and platform dependent parameters comprising data regarding size and placement of the calibration charts, and storing the calibration parameters in the SV camera system.

Calibration of a surround view camera system

A method of image processing in a structured light imaging system is provided that includes receiving a captured image of a scene, wherein the captured image is captured by a camera of a projector-camera pair, and wherein the captured image includes a binary pattern projected into the scene by the projector, applying a filter to the rectified captured image to generate a local threshold image, wherein the local threshold image includes a local threshold value for each pixel in the rectified captured image, and extracting a binary image from the rectified captured image wherein a value of each location in the binary image is determined based on a comparison of a value of a pixel in a corresponding location in the rectified captured image to a local threshold value in a corresponding location in the local threshold image.

Structured light depth imaging under various lighting conditions

Disclosed embodiments include apparatuses, systems, and methods for determining a field in a path of travel of a vehicle for detection of objects in the path of travel. In an illustrative embodiment, an apparatus includes a path identifier configured to identify a path of travel for a vehicle. At least one camera is disposed on the vehicle and is configured to capture image data of an area including the path of travel. A region of interest selector is configured to select a region of interest within the image data. A horizon identifier is configured to identify a horizon in the image data. A field determiner is configured to project the horizon onto the region of interest to isolate a field specified by the path of travel and the horizon, the field being evaluatable for the presence of objects in the path of travel of the vehicle.

A method for automatic keystone correction in a projection system is provided that includes rectifying a first image to be projected using current keystone correction parameters to generate a first rectified image, projecting, by a projector in the projection system, the first rectified image on a projection surface, determining whether or not current keystone correction parameters are providing sufficient keystone correction after the first rectified image is projected, rectifying a second image to be projected using the current keystone correction parameters to generate a second rectified image if the current keystone correction parameters are providing sufficient keystone correction, computing new keystone correction parameters if the current keystone correction parameters are not providing sufficient keystone correction and rectifying the second image to be projected using the new keystone correction parameters to generate the second rectified image, and projecting, by the projector, the second rectified image on the projection surface.

Automatic keystone correction in a projection system

An apparatus and method for geometrically correcting a distorted input frame and generating an undistorted output frame. The apparatus includes an external memory block that stores the input frame, a counter block to compute output coordinates of the output frame for a region based on a block size of the region, a back mapping block to generate input coordinates corresponding to each of the output coordinates, a bounding module to compute input blocks corresponding to each of the input coordinates, a buffer module to fetch data corresponding to each of the input blocks, an interpolation module to interpolate data from the buffer module and a display module that receives the interpolated data for each of the regions and stitch an output image. The method includes determining the size of the output block based on a magnification data.

Methods and system for efficient processing of generic geometric correction engine

A method of image processing in a structured light imaging device is provided that includes capturing a plurality of images of a scene into which a structured light pattern is projected by a projector in the structured light imaging device, extracting features in each of the captured images, finding feature matches between a reference image of the plurality of captured images and each of the other images in the plurality of captured images, rectifying each of the other images to align with the reference image, wherein each image of the other images is rectified based on feature matches between the image and the reference image, combining the rectified other images and the reference image using interpolation to generate a high resolution image, and generating a depth image using the high resolution image.

Super-resolution in structured light imaging

A method for displaying a surround view on a single display screen is disclosed. A plurality of image frames for a particular time may be received from a corresponding plurality of cameras. A viewpoint warp map corresponding to a predetermined first virtual viewpoint may be selected, wherein the viewpoint warp map defines a source pixel location in the plurality of image frames for each output pixel location in the display screen. The warp map was predetermined offline and stored for later use. An output image is synthesized for the display screen by selecting pixel data for each pixel of the output image from the plurality of image frames in accordance with the viewpoint warp map. The synthesized image is then displayed on a display screen.

Three dimensional rendering for surround view using predetermined viewpoint lookup tables

A method for generating a surround view (SV) image for display in a view port of an SV processing system is provided that includes capturing, by at least one processor, corresponding images of video streams from each camera of a plurality of cameras and generating, by the at least one processor, the SV image using ray tracing for lens remapping to identify coordinates of pixels in the images corresponding to pixels in the SV image.

Growing community of inventors

San Jose, CA, United States of America

Vikram Vijayanbabu Appia