Black Sesame Technologies Inc.

Nvidia Corporation

Black Sesame International Holding Limited

Sesame Technologies Inc.

Zuoguan Wang

Qun Gu

Jizhang Shan

Minwoo Park

John R Zedlewski

Pekka Janis

Gordon Grigor

Miguel Sainz

Davide Onofrio

Yifang Xu

Chia-Chih Chen

Carolina Parada

Vijay Chintalapudi

Jan Nikolaus Fritsch

I-Kuei Chen

Xin Liu

Mehdi Sajjadi Mohammadabadi

Ozan Tonkal

Yilin Song

Tian Zhou

Zheng Li

Ying Zhou

Zheng Qi

Yuekun Zhang

Liying Liu

Chenghao Zhang

In various examples, sensor data representative of an image of a field of view of a vehicle sensor may be received and the sensor data may be applied to a machine learning model. The machine learning model may compute a segmentation mask representative of portions of the image corresponding to lane markings of the driving surface of the vehicle. Analysis of the segmentation mask may be performed to determine lane marking types, and lane boundaries may be generated by performing curve fitting on the lane markings corresponding to each of the lane marking types. The data representative of the lane boundaries may then be sent to a component of the vehicle for use in navigating the vehicle through the driving surface.

Real-time detection of lanes and boundaries by autonomous vehicles

An example fused convolutional layer, comprising, a comparator capable of reception of a first zero point and a multiply-accumulation result, a first multiplexer coupled to the comparator, wherein the first multiplexer receives a plurality of power-of-two exponent values, a shift normalizer, coupled to the first multiplexer, wherein the shift normalizer is capable of receiving the multiply-accumulation result and the plurality of power-of-two exponent values, wherein the shift normalizer limits a quantization of the multiply-accumulation result to a power-of-two scale and a second multiplexer coupled to an output of the shift normalizer, the first multiplexer and receives a second zero point and outputs an activation.

Integer-based fused convolutional layer in a convolutional neural network

A method of hybrid lane modeling, including, receiving a roadway image, extracting a set of lane points from the roadway image, fitting a polynomial line to the set of lane points, determining a fitted error of the fitted polynomial line, outputting the polynomial line if the fitted error is less than a predetermined threshold, selecting a set of clean lane points from the set of lane points if the fitted error is greater than the predetermined threshold and interpolating a cubic spline line to the set of clean lane points.

Hybrid lane model

A method of pruning a convolutional neural network, comprising at least one of determining a number of channels (N) between a network input and a network output, constructing N lookup tables, each lookup table matched to a respective channel and pruning filters in the convolutional neural network to create a shortcut between the network input and the network output based on the N lookup tables.

Method of structured network pruning and sparsity speed-up

A method of adaptive quantization for a convolutional neural network, includes at least one of receiving an acceptable model accuracy, determining a float value multiply accumulate for the layer based on a float value weight and a float value input, quantizing the float value weight at multiple weight quantization precisions, quantizing the float value input at multiple input quantization precisions, determining a multiply accumulate at multiple multiply accumulate quantization precisions based on the weight quantization precisions and the input quantization precisions, determining multiple quantization errors based on differences between the float value multiply accumulate and the multiple multiply accumulate quantization precisions and selecting one of the multiple weight quantization precisions, one of the multiple input quantization precisions and one of the multiple multiply accumulate quantization precisions based on the predetermined acceptable model accuracy and the multiple quantization errors.

Adaptive quantization and mixed precision in a network

A method of depth estimation utilizing heterogeneous cameras, comprising, homogenizing a first camera image and a second camera image based on a first camera calibration dataset and a second camera calibration dataset respectively, wherein the first camera image and second camera image are distortion corrected and are zoom compensated, determining an initial image pair rectification transformation matrix of the homogenized first camera image and the homogenized second camera image, determining a delta image pair rectification transformation matrix based on the initial image pair rectification transformation matrix, determining, a final image pair rectification transformation matrix based on the initial image pair rectification transformation matrix and the delta image pair rectification transformation matrix resulting in a final rectified image pair and disparity mapping the final rectified image pair based on a depth net regression.

Stereo vision with weakly aligned heterogeneous cameras

A method of partial frame perception, comprising receiving a bottom portion of an inverted image, detecting traffic areas of interest in the bottom portion of the inverted image, streaming the detected traffic areas of interest to a perception processor, receiving a top portion of the inverted image, detecting stationary areas of interest in the top portion of the inverted image and streaming the detected stationary areas of interest to the perception processor.

Partial frame perception

A method of depth estimation utilizing heterogeneous cameras, comprising, homogenizing a first camera image and a second camera image based on a first camera calibration dataset and a second camera calibration dataset respectively, wherein the first camera image and second camera image are distortion corrected and are zoom compensated, determining an initial image pair rectification transformation matrix of the homogenized first camera image and the homogenized second camera image, determining a delta image pair rectification transformation matrix based on the initial image pair rectification transformation matrix, determining a final image pair rectification transformation matrix based on the initial image pair rectification transformation matrix and the delta image pair rectification transformation matrix resulting in a final rectified image pair and disparity mapping the final rectified image pair based on a depth net regression.

A method of distant on-road object detection, comprising, capturing an initial frame of a roadway lane in a direction of travel, detecting lane edges in the initial frame, capturing a subsequent frame of the roadway lane, cropping the subsequent frame based on the detected lane edges of the initial frame resulting in an image patch, detecting an image patch object within the image patch, down-sampling the subsequent frame, detecting a down-sampled object within the down-sampled subsequent frame and merging the detections within the image patch and the detections within the down-sampled subsequent frame.

Distant on-road object detection

A method of parking lot tracking including receiving a plurality of camera images from a plurality of cameras attached to a vehicle in motion, stitching the plurality of camera images to simulate a surround view of the vehicle, recognizing at least one potential parking space within the surround view of the vehicle, estimating a motion parameter by camera motion estimation of the plurality of cameras and tracking the at least one potential parking space based on the motion parameter.

Growing community of inventors

Los Gatos, CA, United States of America

Zuoguan Wang