Stradvision, Inc.

Stradivision, Inc.

Insu Kim

Hongmo Je

Yongjoong Kim

Wooju Ryu

Hojin Cho

SukHoon Boo

Taewoong Jang

Myungchul Sung

Kyungjoong Jeong

Donghun Yeo

Kye-Hyeon Kim

Hak-Kyoung Kim

Woonhyun Nam

Woonhyu Nam

Sukhoon Boo

A method for learning parameters of a CNN using a 1&#xd7;K convolution operation or a K&#xd7;1 convolution operation is provided to be used for hardware optimization which satisfies KPI. The method includes steps of: a learning device (a) instructing a reshaping layer to two-dimensionally concatenate features in each group comprised of corresponding K channels of a training image or its processed feature map, to thereby generate a reshaped feature map, and instructing a subsequent convolutional layer to apply the 1&#xd7;K or the K&#xd7;1 convolution operation to the reshaped feature map, to thereby generate an adjusted feature map; and (b) instructing an output layer to refer to features on the adjusted feature map or its processed feature map, and instructing a loss layer to calculate losses by referring to an output from the output layer and its corresponding GT.

Learning method and learning device for CNN using 1xK or Kx1 convolution to be used for hardware optimization, and testing method and testing device using the same

A method for on-device continual learning of a neural network which analyzes input data is provided to be used for smartphones, drones, vessels, or a military purpose. The method includes steps of: a learning device, (a) sampling new data to have a preset first volume, instructing an original data generator network, which has been learned, to repeat outputting synthetic previous data corresponding to a k-dimension random vector and previous data having been used for learning the original data generator network, such that the synthetic previous data has a second volume, and generating a batch for a current-learning; and (b) instructing the neural network to generate output information corresponding to the batch. The method can be performed by generative adversarial networks (GANs), online learning, and the like. Also, the present disclosure has effects of saving resources such as storage, preventing catastrophic forgetting, and securing privacy.

Method and device for on-device continual learning of a neural network which analyzes input data, and method and device for testing the neural network to be used for smartphones, drones, vessels, or military purpose

A method for generating a lane departure warning (LDW) alarm by referring to information on a driving situation is provided to be used for ADAS, V2X or driver safety which are required to satisfy level 4 and level 5 of autonomous vehicles. The method includes steps of: a computing device instructing a LDW system (i) to collect information on the driving situation including information on whether a specific spot corresponding to a side mirror on a side of a lane, into which the driver desires to change, belongs to a virtual viewing frustum of the driver and (ii) to generate risk information on lane change by referring to the information on the driving situation; and instructing the LDW system to generate the LDW alarm by referring to the risk information. Thus, the LDW alarm can be provided to neighboring autonomous vehicles of level 4 and level 5.

Method and device for learning generating lane departure warning (LDW) alarm by referring to information on driving situation to be used for ADAS, V2X or driver safety required to satisfy level 4 and level 5 of autonomous vehicles

A learning method of a CNN (Convolutional Neural Network) for monitoring one or more blind spots of a monitoring vehicle is provided. The learning method includes steps of: a learning device, if training data corresponding to output from a detector on the monitoring vehicle is inputted, instructing a cue information extracting layer to uses class information and location information on a monitored vehicle included in the training data, thereby outputting cue information on the monitored vehicle; instructing an FC layer for monitoring the blind spots to perform neural network operations by using the cue information, thereby outputting a result of determining whether the monitored vehicle is located on one of the blind spots; and instructing a loss layer to generate loss values by referring to the result and its corresponding GT, thereby learning parameters of the FC layer for monitoring the blind spots by backpropagating the loss values.

Learning method and testing method for monitoring blind spot of vehicle, and learning device and testing device using the same

A method for optimizing a hyperparameter of an auto-labeling device performing auto-labeling and auto-evaluating of a training image to be used for learning a neural network is provided for computation reduction and achieving high precision. The method includes steps of: an optimizing device, (a) instructing the auto-labeling device to generate an original image with its auto label and a validation image with its true and auto label, to assort the original image with its auto label into an easy-original and a difficult-original images, and to assort the validation image with its own true and auto labels into an easy-validation and a difficult-validation images; and (b) calculating a current reliability of the auto-labeling device, generating a sample hyperparameter set, calculating a sample reliability of the auto-labeling device, and optimizing the preset hyperparameter set. This method can be performed by a reinforcement learning with policy gradient algorithms.

Method for optimizing hyperparameters of auto-labeling device which auto-labels training images for use in deep learning network to analyze images with high precision, and optimizing device using the same

A method for learning a pedestrian detector to be used for robust surveillance or military purposes based on image analysis is provided for a solution to a lack of labeled images and for a reduction of annotation costs. The method can be also performed by using generative adversarial networks (GANs). The method includes steps of: a learning device generating an image patch by cropping each of regions on a training image, and instructing an adversarial style transformer to generate a transformed image patch by converting each of pedestrians into transformed pedestrians capable of impeding a detection; and generating a transformed training image by replacing each of the regions with the transformed image patch, instructing the pedestrian detector to detecting the transformed pedestrians, and learning parameters of the pedestrian detector to minimize losses. This learning, as a self-evolving system, is robust to adversarial patterns by generating training data including hard examples.

Learning method and learning device of pedestrian detector for robust surveillance based on image analysis by using GAN and testing method and testing device using the same

A method for on-device continual learning of a neural network which analyzes input data is provided for smartphones, drones, vessels, or a military purpose. The method includes steps of: a learning device, (a) uniform-sampling new data to have a first volume, instructing a boosting network to convert a k-dimension random vector into a k-dimension modified vector, instructing an original data generator network to repeat outputting synthetic previous data of a second volume corresponding to the k-dimension modified vector and previous data having been used for learning, and generating a batch for a current-learning; and (b) instructing the neural network to generate output information corresponding to the batch. The method can be used for preventing catastrophic forgetting and an invasion of privacy, and for optimizing resources such as storage and sampling processes for training images. Further the method can be performed through a learning for Generative adversarial networks (GANs).

Method and device for on-device continual learning of neural network which analyzes input data by optimized sampling of training images, and method and device for testing the neural network for smartphones, drones, vessels, or military purpose

A method for learning parameters of an object detector based on a CNN adaptable to customer's requirements such as KPI by using a target object estimating network and a target object merging network is provided. The CNN can be redesigned when scales of objects change as a focal length or a resolution changes depending on the KPI. The method includes steps of: a learning device instructing convolutional layers to generate a k-th feature map by applying convolution operations to a k-th manipulated image which corresponds to the (k&#x2212;1)-th target region on an image; and instructing the target object merging network to merge a first to an n-th object detection information, outputted from an FC layer, and backpropagating losses generated by referring to merged object detection information and its corresponding GT. The method can be useful for multi-camera, SVM (surround view monitor), and the like, as accuracy of 2D bounding boxes improves.

Learning method and learning device for object detector based on reconfigurable network for optimizing customers' requirements such as key performance index using target object estimating network and target object merging network, and testing method and testing device using the same

There is provided a method for determining an FL value to be used for optimizing hardware applicable to mobile devices, compact networks, and the like with high precision. The method includes steps of: a computing device (a) applying quantization operations to original values included in an original vector by referring to a BW value and each of FL candidate values, to thereby generate each of quantized vectors, including the quantized values, corresponding to each of the FL candidate values; (b) generating each of weighted quantization loss values, corresponding to each of the FL candidate values, by applying weighted quantization loss operations to information on each of differences between the original values and the quantized values included in each of the quantized vectors; and (c) determining the FL value among the FL candidate values by referring to the weighted quantization loss values and a device using the same.

Determining FL value by using weighted quantization loss values to thereby quantize CNN parameters and feature values to be used for optimizing hardware applicable to mobile devices or compact networks with high precision

A learning method for a CNN (Convolutional Neural Network) capable of encoding at least one training image with multiple feeding layers, wherein the CNN includes a 1st to an n-th convolutional layers, which respectively generate a 1st to an n-th main feature maps by applying convolution operations to the training image, and a 1st to an h-th feeding layers respectively corresponding to h convolutional layers (1&#x2264;h&#x2264;n&#x2212;1)) is provided. The learning method includes steps of: a learning device instructing the convolutional layers to generate the 1st to the n-th main feature maps, wherein the learning device instructs a k-th convolutional layer to acquire a (k&#x2212;1)-th main feature map and an m-th sub feature map, and to generate a k-th main feature map by applying the convolution operations to the (k&#x2212;1)-th integrated feature map generated by integrating the (k&#x2212;1)-th main feature map and the m-th sub feature map.

Learning method, learning device with multi-feeding layers and testing method, testing device using the same

A method for evaluating a reliability of labeling training images to be used for learning a deep learning network is provided. The method includes steps of: a reliability-evaluating device instructing a similar-image selection network to select validation image candidates with their own true labels having shooting environments similar to those of acquired original images, which are unlabeled images, and instructing an auto-labeling network to auto-label the validation image candidates with their own true labels and the original images; and (i) evaluating a reliability of the auto-labeling network by referring to true labels and auto labels of easy-validation images, and (ii) evaluating a reliability of a manual-labeling device by referring to true labels and manual labels of difficult-validation images. This method can be used to recognize surroundings by applying a bag-of-words model, to optimize sampling processes for selecting a valid image among similar images, and to reduce annotation costs.

Method for automatically evaluating labeling reliability of training images for use in deep learning network to analyze images, and reliability-evaluating device using the same

A method for generating at least one data set for learning to be used for detecting at least one obstruction in autonomous driving circumstances is provided. The method includes steps of: a computing device (a) obtaining a first original image indicating a driving situation, and a first segmentation ground truth (GT) image corresponding to the first original image; (b) obtaining a second original image including a specific object, and a second segmentation GT image which includes segmentation information for the specific object and corresponds to the second original image; (c) obtaining a third original image by cutting a portion corresponding to the specific object, and a third segmentation GT image by cutting pixels corresponding to an area where the specific object is located; and (d) creating the data set for learning which includes a fourth original image and a fourth segmentation GT image corresponding to the fourth original image.

Method and computing device for generating image data set for learning to be used for detection of obstruction in autonomous driving circumstances and learning method and learning device using the same

A method for providing an Advanced Pedestrian Assistance System to protect a pedestrian preoccupied with a smartphone is provided. The method includes steps of: the smartphone instructing a locating unit to acquire 1-st information including location and velocity information of the pedestrian and location and velocity information of the smartphone; instructing a detecting unit to acquire 2-nd information including hazard statuses of hazardous areas near the pedestrian and location information and velocity information of hazardous objects, by referring to images acquired by phone cameras linked with the smartphone and the 1-st information; and instructing a control unit to calculate a degree of pedestrian safety of the pedestrian by referring to the 1-st and the 2-nd information, and to transmit a hazard alert to the pedestrian via the smartphone. Further, the method can be used for surveillance or a military purpose.

Method and device for providing advanced pedestrian assistance system to protect pedestrian preoccupied with smartphone

A learning method for improving segmentation performance to be used for detecting road user events including pedestrian events and vehicle events using double embedding configuration in a multi-camera system is provided. The learning method includes steps of: a learning device instructing similarity convolutional layer to generate similarity embedding feature by applying similarity convolution operations to a feature outputted from a neural network; instructing similarity loss layer to output a similarity loss by referring to a similarity between two points sampled from the similarity embedding feature, and its corresponding GT label image; instructing distance convolutional layer to generate distance embedding feature by applying distance convolution operations to the similarity embedding feature; instructing distance loss layer to output a distance loss for increasing inter-class differences among mean values of instance classes and decreasing intra-class variance values of the instance classes; backpropagating at least one of the similarity loss and the distance loss.

Learning method and learning device for improving segmentation performance to be used for detecting road user events using double embedding configuration in multi-camera system and testing method and testing device using the same

Growing community of inventors

Gyeongsangbuk-do, South Korea

Insu Kim