Stradvision, Inc.

Intel Corporation

Stadvision, Inc.

Stradivision, Inc.

Hongmo Je

Wooju Ryu

Yongjoong Kim

Hojin Cho

SukHoon Boo

Taewoong Jang

Myungchul Sung

Donghun Yeo

Kyungjoong Jeong

Woonhyun Nam

Kye-Hyeon Kim

Hak-Kyoung Kim

Insu Kim

Hyungsoo Lee

Myeong-Chun Lee

DongSoo Shin

Dongsoo Shin

Bongnam Kang

Sung An Gweon

Tae-Hoon Kim

Dongkyu Yu

YongChan Park

Jinwook Lee

Taesup Kim

Minje Park

Yeongjae Cheon

Woonhyu Nam

Sukhoon Boo

Sungan Gweon

Taewong Jang

A method for learning parameters of a CNN using a 1&#xd7;K convolution operation or a K&#xd7;1 convolution operation is provided to be used for hardware optimization which satisfies KPI. The method includes steps of: a learning device (a) instructing a reshaping layer to two-dimensionally concatenate features in each group comprised of corresponding K channels of a training image or its processed feature map, to thereby generate a reshaped feature map, and instructing a subsequent convolutional layer to apply the 1&#xd7;K or the K&#xd7;1 convolution operation to the reshaped feature map, to thereby generate an adjusted feature map; and (b) instructing an output layer to refer to features on the adjusted feature map or its processed feature map, and instructing a loss layer to calculate losses by referring to an output from the output layer and its corresponding GT.

Learning method and learning device for CNN using 1xK or Kx1 convolution to be used for hardware optimization, and testing method and testing device using the same

A method for on-device continual learning of a neural network which analyzes input data is provided to be used for smartphones, drones, vessels, or a military purpose. The method includes steps of: a learning device, (a) sampling new data to have a preset first volume, instructing an original data generator network, which has been learned, to repeat outputting synthetic previous data corresponding to a k-dimension random vector and previous data having been used for learning the original data generator network, such that the synthetic previous data has a second volume, and generating a batch for a current-learning; and (b) instructing the neural network to generate output information corresponding to the batch. The method can be performed by generative adversarial networks (GANs), online learning, and the like. Also, the present disclosure has effects of saving resources such as storage, preventing catastrophic forgetting, and securing privacy.

Method and device for on-device continual learning of a neural network which analyzes input data, and method and device for testing the neural network to be used for smartphones, drones, vessels, or military purpose

A method for updating an object detector of an autonomous vehicle to adapt the object detector to a driving circumstance is provided. The method includes steps of: a learning device (a) (i) inputting a training image, corresponding to a driving circumstance, into a circumstance-specific object detector to apply (i-1) convolution to the training image to generate a circumstance-specific feature map, (i-2) ROI pooling to the circumstance-specific feature map to generate a circumstance-specific pooled feature map, and (i-3) fully-connected operation to the circumstance-specific pooled feature map to generate circumstance-specific object detection information and (ii) inputting the circumstance-specific feature map into a circumstance-specific ranking network to (ii-1) apply deconvolution to the circumstance-specific feature map and generate a circumstance-specific segmentation map and (ii-2) generate a circumstance-specific rank score via a circumstance-specific discriminator; and (b) training the circumstance-specific object detector, the circumstance-specific deconvolutional layer, the circumstance-specific convolutional layer, and the circumstance-specific discriminator.

Learning method and learning device for updating object detector, based on deep learning, of autonomous vehicle to adapt the object detector to driving circumstance, and updating method and updating device using the same

A method for performing on-device learning of embedded machine learning network of autonomous vehicle by using multi-stage learning with adaptive hyper-parameter sets is provided. The processes include: (a) dividing the current learning into a 1-st stage learning to an n-th stage learning, assigning 1-st stage training data to n-th stage training data, generating a 1_1-st hyper-parameter set candidate to a 1_h-th hyper-parameter set candidate, training the embedded machine learning network in the 1-st stage learning, and determining a 1-st adaptive hyper-parameter set; (b) generating a k_1-st hyper-parameter set candidate to a k_h-th hyper-parameter set candidate, training the (k&#x2212;1)-th stage-completed machine learning network in the k-th stage learning, and determining a k-th adaptive hyper-parameter set; and (c) generating an n-th adaptive hyper-parameter set, and executing the n-th stage learning, to thereby complete the current learning.

Method for performing on-device learning of machine learning network on autonomous vehicle by using multi-stage learning with adaptive hyper-parameter sets and device using the same

A method of on-vehicle active learning for training a perception network of an autonomous vehicle is provided. The method includes steps of: an on-vehicle active learning device, (a) if a driving video and sensing information are acquired from a camera and sensors on an autonomous vehicle, inputting frames of the driving video and the sensing information into a scene code assigning module to generate scene codes including information on scenes in the frames and on driving events; and (b) at least one of selecting a part of the frames, whose object detection information satisfies a condition, as specific frames by using the scene codes and the object detection information and selecting a part of the frames, matching a training policy, as the specific frames by using the scene codes and the object detection information, and storing the specific frames and specific scene codes in a frame storing part.

Method and device for on-vehicle active learning to be used for training perception network of autonomous vehicle

A method for explainable active learning, to be used for an object detector, by using a deep autoencoder is provided. The method includes steps of an active learning device (a) (i) inputting acquired test images into the object detector to detect objects and output bounding boxes, (ii) cropping regions, corresponding to the bounding boxes, in the test images, (iii) resizing the test images and the cropped images into a same size, and (iv) inputting the resized images into a data encoder of the deep autoencoder to output data codes, and (b) (i) confirming reference data codes corresponding to the number of the resized images less than a counter threshold by referring to a data codebook, (ii) extracting specific data codes from the data codes, (iii) selecting specific test images as rare samples, and (iv) updating the data codebook by referring to the specific data codes.

Method for explainable active learning, to be used for object detector, by using deep encoder and active learning device using the same

A method of self-supervised learning for detection network using deep Q-network includes steps of: performing object detection on first unlabeled image through the detection network trained with training database to generate first object detection information and performing learning operation on a first state set corresponding to the first object detection information to generate a Q-value, if an action of the Q-value accepts the first unlabeled image, testing the detection network, retrained with the training database additionally containing a labeled image of the first unlabeled image, to generate a first accuracy, and if the action rejects the first unlabeled image, testing the detection network without retraining, to generate a second accuracy, and storing the first state set, the action, a reward of the first or the second accuracy, and a second state set of a second unlabeled image as transition vector, and training the deep Q-network by using the transition vector.

Methods for performing self-supervised learning of deep-learning based detection network by using deep Q-network and devices using the same

A method for learning a recurrent neural network to check an autonomous driving safety to be used for switching a driving mode of an autonomous vehicle is provided. The method includes steps of: a learning device (a) if training images corresponding to a front and a rear cameras of the autonomous vehicle are acquired, inputting each pair of the training images into corresponding CNNs, to concatenate the training images and generate feature maps for training, (b) inputting the feature maps for training into long short-term memory models corresponding to sequences of a forward RNN, and into those corresponding to the sequences of a backward RNN, to generate updated feature maps for training and inputting feature vectors for training into an attention layer, to generate an autonomous-driving mode value for training, and (c) allowing a loss layer to calculate losses and to learn the long short-term memory models.

Learning method and learning device of recurrent neural network for autonomous driving safety check for changing driving mode between autonomous driving mode and manual driving mode, and testing method and testing device using them

A learning method for acquiring at least one personalized reward function, used for performing a Reinforcement Learning (RL) algorithm, corresponding to a personalized optimal policy for a subject driver is provided. And the method includes steps of: (a) a learning device performing a process of instructing an adjustment reward network to generate first adjustment rewards, by referring to the information on actual actions and actual circumstance vectors in driving trajectories, a process of instructing a common reward module to generate first common rewards by referring to the actual actions and the actual circumstance vectors, and a process of instructing an estimation network to generate actual prospective values by referring to the actual circumstance vectors; and (b) the learning device instructing a first loss layer to generate an adjustment reward and to perform backpropagation to learn parameters of the adjustment reward network.

Learning method and learning device for supporting reinforcement learning by using human driving data as training data to thereby perform personalized path planning

A method for generating a lane departure warning (LDW) alarm by referring to information on a driving situation is provided to be used for ADAS, V2X or driver safety which are required to satisfy level 4 and level 5 of autonomous vehicles. The method includes steps of: a computing device instructing a LDW system (i) to collect information on the driving situation including information on whether a specific spot corresponding to a side mirror on a side of a lane, into which the driver desires to change, belongs to a virtual viewing frustum of the driver and (ii) to generate risk information on lane change by referring to the information on the driving situation; and instructing the LDW system to generate the LDW alarm by referring to the risk information. Thus, the LDW alarm can be provided to neighboring autonomous vehicles of level 4 and level 5.

Method and device for learning generating lane departure warning (LDW) alarm by referring to information on driving situation to be used for ADAS, V2X or driver safety required to satisfy level 4 and level 5 of autonomous vehicles

A method for achieving better performance in autonomous driving while saving computing power, by using confidence scores representing a credibility of an object detection which is generated in parallel with an object detection process is provided. And the method includes steps of: (a) a computing device acquiring at least one circumstance image on surroundings of a subject vehicle, through at least one image sensor installed on the subject vehicle; (b) the computing device instructing a convolutional neural network (CNN) to apply at least one CNN operation to the circumstance image, thereby to generate initial object information and initial confidence information on the circumstance image; and (c) the computing device generating final object information on the circumstance image by referring to the initial object information and the initial confidence information with a support of a Reinforcement learning (RL) reinforcement learning (RL) agent, and through V2X communications with at least part of surrounding objects.

Method and device for attention-driven resource allocation by using reinforcement learning and V2X communication to thereby achieve safety of autonomous driving

A learning method of a CNN (Convolutional Neural Network) for monitoring one or more blind spots of a monitoring vehicle is provided. The learning method includes steps of: a learning device, if training data corresponding to output from a detector on the monitoring vehicle is inputted, instructing a cue information extracting layer to uses class information and location information on a monitored vehicle included in the training data, thereby outputting cue information on the monitored vehicle; instructing an FC layer for monitoring the blind spots to perform neural network operations by using the cue information, thereby outputting a result of determining whether the monitored vehicle is located on one of the blind spots; and instructing a loss layer to generate loss values by referring to the result and its corresponding GT, thereby learning parameters of the FC layer for monitoring the blind spots by backpropagating the loss values.

Learning method and testing method for monitoring blind spot of vehicle, and learning device and testing device using the same

A method for training an object detection network by using attention maps is provided. The method includes steps of: (a) an on-device learning device inputting the training images into a feature extraction network, inputting outputs of the feature extraction network into a attention network and a concatenation layer, and inputting outputs of the attention network into the concatenation layer; (b) the on-device learning device inputting outputs of the concatenation layer into an RPN and an ROI pooling layer, inputting outputs of the RPN into a binary convertor and the ROI pooling layer, and inputting outputs of the ROI pooling layer into a detection network and thus to output object detection data; and (c) the on-device learning device train at least one of the feature extraction network, the detection network, the RPN and the attention network through backpropagations using an object detection losses, an RPN losses, and a cross-entropy losses.

Learning method and learning device for training an object detection network by using attention maps and testing method and testing device using the same

A method for optimizing an on-device neural network model by using a Sub-kernel Searching Module is provided. The method includes steps of a learning device (a) if a Big Neural Network Model having a capacity capable of performing a targeted task by using a maximal computing power of an edge device has been trained to generate a first inference result on an input data, allowing the Sub-kernel Searching Module to identify constraint and a state vector corresponding to the training data, to generate architecture information on a specific sub-kernel suitable for performing the targeted task on the training data, (b) optimizing the Big Neural Network Model according to the architecture information to generate a specific Small Neural Network Model for generating a second inference result on the training data, and (c) training the Sub-kernel Searching Module by using the first and the second inference result.

Method for optimizing on-device neural network model by using sub-kernel searching module and device using the same

A method for training a deep learning network based on artificial intelligence is provided. The method includes steps of: a learning device (a) inputting unlabeled data into an active learning network to acquire sub unlabeled data and inputting the sub unlabeled data into an auto labeling network to generate new labeled data; (b) allowing a continual learning network to sample the new labeled data and existing labeled data to generate a mini-batch, and train the existing learning network using the mini-batch to acquire a trained learning network, wherein part of the mini-batch are selected by referring to specific existing losses; and (c) (i) allowing an explainable analysis network to generate insightful results on validation data and transmit the insightful results to a human engineer to transmit an analysis of the trained learning network and (ii) modifying at least one of the active learning network and the continual learning network.

Growing community of inventors

Pohang-si, South Korea

Hongmo Je