Samsung Electronics Co., Ltd.

Qualcomm Incorporated

Mostafa El-Khamy

Jungwon Lee

Inyup Kang

Haoyu Ren

Yoo Jin Choi

Hessam Mahdavifar

Qingfeng Liu

Jaeyoung Kim

Arvind Yedla

Jinhong Wu

Dongwoon Bai

Marcel Nassar

Xianzhi Du

Amin Fazeli

Andrea Kang

Sanjiv Nanda

Mehmet Yavuz

Gennady Feygin

Hsien-Ping Lin

Rama Mythili Vadali

Maiyuran Wijay

Jaewon Yoo

Behnam Babagholami Mohamadabadi

Farhad Meshkati

Hyunsang Cho

Jongha Ryu

Aman Raj

Ryan Szeto

Tae-ui Kim

Naga Bhushan

Jilei Hou

Sang-Hyuck Ha

Shuangquan Wang

Young-Han Kim

Tae-Ui Kim

Amin Kheradmand

JaeYoung Kim

Weituo Hao

SangHyuck Ha

Igor Fedorov

Zhizhong Li

Keke Liu

A method includes receiving a binary annotation of source text; performing a close operation on the binary annotation to generate a closed annotation using an initial kernel size; defining one or more contours in the closed annotation using one or more bounding boxes, respectively; determining a subset of the one or more contours for which a percentage of area occupied by text within a corresponding bounding box exceeds a threshold; and generating a final annotation of the source text based on the subset of the one or more contours.

Methods and systems for semantically segmenting a source text image based on a text area threshold determination

A computer vision system including: one or more processors; and memory including instructions that, when executed by the one or more processors, cause the one or more processors to: determine a semantic multi-scale context feature and an instance multi-scale context feature of an input scene; generate a joint attention map based on the semantic multi-scale context feature and the instance multi-scale context feature; refine the semantic multi-scale context feature and instance multi-scale context feature based on the joint attention map; and generate a panoptic segmentation image based on the refined semantic multi-scale context feature and the refined instance multi-scale context feature.

System and method for dual-value attention and instance boundary aware regression in computer vision system

A convolutional neural network (CNN) system for generating a classification for an input image is presented. The CNN system comprises circuitry running on clock cycles and configured to compute a product of two received values, and at least one non-transitory computer-readable medium that stores instructions for the circuitry to derive a feature map based on at least the input image; puncture at least one selection among the feature map and a kernel by setting the value of an element at an index of the at least one selection to zero and cyclic shifting a puncture pattern to achieve a 1/d reduction in number of clock cycles, where d is an integer and puncture interval value >1. The feature map is convolved with the kernel to generate an output, and a classification of the input image is generated based on the output.

Method and apparatus for reducing computational complexity of convolutional neural networks

A method and system for training a neural network are provided. The method includes receiving an input image, selecting at least one data augmentation method from a pool of data augmentation methods, generating an augmented image by applying the selected at least one data augmentation method to the input image, and generating a mixed image from the input image and the augmented image.

Method and apparatus for data efficient semantic segmentation

Methods and devices are provided for performing federated learning. A global model is distributed from a server to a plurality of client devices. At each of the plurality of client devices: model inversion is performed on the global model to generate synthetic data; the global model is on an augmented dataset of collected data and the synthetic data to generate a respective client model; and the respective client model is transmitted to the server. At the server: client models are received from the plurality of client devices, where each client model is received from a respective client device of the plurality of client devices; model inversion is performed on each client model to generate a synthetic dataset; the client models are averaged to generate an averaged model; and the averaged model is trained using the synthetic dataset to generate an updated model.

Method and apparatus for federated learning

Apparatuses and methods of manufacturing same, systems, and methods for performing network parameter quantization in deep neural networks are described. In one aspect, multi-dimensional vectors representing network parameters are constructed from a trained neural network model. The multi-dimensional vectors are quantized to obtain shared quantized vectors as cluster centers, which are fine-tuned. The fine-tuned and shared quantized vectors/cluster centers are then encoded. Decoding reverses the process.

Method and apparatus for neural network quantization

A method for training a generator, by a generator training system including a processor and memory, includes: extracting training statistical characteristics from a batch normalization layer of a pre-trained model, the training statistical characteristics including a training mean μ and a training variance σ; initializing a generator configured with generator parameters; generating a batch of synthetic data using the generator; supplying the batch of synthetic data to the pre-trained model; measuring statistical characteristics of activations at the batch normalization layer and at the output of the pre-trained model in response to the batch of synthetic data, the statistical characteristics including a measured mean {circumflex over (μ)}and a measured variance {circumflex over (σ)}; computing a training loss in accordance with a loss function Lbased on μ, σ, {circumflex over (μ)}, and {circumflex over (σ)}; and iteratively updating the generator parameters in accordance with the training loss until a training completion condition is met to compute the generator.

Method and apparatus for data-free post-training network quantization and generating synthetic data based on a pre-trained machine learning model

A method and system for providing Gaussian weighted self-attention for speech enhancement are herein provided. According to one embodiment, the method includes receiving an input noise signal, generating a score matrix based on the received input noise signal, and applying a Gaussian weighted function to the generated score matrix.

Transformer with Gaussian weighted self-attention for speech enhancement

A system for performing echo cancellation includes: a processor configured to: receive a far-end signal; record a microphone signal including: a near-end signal; and an echo signal corresponding to the far-end signal; extract far-end features from the far-end signal; extract microphone features from the microphone signal; compute estimated near-end features by supplying the microphone features and the far-end features to an acoustic echo cancellation module including a recurrent neural network including: an encoder including a plurality of gated recurrent units; and a decoder including a plurality of gated recurrent units; compute an estimated near-end signal from the estimated near-end features; and transmit the estimated near-end signal to the far-end device. The recurrent neural network may include a contextual attention module; and the recurrent neural network may take, as input, a plurality of error features computed based on the far-end features, the microphone features, and acoustic path parameters.

System and method for acoustic echo cancelation using deep multitask recurrent neural networks

A system and a method to train a neural network are disclosed. A first image is weakly and strongly augmented. The first image, the weakly and strongly augmented first images are input into a feature extractor to obtain augmented features. Each weakly augmented first image is input to a corresponding first expert head to determine a supervised loss for each weakly augmented first image. Each strongly augmented first image is input to a corresponding second expert head to determine a diversity loss for each strongly augmented first image. The feature extractor is trained to minimize the supervised loss on weakly augmented first images and to minimize a multi-expert consensus loss on strongly augmented first images. Each first expert head is trained to minimize the supervised loss for each weakly augmented first image, and each second expert head is trained to minimize the diversity loss for each strongly augmented first image.

Multi-expert adversarial regularization for robust and data-efficient deep supervised learning

A computer vision (CV) training system, includes: a supervised learning system to estimate a supervision output from one or more input images according to a target CV application, and to determine a supervised loss according to the supervision output and a ground-truth of the supervision output; an unsupervised learning system to determine an unsupervised loss according to the supervision output and the one or more input images; a weakly supervised learning system to determine a weakly supervised loss according to the supervision output and a weak label corresponding to the one or more input images; and a joint optimizer to concurrently optimize the supervised loss, the unsupervised loss, and the weakly supervised loss.

System and method for deep machine learning for computer vision applications

A method and apparatus for variable rate compression with a conditional autoencoder is herein provided. According to one embodiment, a method for compression includes receiving a first image and a first scheme as inputs for an autoencoder network; determining a first Lagrange multiplier based on the first scheme; and using the first image and the first Lagrange multiplier as inputs, computing a second image from the autoencoder network. The autoencoder network is trained using a plurality of Lagrange multipliers and a second image as training inputs.

Method and apparatus for variable rate compression with a conditional autoencoder

A method and system for multi-frame contextual attention are provided. The method includes obtaining a reference frame to be processed, identifying context frames with respect to the reference frame, and producing a refined reference frame by processing the obtained reference frame based on the context frames.

System and method for multi-frame contextual attention for multi-frame image and video processing using deep neural networks

Apparatuses and methods of manufacturing same, systems, and methods for generating a convolutional neural network (CNN) are described. In one aspect, a minimal CNN having, e.g., three or more layers is trained. Cascade training may be performed on the trained CNN to insert one or more intermediate layers until a training error is less than a threshold. When cascade training is complete, cascade network trimming of the CNN output from the cascade training may be performed to improve computational efficiency. To further reduce network parameters, convolutional filters may be replaced with dilated convolutional filters with the same receptive field, followed by additional training/fine-tuning.

System and method for designing efficient super resolution deep convolutional neural networks by cascade network training, cascade network trimming, and dilated convolutions

Method and systems are provided for robust disparity estimation based on cost-volume attention. A method includes extracting first feature maps from left images captured by a first camera; extracting second feature maps from right images captured by a second camera; calculating a matching cost based on a comparison of the first and second feature maps to generate a cost volume; generating an attention-aware cost volume from the generated cost volume; and aggregating the attention-aware cost volume to generate an output disparity.

Growing community of inventors

San Diego, CA, United States of America

Mostafa El-Khamy