Gyrfalcon Technology Inc.

Baohua Sun

Lin Yang

Patrick Dong

Jason Zeng Dong

Charles Jin Young

Yequn Zhang

Yongxiong Ren

Wenhan Zhang

Qi Dong

Michael Lin

Xiaochun Li

Bin Yang

Hao Sha

Dan Bin Liu

Catherine Chi

Yi Fan

A pixel feature vector extraction system for extracting multi-scale features contains a cellular neural networks (CNN) based integrated circuit (IC) for extracting pixel feature vector out of input imagery data by performing convolution operations using pre-trained filter coefficients of ordered convolutional layers in a deep learning model. The ordered convolutional layers are organized in a number of groups with each group followed by a pooling layer. Each group is configured for a different size of feature map. Pixel feature vector contains a combination of feature maps from at least two groups, for example, concatenation of the feature maps. The first group of the at least two groups contains the largest size of the feature maps amongst all of the at least two groups. Feature maps of the remaining of the at least two groups are modified to match the size of the feature map of the first group.

Apparatus and methods of obtaining multi-scale feature vector using CNN based integrated circuits

A system for training an artificial intelligence (AI) model for an AI chip to implement an AI task may include an AI training unit to train weights of an AI model in floating point, a convolution quantization unit for quantizing the trained weights to a number of quantization levels, and an activation quantization unit for updating the weights of the AI model so that output of the AI model based at least on the updated weights are within a range of activation layers of the AI chip. The updated weights may be stored in fixed point and uploadable to the AI chip. The various units may be configured to account for the hardware constraints in the AI chip to minimize performance degradation when the trained weights are uploaded to the AI chip and expedite training convergence. Forward propagation and backward propagation may be combined in training the AI model.

Using quantization in training an artificial intelligence model in a semiconductor solution

A system may include multiple client devices and a processing device communicatively coupled to the client devices. Each client device includes an artificial intelligence (AI) chip and is configured to generate an AI model. The processing device may be configured to (i) receive a respective AI model and an associated performance value of the respective AI model from each of the plurality of client devices; (ii) determine an optimal AI model based on the performance values associated with the respective AI models from the plurality of client devices; and (iii) determine a global AI model based on the optimal AI model. The system may load the global AI model into an AI chip of a client device to cause the client device to perform an AI task based on the global AI model in the AI chip. The AI model may include a convolutional neural network.

Systems and methods for determining an artificial intelligence model in a communication system

A device for obtaining a local optimal AI model may include an artificial intelligence (AI) chip and a processing device configured to receive a first initial AI model from the host device. The device may load the initial AI model into the AI chip to determine a performance value of the AI model based on a dataset, and determine a probability that a current AI model should be replaced by the initial AI model. The device may determine, based on the probability, whether to replace the current AI model with the initial AI model. If it is determined that the current AI model be replaced, the device may replace the current AI model with the initial AI model. The device may repeat the above processes and obtain a final current AI model. The device may transmit the final current AI model to the host device.

P feature encoding values are obtained for each of the Q frames in a video clip by image transformations of each frame along with performing computations of a specific succession of convolution and pooling layers of a CNN based deep learning model followed with operations of a nested invariance pooling layer. Each feature encoding value is then converted from real number to a corresponding integer value within a range designated for color display intensity according to a quantization scheme. A 2-D graphical symbol that contains N&#xd7;N pixels is formed by placing respective color display intensities into the N&#xd7;N pixels according to a data arrangement pattern for representing all frames of the video clip in form of P&#xd7;Q feature encoding values, such that the 2-D graphical symbol possesses a semantic meaning of the video clip that can be recognized via image classification task using another trained CNN based deep learning model.

2-D graphical symbols for representing semantic meaning of a video clip

A system may include a decentralized communication network and multiple processing devices on the network. Each processing device may have an artificial intelligence (AI) chip, the device may be configured to generate an AI model, determine the performance value of the AI model on the AI chip, receive a chain from the network where the chain contains a performance measure. If the performance value of the AI model is better than the performance measure, then the processing device may broadcast the AI model to the network for verification. If the AI model is verified by the network, the device may update the chain with the performance value so that the chain can be shared by the multiple processing devices on the network. Any processing device on the network may also verify an AI model broadcasted by any other device. Methods for generating the AI model are also provided.

System and method for determining an artificial intelligence model in a decentralized network

An image and the maximum number of tokens for a to-be-created image caption are received in a computing system. Font size of graphical image of the token is calculated from the maximum number of tokens and the dimension of desired input image for prediction-style image classification technique. Desired input image is divided into first and second portions. A 2-D symbol is formed by placing a resized image derived from the received image with substantially similar contents in the first portion and by initializing the second portion with blank images. Next token of the image caption is predicted by classifying the 2-D symbol using the prediction-style image classification technique. 2-D symbol is modified by appending the graphical image of just-predicted token to the existing image caption in the second portion, if termination condition for image caption creation is false. Next token is repeatedly predicted until termination condition becomes true.

Artificial intelligence based image caption creation systems and methods thereof

An ensemble learning based image classification system contains multiple cellular neural networks (CNN) based integrated circuits (ICs) operatively coupling together as a set of base learners of an ensemble for an image classification task. Each CNN based IC is configured with at least one distinct deep learning model in form of filter coefficients. The ensemble learning based image classification system further contains a controller configured as a meta learner of the ensemble and a memory based data buffer for holding various data used in the ensemble by the controller and the CNN based ICs. Various data may include input imagery data to be classified. Various data may also include extracted feature vectors or image classification outputs out of the set of base learners. The extracted feature vectors or image classification outputs are then used by the meta learner to further perform the image classification task.

Ensemble learning based image classification systems

Two-dimensional symbols with each containing multiple ideograms for facilitating machine learning are disclosed. Two-dimensional symbol comprises a matrix of N&#xd7;N pixels of data representing a 'super-character'. The matrix is divided into M&#xd7;M sub-matrices with each of the sub-matrices containing (N/M)&#xd7;(N/M) pixels. N and M are positive integers or whole numbers, and N is preferably a multiple of M. Each of the sub-matrices represents one ideogram defined in an ideogram collection set. 'Super-character' represents at least one meaning each formed with a specific combination of a plurality of ideograms. Ideogram collection set includes, but is not limited to, pictograms, logosyllabic characters, Japanese characters, Korean characters, punctuation marks, numerals, special characters. Logosyllabic characters may contain one or more of Chinese characters, Japanese characters, Korean characters. Features of each ideogram can be represented by more than one layer of two-dimensional symbol.

Two-dimensional symbol for facilitating machine learning of combined meaning of multiple ideograms contained therein

A local processing device contains a bus, an input interface and at least one cellular neural networks (CNN) based integrated circuit (IC). The input interface is receiving a 2-D symbol representing a Chinese poetry or verse. The 2-D symbol is a matrix of N&#xd7;N pixels of K-bit data that contains a 'super-character'. The matrix is divided into M&#xd7;M sub-matrices each containing (N/M)&#xd7;(N/M) pixels. Each of the sub-matrices represents an ideogram. K, N and M are positive integers, and N is a multiple of M. CNN based IC is configured for understanding semantic meaning of the Chinese poetry or verse within the 'super-character' contained in the 2-D symbol. The ideogram is created by embedded fonts of all of the characters contained in a corresponding phrase of the Chinese poetry or verse or is a pictogram representing artistic conception of each sentence of the Chinese poetry or verse.

Deep learning device for local processing classical chinese poetry and verse

Image classification system contains a CNN based IC configured for extracting features out of input data by performing convolution operations using filter coefficients of ordered convolutional layers and a classifier IC configured for classifying the input data using reduced set of the extracted features based on a light-weight classifier. Light-weight classifier is derived by: training filter coefficients of the ordered convolutional layers using a dataset containing N labeled data, the trained filter coefficients are for the CNN based IC; outputting respective extracted features of the N labeled data after performing convolution operations of ordered convolutional layers using the trained filter coefficients, each labeled data contains X features; creating the reduced set of the extracted features by eliminating those of the X features that contain zeros in at least M of the N labeled data; and adjusting M until the light-weight classifier achieves satisfactory results using the reduced set.

Image classification systems based on CNN based IC and light-weight classifier

A deep learning object detection and recognition system contains a number of cellular neural networks (CNN) based integrated circuits (ICs) operatively coupling together via the network bus. The system is configured for detecting and then recognizing one or more objects out of a two-dimensional (2-D) imagery data. The 2-D imagery data is divided into N set of distinct sub-regions in accordance with respective N partition schemes. CNN based ICs are dynamically allocated for extracting features out of each sub-region for detecting and then recognizing an object potentially contained therein. Any two of the N sets of sub-regions overlap each other. N is a positive integer. Object detection is achieved with a two-category classification using a deep learning model based on approximated fully-connected layers, while object recognition is performed using a local database storing feature vectors of known objects.

Object detection and recognition apparatus based on CNN based integrated circuits

Multiple 3&#xd7;3 convolutional filter kernels are used for approximating operations of fully-connected (FC) layers. Image classification task is entirely performed within a CNN based integrated circuit. Output at the end of ordered convolutional layers contains P feature maps with F&#xd7;F pixels of data per feature map. 3&#xd7;3 filter kernels comprises L layers with each organized in an array of R&#xd7;Q of 3&#xd7;3 filter kernels, Q and R are respective numbers of input and output feature maps of a particular layer of the L layers. Each input feature map of the particular layer comprises F&#xd7;F pixels of data with one-pixel padding added around its perimeter. Each output feature map of the particular layer comprises (F&#x2212;2)&#xd7;(F&#x2212;2) pixels of useful data. Output of the last layer of the L layers contains Z classes. L equals to (F&#x2212;1)/2 if F is an odd number. P, F, Q, R and Z are positive integers.

Growing community of inventors

Fremont, CA, United States of America

Baohua Sun