Weight initialization method and apparatus for stable learning of deep learning model using activation function

Abstract

Provided is an artificial neural network learning apparatus for deep learning. The apparatus includes an input unit configured to acquire an input data or a training data, a memory configured to store the input data, the training data, and a deep learning artificial neural network model, and a processor configured to perform computation based on the artificial neural network model, in which the processor sets the initial weight depending on the number of nodes belonging to a first layer and the number of nodes belonging to a second layer of the artificial neural network model, and determines the initial weight by compensation by multiplying a standard deviation (σ) by a square root of a reciprocal of a probability of a normal probability distribution for a remaining section except for a section in which an output value of the activation function converges to a specific value.