Deep learning model training system

Abstract

A computer trains a neural network. A neural network is executed with a weight vector to compute a gradient vector using a batch of observation vectors. Eigenvalues are computed from a Hessian approximation matrix, a regularization parameter value is computed using the gradient vector, the eigenvalues, and a step-size value, a search direction vector is computed using the eigenvalues, the gradient vector, the Hessian approximation matrix, and the regularization parameter value, a reduction ratio value is computed, an updated weight vector is computed from the weight vector, a learning rate value, and the search direction vector or the gradient vector based on the computed reduction ratio value, and an updated Hessian approximation matrix is computed from the Hessian approximation matrix, the predefined learning rate value, and the search direction vector or the gradient vector based on the reduction ratio value. The step-size value is updated using the search direction vector.