Discriminative training in machine learning

Abstract

Systems, methods, and apparatuses including computer program products for machine learning are provided. A method is provided that includes distributing a parameterized model to each worker of a hierarchy of workers, the parameterized model including a plurality of feature functions and corresponding model parameters, processing a portion of training data at each worker of the plurality of workers according to the parameterized model to calculate updates to model parameters, for each worker at a lowest level of the hierarchy of workers, sending the calculated updates to a next higher level worker, for each other worker in the hierarchy of workers, combining updates of the respective worker with updates received from one or more lower level workers, collecting all updates from the workers at a master to generate real updates to the model parameters, and generating an updated model using the real updates to the model parameters.