Method, electronic device, and computer program product for managing inference process

Abstract

Implementations of the present disclosure relate to a method, an electronic device, and a computer program product for managing an inference process. Here, the inference process is implemented based on a machine learning model. A method includes: determining, based on a computational graph defining the machine learning model, dependency relationships between a set of functions for implementing the inference process; acquiring, in at least one edge device located in an edge computing network, a set of computing units available to execute the inference process; selecting at least one computing unit for executing the set of functions from the set of computing units; and causing the at least one computing unit to execute the set of functions based on the dependency relationships. With example implementations of the present disclosure, the inference process is implemented by making use of a variety of computing units in the edge computing network, thereby improving performance.