Secure multi-model hosting and experimentation in a cloud provider network

Abstract

Techniques for secure multi-model hosting and experimentation in a cloud provider network are described. A request to perform an action involving one of multiple machine learning (ML) models is received at a cloud provider network. A user associated with the request can be consistently mapped to a particular experience from multiple experiences of an experiment. Based on this experience, a request to use a ML model can be sent by a data plane component that includes a path value associated with the experience. The request can be sent from the data plane component to a private connectivity endpoint, which securely provides the request into another virtual private cloud including the ML models without the request transiting the public internet. The request can be routed via path-based routing and the included path value to a ML model providing the experience.