Multi-variable optimization for routing requests to language models

Abstract

Systems, methods, and devices that relate to routing requests to large language models (LLMs) are disclosed. In one example aspect, the system receives session-specific data elements in response to a request to generate an output using LLMs. The system determines a hierarchy of operational constraints including privacy protocols and performance requirements. Weights for a multi-variable optimization are dynamically updated using the session-specific data elements. The system executes the multi-variable optimization across candidate LLMs that satisfy privacy constraints and optimize performance constraints. Based on the optimization, at least one candidate LLM is selected and the request is routed to it. In response to performance feedback, the system automatically selects a different LLM to improve one constraint, resulting in degradation of another constraint.