Ualink non-transparent memory bridging for ai infrastructures comprising gpus, accelerators, and memory switches

Abstract

Modern datacenters require efficient mechanisms for memory resource sharing between accelerators and host processors to support AI/ML workloads, HPC applications, and distributed computing environments. Embodiments herein disclose systems incorporating RPUs that enable entities to access host memory through UALink-based interconnects. The processor utilizes a coherent interconnect coupling processing cores to memory controllers, with an MMU mapping virtual addresses to physical addresses within the processor's physical address space. The RPU performs hardware-accelerated physical address translations between UALink-associated addresses and the processor's physical address space, enabling entities to access memory via the UALink port, coherent interconnect, and memory controllers. Some embodiments support multiple RPUs with independent UALink ports serving entities with distinct physical address spaces, enabling dynamic resource allocation and memory pooling, which address memory disaggregation challenges for GenAI inference, LLM training, and next-generation datacenter architectures requiring flexible memory sharing across heterogeneous compute elements.