Sara Jane Mcallister

Title: Sara Jane Mcallister: Innovator in Generative Model Serving

Introduction

Sara Jane Mcallister is a prominent inventor based in Pittsburgh, PA (US). She has made significant contributions to the field of generative models, particularly in improving performance and fault tolerance in model serving. Her innovative approach has garnered attention in the tech community.

Latest Patents

Sara holds a patent titled "KV-cache streaming for improved performance and fault tolerance in generative model serving." This patent describes a method of serving a generative transformer model that includes determining a batch size for processing inference requests. It involves allocating at least one prompt pipeline and one token pipeline to the generative transformer model to efficiently handle these requests. The design of the pipelines is based on various factors, including batch size, average prompt length, cache requirements, and memory footprint of model weights. The method utilizes cache streaming to enhance the generation of tokens through gather-copy operations performed using compute kernels.

Career Highlights

Sara is currently employed at Microsoft Technology Licensing, LLC, where she continues to innovate and develop cutting-edge technologies. Her work focuses on enhancing the capabilities of generative models, which are increasingly important in various applications, including artificial intelligence and machine learning.

Collaborations

Sara has collaborated with notable colleagues such as Amar Phanishayee and Jakub Michał Tarnawski. These partnerships have contributed to her success and the advancement of her projects.

Conclusion

Sara Jane Mcallister is a trailblazer in the field of generative model serving, with her patent showcasing her innovative spirit and technical expertise. Her contributions are paving the way for future advancements in this exciting area of technology.