Maryna Diakonova

Title: Maryna Diakonova: Innovator in Augmented Reality Captioning

Introduction

Maryna Diakonova is a prominent inventor based in Playa Vista, CA (US). She has made significant contributions to the field of augmented reality, particularly in the area of automated captioning for AR effects in videos. Her innovative techniques are paving the way for enhanced user experiences in digital media.

Latest Patents

Maryna holds a patent for "Automated captioning of augmented reality effects in videos." This patent describes techniques for generating captions for augmented reality (AR) effects in videos using an adapted multimodal large language model (MLLM). The method involves sampling frames from both base and AR-applied videos, combining them into concatenated frames, and processing them with a hybrid vision encoder. Visual tokens are projected into a language model token space, reshaped, downsampled, and interleaved with text tokens. A fine-tuned large language model processes this input sequence to generate AR effect captions. Additionally, optical character recognition extracts text from AR frames, which is then combined with generated captions and metadata to produce merged captions and content tags. This approach enables accurate descriptions of temporal AR effects and facilitates downstream applications like search and ranking.

Career Highlights

Maryna is currently employed at Snap Inc., where she continues to innovate in the realm of augmented reality. Her work is instrumental in enhancing the functionality and accessibility of AR technologies.

Collaborations

Maryna has collaborated with talented individuals such as Kwot Sin Lee and Varnith Uttam Chordia, contributing to a dynamic and innovative work environment.

Conclusion

Maryna Diakonova's contributions to augmented reality captioning represent a significant advancement in the field. Her innovative techniques are set to transform how users interact with AR content, making it more accessible and engaging.