Snap Inc.

Tsai-Shien Chen

Hsin-Ying Lee

Yuwei Fang

Automatic captioning pipelines and methods for automatically annotating video data with subtitles, which can be obtained using automatic speech recognition (ASR). An automatic captioning pipeline with inputs of multimodal data scales up the dataset of high-quality video-caption pairs. The automatic captioning pipeline generates video-caption pairs by establishing and using a large video-language dataset along with an automatic captioning approach leveraging multimodal inputs, such as textual video description, subtitles, and individual video frames.

Growing community of inventors

Merced, CA, United States of America

Tsai-Shien Chen