University of Central Florida Research Foundation Inc

Chen Chen

Mubarak Shah

Umar Khalid

Ishan Rajendrakumar Dave

Nazmul Karim

Nazanin Rahnavard

The invention provides a method for adapting a text-to-image (T2I) diffusion model for video editing by using spectral decomposition to achieve controlled spectral shifts in the model's weights. This adaptation involves maintaining constant singular vectors while selectively adjusting singular values in response to a text prompt. A spectral shift regularizer constrains adjustments, particularly limiting changes to larger singular values to ensure minimal deviation from the original model's structure. This approach allows efficient, prompt-driven video editing by modifying specific elements according to the prompt while preserving the original video context. By focusing on selective spectral adjustments, the method reduces adaptation time and computational demands, making it suitable for real-time and resource-sensitive applications, such as dynamic video editing for streaming services.

Image diffusion framework for text-guided video editing

This invention relates to a self-supervised privacy preservation action recognition system leveraging a learnable transformation anonymization function. The system is designed to process videos by removing spatial cues to protect privacy while retaining critical information for action recognition. The anonymization function, based on an encoder-decoder model, undergoes iterative training on a dataset to optimize the balance between obscuring privacy-sensitive information and preserving capability of recognizing actions. This training involves freezing and adjusting the weights of an action recognition branch and a self-supervised privacy removal branch to refine the model's effectiveness. The outcome is an anonymized video with minimized privacy information leakage, suitable for action analysis without privacy labels. Additional functionalities include frame sampling for enhanced privacy evaluation, data augmentation for model robustness, and a minimax optimization protocol ensuring optimal action recognition accuracy alongside privacy preservation.

Growing community of inventors

Orlando, FL, United States of America

Chen Chen