Systems and methods for multi-user video communication with engagement detection and adjustable fidelity

Abstract

In one embodiment, a method includes maintaining a video communication between client devices with each being associated with a respective video stream, which is associated with a respective match scale measured based on a height of frames of the video stream and a depth of subjects within the frames, determining a respective scaling factor and cropping for each video stream, wherein the respective scaling factor is calculated based on the match scale associated with the corresponding video stream and a target match scale determined from the match scales associated with all video streams, and wherein the respective cropping is determined based on a uniformity of positions of the subjects within the frames across all video streams, generating a merged video stream from each video stream based on the respective scaling factor and cropping, and sending instructions for presenting the merged video stream to one or more of the client devices.