Zoom Video Communications, Inc.

Zoom Communications, Inc.

Bo Ling

Jing Wu

Yi Guo

Zhichu He

Yichen Zhang

Nitasha Walia

Rui Li

Abhishek Balaji

Juliana Park

Huixi Zhao

Anna Deng

Pei Li

Minxia Yang

Xingguo Zhu

Shiyan Zhang

Tianming Gu

Chichen Fu

Wenchong Lin

Lin Han

Jeffrey William Smith

Cynthia Eshiuan Lee

Hailei Sheng

Qiyong Liu

Zhaofeng Jia

Qiang Li

Yuhui Chen

Cheng Lun Hu

Ruizhen Wang

Jianpeng Wang

Yian Zhu

Gengdai Liu

Min Miao

Dewang Hou

Xiran Wang

Chong Lv

Yanjia Duan

Juntao Feng

Menglin Wang

In one example, a video encoder divides a superblock of a video frame into subblocks. The dividing includes dividing the superblock according to a first and second partitioning schemes to generate a first and second combination of subblocks, respectively. The number of non-zero residuals in each subblock in the first and second combinations of subblocks is less than a corresponding threshold set for a size of the subblock. The encoder determines the subblocks for the superblock as the first combination of subblocks based on a first cost value associated with the first combination of subblocks being lower than a second cost value associated with the second combination of subblocks. The encoder further encodes the superblock into a video bitstream representing the video by encoding the subblocks.

Reducing grid artifacts in video frames

Fast parameter search for chroma from luma (CFL) intra prediction in video encoding is provided. A video encoder determines a value of a parameter of a linear CFL predictor for a coding block of a frame of a video. The determining includes dividing a search range within a value range of the parameter into three portions by a starting value: the starting value and values on two sides of the starting value. The encoder determines cost values of parameter values from the three portions and select the value of the parameter that corresponds to the lowest cost value. The encoder further encodes the coding block of the frame into a bitstream representing the video using the CFL predictor with the selected value of the parameter.

Fast parameter search for chroma from luma intra prediction in video coding

Deriving in-loop filter parameters via training for video encoding is provided. A video encoder performs inter prediction for a frame in a set of frames of the video to generate prediction residuals for the frame. The inter prediction for the frame is performed based on a reconstructed frame in the set of frames filtered using an in-loop filter. The value of a parameter of the in-loop filter is determined by determining, for each candidate in-loop filter parameter value, a visual quality metric for a set of training frames in training video sequences filtered by the in-loop filter. The candidate in-loop filter parameter value that corresponds the highest visual quality metric can be selected as the value of the parameter of the in-loop filter. The video encoder further encodes the prediction residues of the frame and the parameter of the in-loop filter into a bitstream representing the video.

Deriving in-loop filter parameters for video coding

Methods and systems provide for applying a video effect to a video corresponding to a participant within a video communication session. The system displays a video for each of at least a subset of the participants and a user interface including a selectable video effects UI element. The system receives a selection by a participant of the video effects UI element. In response to receiving the selection, the system displays a variety of video effects options for modifying the appearance of the video and/or modifying a visual representation of the participant. The system then receives a selection by the participant of a video effects option, and further receives a subselection for customizing the amount of the video effect to be applied. The system then applies, in real time or substantially real time, the selected video effect in the selected amount to the video corresponding to the participant.

Applying video effects corresponding to each participant within a video communication session

Constructing a motion vector candidate list for inter-prediction in video coding is provided. A video encoder performs inter prediction for a frame of the video to generate prediction residuals. The inter prediction is performed using motion vectors (MVs) for blocks of the frame. To determine the motion vector of a block, the encoder determines an integer-valued MV for the block based on a reference frame and selects, based on the integer-valued MV, a subset of candidate MVs from a list of MVs including MVs of neighboring blocks. The encoder identifies a MV from the subset of candidate MVs for the block and searches in a neighborhood of the identified MV for a refined MV. The encoder calculates the prediction residuals for the block based on a reference block in the reference frame pointed by the refined MV and encodes the prediction residuals into a bitstream representing the video.

Constructing motion vector candidates list for video coding

Methods and systems relate to a method for providing concurrent sharing of screen elements by multiple users within a communication session. First, a system connects multiple participants of a communication session. The system receives, from a first participant, a request to initiate sharing of one or more screen elements from a user device associated with the first participant. In response to the request, the system captures and displays the one or more screen elements from the user device of the first participant to the communication interfaces of the other participants. The system then receives, from one or more additional participants of the communication session, additional requests to initiate sharing of one or more additional screen elements. In response, the system captures and displays the additional screen elements of the additional participants to the communication interfaces of the other participants, such that multiple screen elements from multiple users are displayed concurrently.

Concurrent screen sharing by multiple users within a communication session

Various embodiments of an apparatus, method(s), system(s) and computer program product(s) described herein are directed to a Viseme Engine. The Viseme Engine receives audio data associated with a user account. The Viseme Engine predicts at least one viseme that corresponds with a portion of phoneme audio data and identifies one or more facial expression parameters associated with the predicted viseme. The facial expression parameters being applicable to a face model. The Viseme Engine renders the predicted viseme according to the one or more facial expression parameters.

Speech driven talking face

Methods and systems provide for low lighting adjustments within a video communication session. First, the system receives video content within a video communication session of a video communication platform, the video content having multiple video frames. The system then receives or generates a lighting adjustment request including a lighting adjustment depth, then detects an amount of lighting in the video content. The system then modifies the video content to adjust the amount of lighting, wherein the amount of adjustment of lighting corresponds to the adjustment depth, and wherein adjusting the amount of lighting is performed in real time or substantially real time upon receiving the lighting adjustment request.

Dynamic low lighting adjustment within a video communication system

Applying video effects within a video communication session

Methods, systems, and apparatus, including computer programs encoded on computer storage media related to multi-stream video encoding for screen sharing a communications session. The system may determine an active pixel area and a remaining pixel area of a video region. of a video region. A first video stream of the active pixel area is generated at a first frame rate. A second video stream of the remaining pixel area is generated at second frame rate, where the second frame rate is a frame rate lower than the first frame rate. A client device may transmit the first video stream and the second video stream to a second client device.

Multiple video stream generation based on different pixel sets within an initial video stream

Methods, systems, and apparatus, including computer programs encoded on computer storage media related to video bandwidth optimization, and more particularly, to systems and methods for video resolution downscaling and upscaling a video communications platform. A first video stream comprising first multiple image frames of a first resolution is received. A modified video stream of a second resolution higher than the first resolution, is generated using a trained machine learning network. A user interface may display the modified video stream.

Video bandwidth optimization within a video communications platform

Methods and systems provide for video appearance adjustments within a video communication session. First, the system receives video content. The system then receives an appearance adjustment request comprising an adjustment depth, and detects imagery of a user within the video content. The system then detects a face region within the video content. The system segments the face region into a number of skin areas. For each of the plurality of skin areas, the system classifies the skin area as a smooth texture region or rough texture region. If the skin area is classified as a smooth texture region, the system modifies the imagery of the user in real time or substantially real time by applying a smoothing process to the skin area, where the amount of smoothing applied corresponds to the adjustment depth.

Providing video appearance adjustments within a video communication system

A best available video stream is determined for each of multiple conference participants within a conference room including multiple cameras based on scores determined for video streams obtained from the cameras. The scores are determined based on representations of the conference participants within the video streams, for example, based on percentages of conference participant faces visible within the video streams, directions of conference participant faces relative to the cameras, directions of eye gaze of the conference participants relative to the cameras, and/or degrees to which conference participant faces are obscured within the video streams. The best available video streams are output for rendering within separate user interface tiles of conferencing software.

Multi-camera video stream selection for in-person conference participants

An adaptive screen encoding method comprising: using a computer, creating and storing, in computer memory, a plurality of conditions for use by a server configured to determine which of picture coding type to select; detecting a current picture by a sender for a content type including textual content, graphical content, and natural image content; determining a percentage of static macroblocks corresponding to the current picture; selecting the picture coding type based on the content type, the plurality of conditions, and the percentage of static macroblocks, wherein the method is performed by one or more special-purpose computing devices.

Reference picture selection and coding type decision processing based on scene contents

Growing community of inventors

Saratoga, CA, United States of America

Bo Ling