Google Inc.

Lars Henrik Ellner

Turaj Zakizadeh Shabestary

Niklas Enbom

Andrew John MacDonald

Methods and systems are provided for reusing audio streams in an audio mixing environment such that processing resources (e.g., CPU cycles) are not wasted on encoding the same audio streams multiple times (e.g., more than once). Instead, processing resources are conserved without comprising audio quality. The methods described herein are particularly useful when a participant transitions from using a shared encoder to using the participant's own encoder, and when the participant transitions from using the participant's own encoder to using a shared encoder.

Stream caching for audio mixers

Methods and systems are provided for reusing audio streams in an audio mixing environment such that processing resources are not wasted on encoding the same audio streams multiple times. Instead, processing resources are conserved without comprising audio quality. The methods described herein are particularly useful when a participant transitions from using a shared encoder to using the participant's own encoder, and when the participant transitions from using the participant's own encoder to using a shared encoder.

Introduction of one or more multicast engines to a video conferencing system permits an improvement of overall network bandwidth used for a video conference. Video streams are received at a multicast engine from each of at least two participants in the video conference. The multicast engine transmits only the first of the two video streams based on a determination that only the first video stream is a candidate for a conference video stream to be transmitted to at least some of a plurality of conference participants from the server station. The conference video stream is received at the multicast engine from the server station, and the stream is transmitted from the multicast engine to each of the participants in the video conference that are connected to the multicast engine.

Video multicast engine

Introduction of one or more multicast engines to a video conferencing system permits an improvement of overall network bandwidth used for a video conference. A video stream is received at a multicast engine connected to a participant of a plurality of participants in the conference where the video stream is supplied by the participant. The multicast engine selectively transmits the video stream to a server station based on determining whether the video stream is a candidate for a conference video stream to be transmitted to at least some of the plurality of participants. The conference video stream is received at the multicast engine from the server station, and the stream is transmitted from the multicast engine to any participants in the video conference that are connected to the multicast engine.

A method for video conferencing includes receiving, at one or more computers from at least some remote clients from a plurality of remote clients, information representing a plurality of media frames. The method also includes receiving, at the one or more computers from at least some of the plurality of remote clients, a plurality of video-based speech activity signals each associated with a respective media frame from the plurality of media frames. The method further includes selecting, at the one or more computers, at least some media frames from the plurality of media frames based on the video-based speech activity signals, decoding the selected media frames, generating a mixed media stream by combining the decoded media frames, transmitting, from the one or more computers to at least some remote clients from the plurality of remote clients, the mixed media stream.

Video mixing using video speech detection

Video conferencing with reduced bandwidth usage is disclosed. One method comprises receiving an audio stream at a server station connected to a plurality of participants in the video conference, at least one participant being a client station connected to the server station by a network and supplying the audio stream, the server station including a first memory and processor, and the client station including a second memory and processor. A mixing algorithm is performed that determines which video streams of the participants to mix to form a conference video stream for display by at least one of the participants. A video stream from the client station is selectively transmitted to the server station responsive to the mixing algorithm, and the server station transmits the conference video stream and/or the streams to be mixed to the participant(s).

Growing community of inventors

San Francisco, CA, United States of America

Lars Henrik Ellner