Facebook, Inc.

Parama Jyothi Reddappagari

Naizhi Li

Christopher Vander Mey

Alex Narest

Chi Wang Ho

Cameron James Pickett

Tomi Yiu

Shuangtiao Huang

Alexander Narest

Techniques for providing multi-modal multi-party calling include receiving a join request at a multiway server (MWS) from a first client, the join request identifying a second client; sending a call invitation to the second client from the MWS; receiving a connection from the second client to the MWS; receiving a first media status from one of the first client or the second client while the first client and the second client are in a peer-to-peer mode; and forwarding the first media status to the other of the first client or the second client. Other embodiments are described and claimed.

Techniques for providing multi-modal multi-party calling

Various of the disclosed embodiments enable managing and augmenting 'comfort noise' during a network call, such as a Voice Over Internet Protocol (VOIP) connection. Particularly, traditional systems typically send machine-generated comfort noise, or a command to generate comfort noise at the recipient, on a channel separate from the conversation content. Some embodiments reduce this overhead by embedding the comfort noise in the media stream. In other embodiments, audio encoding is stopped at the source when the speaker falls silent and the recipient, after detecting the cessation, will generate white noise at its end. These approaches may be used in conjunction with a determination of the available bandwidth and channel parameters.

Advanced comfort noise techniques

Various of the disclosed embodiments reduce the impact of RTCP overhead by including RTCP information in the media packets themselves. The values in the RTCP headers may be selected based on the context and organized in a unique format for transport in the media packets. For example, RTT, packet loss, and bandwidth estimates may dictate when and how RTCP data is moved into the media packet. An interface may be provided for extracting the data so that clients may easily integrate the embodiments with existing RTCP-based systems. Inclusion of the RTCP information in the media packet may increase the media packet size, which may be anticipated in bandwidth assessments and accommodations.

Embedded RTCP packets

In various embodiments, a corpus of codecs may be correlated with different, partially overlapping ranges of transmission characteristics. As channel conditions degrade or improve, the system may select a new codec with which to continue the connection based upon the corresponding overlapping range. Codecs may not be switched immediately when the transmission characteristics enter overlapping ranges, to avoid degrading the user's experience. If the characteristics remain in the overlap, or manifest a likely progression toward another region, then the transition may be effected.

Seamless codec switching

Various of the disclosed embodiments improve encoding during a network call, such as a Voice Over Internet Protocol (VOIP) connection, by adjusting the size of a data communications packet ('packet'). Particularly, given a corpus of codecs with which to encode data, the embodiments may identify a packet size based upon a common multiple of each codec's minimum raw data size. The packet size may be selected to accommodate the inclusion of data encoded in each codec format, as well as error correction code data, and codec transition commands. The packet size may be tailored to trade off measured latency and data efficiency.

Codec with variable packet size

Various of the disclosed embodiments present systems and methods for improving improve audio and video quality in a Voice Over Internet Protocol (VOIP) connection including that includes both audio and video. Particularly, different audio and video codecs may be used and parameters assigned based upon the context in which the communication occurs. For example, audio quality may take precedence to video quality when discussing a matter in a chatroom. Conversely, video quality may take precedence to audio quality when playing a collaborative video game. VP9 may be used to encode video while a combination of ISAC and SPEEX may be used to encode audio. Bandwidth determinations for each channel may also influence the respective codec selections.

Decoupled audio and video codecs

Various of the disclosed embodiments improve the initial codec selection in a Voice Over Internet Protocol (VOIP) connection. Particularly, rather than select an initial codec for the connection arbitrarily or based on data measured during the connection, embodiments analyze attributes of data exchanged prior to connection establishment to identify the appropriate initial codec. Attributes of the offer message transmission and acknowledgement may be used to infer channel quality. Signal strength, the existence of a WiFi connection, previous codecs used, etc., may also be taken into consideration. Latency measurements may be used as a proxy for measuring available bandwidth. Based on these factors, a codec having appropriate attributes may be selected. Traditional rate shaping methods may be applied subsequent to the initial codec selection.

Growing community of inventors

Redmond, WA, United States of America

Parama Jyothi Reddappagari