Polycom, Inc.

Plantronics, Inc.

Picturetel Corporation

Hewlett-packard Development Company, L.p.

Stephen C Botzko

Gopal Paripally

Gary Joseph Sullivan

Jeffrey G Bernstein

John Paul Spearman

Brian A Howell

Elizabeth M Comstock

William L Kruger

Robert V Seiler

Peter L Chu

Jian David Wang

Richard Flott, Jr

David N Hein

David M Franklin

Michael Horowitz

Yibo Liu

Yong Yan

Jinwei Feng

Moshe Elbaz

Ilan Yona

David Jordan Lindbergh

Charles M Crisler

David A Bryan

John Sievers

Richard E Flott

David Hein

A video conference call control system is provided with an audio mixer to receive an audio speech signal at one or more network devices connected to the video conference call; to generate first and second talker identification values from the audio speech signal; to compare the first and second talker identification values during a timing comparison window to determine if the first and second talker identification values are identical and received simultaneously or substantially contemporaneously; and to selectively mute one or more audio devices at the network device(s) if the first and second talker identification values are identical and received simultaneously or substantially contemporaneously so that microphone input signals to the audio mixer are muted to prevent 'playback' echo effects and so that microphone input and audio loudspeaker output signals to the audio mixer are muted to prevent “double join” echo effects.

Speaker identification-based echo detection and solution

Determining the angle of sound relative to the centerline of a microphone array. The angle of the centerline of a camera field-of-view (FoV) and the angle of the camera FoV is determined. Knowing the angle from the centerline of the microphone array of the particular sound and then the angle of the centerline of the camera FoV and angles of the camera FoV allows a determination if the sound is inside the FoV of the camera. If so, the microphones are unmuted. If not, the microphones are muted. As the camera zooms or pans, the changes in camera FoV and centerline angle are computed and used with the sound angle, so that the muting and unmuting occurs automatically as the camera zoom and pan angle change.

Camera-view acoustic fence

The development of a region of interest (ROI) video frame that includes only ROIs of interest and not other elements and providing the ROI video frames in a single video stream simplifies the development of gallery view continuous presence displays. ROI position and size information metadata can be provided or subpicture concepts of the particular codec can be used to separate the ROIs in the ROI video frame. Metadata can provide perspective/distortion correction values, speaker status and any other information desired about the participant or other ROI, such as name. Only a single encoder and a single decoder is needed, simplifying both transmitting and receiving endpoints. Only a single video stream is needed, reducing bandwidth requirements. As each participant can be individually isolated, the participants can be provided in similar sizes and laid out as desired in a continuous presence display that is pleasing to view.

Representation and compression of gallery view for video conferencing

Utilizing two LTR frames for improved error recovery. By using two LTR frames, much better performance is achieved in terms of error recovery as the likelihood of the decoder having one of the two LTR frames is very high. When the decoder determines a frame is lost, the decoder provides a fast update request (FUR). The FUR includes a listing of the LTR frames present at the decoder. With this indication of the LTR frames present at the decoder, the encoder utilizes one of the LTR frames, preferably the most recent, to use as a reference to send the next frame as a P frame. The P frame is sent with an indication of the LTR frame used as reference. The use of two LTR frames and the feedback of LTR frames present at the decoder allows the minimization of the use of I frames for error recovery.

Long-term reference for error recovery in video conferencing system

A portable immersive telepresence conferencing system which can capture audio data and video data and communicate at least some of the captured data to at least one remote endpoint via a network interface, receive image data and audio data from at least one remote source via the network interface and render a depiction of at least one object, which is substantially the same size of the actual object being shown.

Immersive telepresence anywhere

Disclosed herein is a system and method for reconstruction video information lost as a result of transmission errors. The system and method have four aspects, including: (1) changing the bit and/or packet rate; (2) inserting redundant information into the video bitstream; (3) providing automatic refresh of certain regions of the video on a periodic basis; and (4) interleaving coded macroblocks into diversity groups for transmission to spatially spread the effect of lost packets. The image reconstruction may then take advantage of these three aspects to provide an enhanced result in the presence of transmission losses.

Video error concealment method

A conferencing peripheral for use with a mobile device or laptop or desktop computer can include one or more projectors, cameras, microphones, and speakers. Such a device can work with the mobile device to provide a higher quality conferencing experience than has been provided to date by projecting a substantially full size, high resolution, image of conference participants onto a screen or wall and by providing microphones, speakers, and sufficient audio processing to provide high fidelity audio as part of the conferencing experience. The peripheral may be configured to use the voice and/or data network of the mobile device or may include its own internal network interface.

A conferencing peripheral for use with a mobile device or laptop or desktop computer can include one or more of projectors, cameras, microphones, and speakers. Such a device can work with the mobile device to provide a higher quality conferencing experience than has been provided to date by projecting a substantially full size, high resolution, image of conference participants onto a screen or wall and by providing microphones, speakers, and sufficient audio processing to provide high fidelity audio as part of the conferencing experience. The peripheral may be configured to use the voice and/or data network of the mobile device or may include its own internal network interface.

Use of a scalable audio codec to implement distributed mixing and/or sender bit rate regulation in a multipoint conference is disclosed. The scalable audio codec allows the audio signal from each endpoint to be split into one or more frequency bands and for the transform coefficients within such bands to be prioritized such that usable audio may be decoded from a subset of the entire signal. The subset may be created by omitting certain frequency bands and/or by omitting certain coefficients within the frequency bands. By providing various rules for each endpoint in a conference, the endpoint can determine the importance of its signal to the conference and can select an appropriate bit rate, thereby conserving bandwidth and/or processing power throughout the conference.

Scalable audio in a multi-point environment

Devices and techniques for overcoming lost packets and avoiding congestion when conducting media conferences over packet switched networks are described herein. To avoid the problem of lost packets, redundant information is inserted into the media stream that permits the receiver to reconstruct any lost packets from the redundant information. Congestion avoidance techniques include adjusting the bitrate of the media stream to find the highest bitrate that can be supported without packet loss due to congestion. When increasing the bitrate to a higher rate, the additional bits can come from the redundant information used for lost packet recovery so that any lost packets caused by network congestion will not adversely affect the bitstream.

System and method for lost packet recovery with congestion avoidance

An apparatus and method for digital video encoding is disclosed. The disclosed system provides for a way of improving video quality for a given video coding system design.

Method and apparatus for improving the average image refresh rate in a compressed video bitstream

According to the principles of the invention, there is provided a system, apparatus, and method for managing media in a multimedia conferencing system according to media roles. Each media stream may be explicitly labeled with a role that describes the function or purpose of the stream, such as 'people' or 'content.' The labels may be hierarchical, and may include layers for media type, additional media source description, and the like, e.g., &#x201c;people/presenter&#x201d; or &#x201c;people/presenter/video/&#x201d; A policy manager is provided for managing roles, such that the multimedia conference may be more effectively presented to participants. A token management system may also be provided so that control of the multimedia conference roles can be transferred during the conference.

Media role management in a video conferencing network

According to the principles of the invention, there is provided a system, apparatus, and method for managing media in a multimedia conferencing system according to media roles. Each media stream may be explicitly labeled with a role that describes the function or purpose of the stream, such as 'people' or 'content.' The labels may be hierarchical, and may include layers for media type, additional media source description, and the like, e.g., &#x201c;people/presenter&#x201d; or &#x201c;people/presenter/video.&#x201d; A policy manager is provided for managing roles, such that the multimedia conference may be more effectively presented to participants. A token management system may also be provided so that control of the multimedia conference roles can be transferred during the conference.

Growing community of inventors

Reading, MA, United States of America

Stephen C Botzko