Apple Inc.

Patrick Miauton

Christopher M Garrido

Karthick Santhanam

Ming Jin

David L Biderman

Dazhong Zhang

Luciano Miguel Verger

Eric L Chien

Xiaosong Zhou

Yan Yang

Gurtej Singh G Chandok

Jae Woo Chang

Martin E Johnson

Joe S Abuan

Ajoy K Singh

Sean Anthony Ramprashad

Najeeb M Abdulrahiman

Berkat S Tung

Keith Walter Rauenbuehler

Srinivasan Vasudevan

Daniel K Boothe

Eric A Allamanche

Ian J Baird

Anil G Naik

Deepak Chitlur Lakshman

Peter D Callaway

Xiaoxiao Zheng

Hsien-Po Shiang

Peikang Song

Bradley F Patterson

Sudeng Hu

Austin W Shyu

Gurtej Singh Chandok

Joshua L Hare

Komei Harada

Bess C Chan

Brian Thomas Toombs

Kostyantyn Komarov

James A Forrest

Hsi-Jung Wu

Sean Boland Kelly

Jae Hoon Kim

Hang Yuan

Mitchell R Lerner

Thomas P Devanneaux

Tony S Verma

Charles C Hoyt

Vu H Chiem

Xing Wen

Darius A Satongar

Yichao Shen

Per Håkan Linus Persson

Wenhui Xu

Christopher Maury

Peter Shiang

Kartik P Sathappan

Erik D Hornberger

A computer system outputs audio content via one or more audio output devices. If the audio content includes information that enables spatialization of the audio content, the system outputs the audio content in a simulated three-dimensional environment, including, if the audio content corresponds to a first category of content, causing the one or more audio output devices to simulate production of the audio content in a first virtual space, and if the audio content corresponds to a second category of content, causing the one or more audio output devices to simulate production of the audio content in a second virtual space that has different simulated acoustic properties than simulated acoustic properties of the first virtual space.

Systems, methods, and graphical user interfaces for using spatialized audio during communication sessions

Systems and processes for providing textual representations for a communication session are provided. For example, at least one audio input is received at an electronic device, wherein each audio input of the at least one audio input is associated with a respective priority level. A priority level of an audio input detected at a microphone of the electronic device is determined, wherein a highest priority level among the determined priority level and each received priority level corresponding to the at least one audio input is identified. A textual representation of a respective audio input corresponding to the identified highest priority level is obtained, wherein the obtained textual representation is displayed on a display of the electronic device.

Providing textual representations for a communication session

A method and apparatus of a device that uses a video jitter buffer to buffer the video frames for a received video stream is described. The device uses the video jitter buffer to estimate the delay variability of the frames and pick a target delay that will help harmonize the playback with minimal loss and delay. This is achieved by estimating the delay spread, which is the result of performing statistical analysis of the measured lags of the video frames received. The video jitter buffer provides target delay recommendation and reference frame information to the video player in order to anchor the playback and schedule the frames presentation time.

Systems and methods of video jitter estimation

A method performed a local device that is communicatively coupled with several remote devices, the method includes: receiving, from each remote device with which the local device is engaged in a communication session, an input audio stream; receiving, for each remote device, a set parameters; determining, for each input audio stream, whether the input audio stream is to be 1) rendered individually or 2) rendered as a mix of input audio streams based on the set of parameters; for each input audio stream that is determined to be rendered individually, spatially rendering the input audio stream as an individual virtual sound source that contains only that input audio stream; and for input audio streams that are determined to be rendered as the mix of input audio streams, spatially rendering the mix of input audio streams as a single virtual sound source that contains the mix of input audio streams.

Spatial audio controller

A device implementing the subject technology may include at least one processor configured to transmit an allocation request requesting allocation of a group communication session with a plurality of devices and receive an allocation response in response to the allocation request, the allocation response including credential information for the device to use to join the group communication session. The at least one processor may be further configured to transmit an allocation bind request with the credential information to join the group communication session using the credential information and receive an allocation bind success response in response to the allocation bind request, the allocation bind success response indicating that the device has joined the group communication session. The at least one processor may be further configured to provide a join notification to the plurality of devices via an intermediary device to notify that the device has joined the group communication session.

Multi-device communication management

Aspects of the subject technology provide for transcription of audio content during a conferencing session, such as an audio conferencing session or a video conferencing session. The transcription can be generated by the device at which the audio input is received, and transmitted to a remote device at which the transcription is displayed. Video content can also be provided from the device that generates the transcription to the remote device that displays in the transcription. The transcription can be provided with time information corresponding to time information in the video content, for synchronized display of the transcription and the corresponding video content.

Audio transcription for electronic conferencing

A method performed a local device that is communicatively coupled with several remote devices, the method includes: receiving, from each remote device with which the local device is engaged in a communication session, an input audio stream; receiving, for each remote device, a set parameters; determining, for each input audio stream, whether the input audio stream is to be 1) rendered individually or 2) rendered as a mix of input audio streams based on the set of parameters; for each input audio stream that is determined to be rendered individually, spatial rendering the input audio stream as an individual virtual sound source that contains only that input audio stream; and for input audio streams that are determined to be rendered as the mix of input audio streams, spatial rendering the mix of input audio streams as a single virtual sound source that contains the mix of input audio streams.

A video playback buffer adjustment system is disclosed. The video playback buffer has a target size, measured in units of time of content, and includes a compressed frames queue and an uncompressed frames queue. The uncompressed frames queue has a minimum threshold queue size. If the size of the uncompressed frames queue falls below the minimum threshold size, then a next encoded frame in the compressed frames queue is removed from the compressed frames queue, decoded, and enqueued in the uncompressed frames queue. If a current size of the playback queue exceeds the target size plus &#x394; (a hysteresis value), then a playback frame rate is increased, and a presentation time of one or more frames in the uncompressed frames queue is adjusted according to the increased playback frame rate. If the playback queue size is less than the target size minus &#x394;, the playback speed can be reduced.

Video playback buffer adjustment

Techniques presented herein provide an improved relay user experience and improved management of scarce computing and network resources as the number of relay endpoints increases. A sourcing endpoint device may generate a media feed, such as video and/or audio feed, representing contribution from a conference participant. The sourcing endpoint device may generate a priority value for the media feed, and the priority value may be transmitted to other members of the relay along with the input feed. Priority values of the different relay participants may be used by other devices, for example, intermediate servers or receiving endpoint devices, to manage aspects of the relay. For example, a relay server may prune streams from select endpoint devices based on relative priority values received from those devices. Alternatively, receiving endpoint devices may alter presentation of received feeds based on their associated priority values.

Media feed prioritization for multi-party conferencing

Systems and methods for coding a video to be overlaid by annotations are devised. A motion compensated predictive coding is employed, wherein coding parameters of video pixel blocks are determined based on the pixel blocks' relation to the annotations. A decoder decodes the video and annotates it based on metadata, obtained from the coder or other sources, describing the annotations' appearance and rendering mode.

Efficient coding of video data in the presence of video annotations

A device implementing the subject technology may include at least one processor configured to receive an indication of a task being performed at an other device. The at least one processor is further configured to perform, in response to receipt of the indication, one or more preliminary operations in anticipation of receiving a handoff of the task. The at least one processor is further configured to receive, after performing the one or more preliminary operations, a notification that the task is being handed off. The at least one processor is further configured to receive the handoff of the task from the other device. The at least one processor is further configured to continue to perform the task based at least in part on the one or more preliminary operations that were performed prior to receiving the notification that the task is being handed off.

A device implementing a system for audio-video conferencing using multiple stream identifiers includes a processor configured to receive, from a sending device, indication of a first content stream and a first stream identifier, and indication of a second content stream and a second stream identifier associated. The first content stream and the second content stream correspond to different bit rates of streaming content. The processor is configured to receive, from a receiving device, a request to subscribe to the second content stream, the request including the second stream identifier, and receive, from the sending device, an indication that the second stream identifier has been associated with the first content stream. The processor is configured to forward, to the receiving device, the first content stream based on the request to subscribe to the second content stream and on the indication that the second stream identifier has been associated with the first content stream.

Multiway audio-video conferencing

A device implementing the subject technology may include at least one processor configured to receive a plurality of buffer reports from a plurality of other devices, each of the plurality of buffer reports indicating a current target buffer level for a respective one of the plurality of other devices. The at least one processor may be further configured to determine a group target buffer level based at least in part on the current target buffer levels of the plurality of other devices and a current target buffer level of the at least one buffer. The at least one processor may be further configured to transmit the group target buffer level to the plurality of other devices. The at least one processor may be further configured to manage an occupancy of the at least one buffer based at least in part on the group target buffer level.

Methods and apparatus to manage communication sessions to handover between a direct connection at a secondary wireless device and a relayed connection to the secondary wireless device via a primary wireless device. A connection manager of a secondary wireless device can trigger transfer of a communication session based on measurements of performance metrics for the communication session. Upon detection of performance degradation in a local connection or a backhaul connection or both, the connection manager of the secondary wireless device can determine proximity of and/or capabilities for connections of the primary wireless device and instigate transfer of the communication session between different connection types, such as between a direct connection and a relayed connection. The transfer of the communication session can occur without user intervention or in response to input from the user without interrupting or reestablishing the communication session.

Active communication session handover between direct and relayed connections

In scenes that have low lighting, it may be difficult for a personal electronic device to capture video images that meet a scene lighting criteria. In some low-lighting situations, the receiving party may be unable to sufficiently see the face of the party that they are videoconferencing with. Described herein are techniques for utilizing a device's display to illuminate the scene surrounding the device. Improved lighting may come from global display adjustments, local display adjustments that do not alter the display's contents (other than to change their brightness levels), and/or local display adjustments that alter the display's contents. In some embodiments, background elements on a device's display may be identified for brightening in a first attempt to meet the scene lighting criteria without negatively impacting foreground scene content. If such approaches are unsuccessful, the device may next replace one or more regions of displayed content with regions of brightened pixels.

Video capture in low-light conditions

Growing community of inventors

Redwood City, CA, United States of America

Patrick Miauton