Smule, Inc.

John Shimmin

Perry Raymond Cook

Jeannie Yang

Benjamin Hersh

Randal J Leistikow

Jon Moldover

Kevin Sung

Bona Kim

David Adam Steinwedel

Amanda Chaudhary

Dean Schaffer

Lance Jabr

Stefan Sullivan

Jeff D Smith

Yuning Woo

Michael Berger

Embodiments described herein relate generally to systems comprising a display device, a display device-coupled computing platform, a mobile device in communication with the computing platform, and a content server in which methods and techniques of capture and/or processing of audiovisual performances are described and, in particular, description of techniques suitable for use in connection with display device connected computing platforms for rendering vocal performance captured by a handheld computing device.

Wireless handheld audio capture device and multi-vocalist method for audiovisual media application

Latency on different devices (e.g., devices of differing brand, model, vintage, etc.) can vary significantly and tens of milliseconds can affect human perception of lagging and leading components of a performance. As a result, use of a uniform latency estimate across a wide variety of devices is unlikely to provide good results, and hand-estimating round-trip latency across a wide variety of devices is costly and would constantly need to be updated for new devices. Instead, a system has been developed for crowdsourcing latency estimates.

Crowd-sourced device latency estimation for synchronization of recordings in vocal capture applications

Digital signal processing and machine learning techniques can be employed in a vocal capture and performance social network to computationally generate vocal pitch tracks from a collection of vocal performances captured against a common temporal baseline such as a backing track or an original performance by a popularizing artist. In this way, crowd-sourced pitch tracks may be generated and distributed for use in subsequent karaoke-style vocal audio captures or other applications. Large numbers of performances of a song can be used to generate a pitch track. Computationally determined pitch trackings from individual audio signal encodings of the crowd-sourced vocal performance set are aggregated and processed as an observation sequence of a trained Hidden Markov Model (HMM) or other statistical model to produce an output pitch track.

Crowd-sourced technique for pitch track generation

Vocal audio of a user together with performance synchronized video is captured and coordinated with audiovisual contributions of other users to form composite duet-style or glee club-style or window-paned music video-style audiovisual performances. In some cases, the vocal performances of individual users are captured (together with performance synchronized video) on mobile devices, television-type display and/or set-top box equipment in the context of karaoke-style presentations of lyrics in correspondence with audible renderings of a backing track. Contributions of multiple vocalists are coordinated and mixed in a manner that selects for presentation, at any given time along a given performance timeline, performance synchronized video of one or more of the contributors. Selections are in accord with a visual progression that codes a sequence of visual layouts in correspondence with other coded aspects of a performance score such as pitch tracks, backing audio, lyrics, sections and/or vocal parts.

Automated generation of coordinated audiovisual work based on content captured from geographically distributed performers

Automated generation of coordinated audiovisual work based on content captured geographically distributed performers

Audiovisual media application platform with wireless handheld audiovisual input

Crowd sourced technique for pitch track generation

Display screen or portion thereof with animated graphical user interface

Display screen or portion thereof with graphical user interface

Growing community of inventors

San Francisco, CA, United States of America

John Shimmin