Verbit Software Ltd.

Yaakov Kobi Ben Tsvi

Eric Ariel Shellef

Tom Livne

Iris Getz

Elisha Yehuda Rosensweig

Roman Himmelreich

Eli Asor

Elad Shtilerman

When transcribing audio recordings, such as legal depositions, phrases may be repeated throughout the recordings, but these repeated phrases get transcribed incorrectly by an automatic speech recognition (ASR) system. In order to assist a transcriber to correctly resolve such phrases, some embodiments described herein involve a computer that receives an audio recording that includes speech, generates a transcription of the audio recording utilizing an ASR system, and clusters segments of the audio recording into clusters of similar utterances. The computer provides a transcriber with certain segments of the audio recording, which include similar utterances belonging to a certain cluster, along with transcriptions of the certain segments. The computer receives from the transcriber: an indication of which of the certain segments include repetitions of a phrase, and a correct transcription of the phrase. The computer then updates the transcription of the audio recording based on the indication and the correct transcription.

Human resolution of repeated phrases in a hybrid transcription system

Knowing what accent is spoken can assist automatic speech recondition (ASR) systems to more accurately transcribe audio. In one embodiment, a system includes a frontend server configured to transmit, to a backend server, an audio recording that includes speech of one or more people in a room over a period spanning at least two hours. At sonic time during the first hour of the period, the backend server provides a transcriber with a certain segment of the audio recording, and receives, from the transcriber, after the transcriber listened to a certain segment, an indication indicative of an accent of a person who spoke in the certain segment. The backend server then provides the indication to an ASR system to be utilized to generate a transcription of an additional portion of the audio recording, which was recorded after the first twenty minutes of the period.

Human-based accent detection to assist rapid transcription with automatic speech recognition

Being able to rapidly and accurately transcribe long audio recordings, such as same-day transcription of multi-hour legal depositions, is a challenging task. Hybrid transcription, which involves automatic speech recognition (ASR) systems generating initial transcriptions that are then reviewed by human transcribers, can be used to tackle this challenge. However, hybrid transcription may be stymied when the transcribers cannot resolve certain issues in the ASR-generated transcriptions. This disclosure describes rapid resolution of transcription-related inquiries of transcribers. In one embodiment, a computer receives an audio recording that includes speech of multiple people in a room and generates transcriptions of segments of the audio recording utilizing an ASR system. These transcriptions are provided for review of transcribers. The computer receives questions from the transcribers regarding the transcriptions, and transmits the questions to a server in the room, which transmits back answers to the questions by the people in the room.

Rapid frontend resolution of transcription-related inquiries by backend transcribers

Maintaining adequate audio quality is very important for creating fast and accurate transcriptions, especially in a hybrid transcription setting, in which human transcribers review transcriptions generated by automatic speech recognition (ASR) systems. Some embodiments described herein involve detecting low-quality audio intended for transcription. In one embodiment, a server receives an audio recording that includes speech. The server generates feature values based on a segment of the audio recording and utilizes a model to calculate, based on the feature values, a certain value indicative of expected hybrid transcription quality of the segment. The model is generated based on training data that includes feature values generated based on previously recorded segments of audio, and values of transcription-quality metrics generated based on transcriptions of the previously recorded segments, which were generated at least in part by human transcribers. Optionally, an alert is provided responsive to the certain value being below a threshold.

Real time machine learning-based indication of whether audio quality is suitable for transcription

Early selection of operating parameters for improving accuracy of transcriptions generated by automatic speech recognition (ASR) systems. In one embodiment, a server receives an audio recording that includes speech, taken over a period spanning at least two hours. During the first hour, the server receives a ground truth transcription of a certain segment of the audio recording, created by a transcriber after listening to the certain segment. The server operates an ASR system a plurality of times, using a plurality of sets of operating parameters, to generate a plurality of respective transcriptions of the certain segment. The server evaluates accuracies of the plurality of transcriptions with respect to the ground truth transcription, and selects an optimal set of operating parameters. The server may then apply the optimal set of operating parameters to transcribe additional segments of the audio recording utilizing the ASR system.

Early selection of operating parameters for automatic speech recognition based on manually validated transcriptions

Hybrid transcription of audio relies on having one or more layers of transcribers who review transcriptions generated by automatic speech recognition (ASR) systems in order to correct errors that are found in the transcriptions. When it comes to determining how much human reviewing is needed, such as determining how many layers of review to use, there is a cost/benefit tradeoff to consider. Some embodiments described herein utilize a machine learning-based approach for estimating quality of hybrid transcription of audio. In one embodiment, a computer generates a transcription of a segment of audio using an ASR system, which is subsequently reviewed by a transcriber. The computer then calculates, based on properties of the review by the transcriber, a value indicative of an expected accuracy of the reviewed transcription. The computer may suggest a second transcriber review the reviewed transcription if the value indicative of the expected accuracy is below a threshold.

Quality estimation of hybrid transcription of audio

Described herein are curation of a glossary and its utilization for automatic speech recognition (ASR). In one embodiment, a server receives an audio recording of speech, taken over a period spanning at least two hours. During the first hour, the server generates, utilizing an ASR system, a transcription of a segment of the audio, recorded during the first twenty minutes. The server receives, from a transcriber, a phrase that does not appear in the transcription, but was spoken in the segment, and adds the phrase to a glossary. After the first hour of the period, the server generates, utilizing the ASR system, a second transcription of a second segment of the audio, provides the second transcription and the glossary to a second transcriber, and receives a corrected transcription, in which the second transcriber substituted a second phrase in the second transcription, which was not in the glossary, with the phrase.

Human-curated glossary for rapid hybrid-based transcription of audio

When transcribing large audio files, such as in the case of legal depositions, there are often many transcribers to choose from. Embodiments described herein enable calculation of expected accuracy of transcriptions by transcribers, which can be used to guide the selection of transcribers for specific tasks. In one embodiment, a computer receives a segment of an audio recording that includes speech of a person, and identifies an accent of the person and a topic of the segment. The computer generates feature values based on data that includes the accent and the topic, and utilizes a model to calculate, based on the feature values, an expected accuracy of a transcription of the segment by a certain transcriber. The model is generated based on training data that includes segments of previous audio recordings and values of accuracies of transcriptions, by the certain transcriber, of the segments.

Growing community of inventors

Ramat Hasharon, Israel

Yaakov Kobi Ben Tsvi