Amazon Technologies, Inc.

Raytheon Bbn Technologies Corp.

Shuang Wu

Shiv Naga Prasad Vitaladevuni

Gengshen Fu

Paul McIntyre

Thibaud Senechal

Ilya Sokolov

Rohit Prasad

Yue Liu

Pradeep Natarajan

David Paul Ramos

Premkumar Natarajan

Aaron Challenner

Qingfeng Yu

Michael Thomas Peterson

Andrew Morris Werchniak

Aansh Shah

Raphael Petegrosso

Quan Wang

Daniel Makoto Willenson

A system performs federated learning and retraining of a machine learning model used for processing audio detected by a user device. The system uses both gradient data (which may correspond to false-rejects) and audio data (which may correspond to false-positives) received from devices. The system may also use a teacher model to produce labels for data in an automated fashion, thus allowing retraining to happen in an unsupervised manner.

Federated learning for audio processing

Techniques for using a dynamic wakeword detection threshold are described. A device detects a wakeword in audio data using a first wakeword detection threshold value. Thereafter, the device receives audio including speech. If the device receives the audio within a predetermined duration of time after detecting the previous wakeword, the device attempts to detect a wakeword in second audio data, corresponding to the audio including the speech, using a second, lower wakeword detection threshold value.

Dynamic wakeword detection

A device monitors audio data for a predetermined and/or user-defined wakeword. The device detects an error in detecting the wakeword in the audio data, such as a false-positive detection of the wakeword or a false-negative detection of the wakeword. Upon detecting the error, the device updates a model trained to detect the wakeword to create an updated trained model; the updated trained model reduces or eliminates further errors in detecting the wakeword. Data corresponding to the updated trained model may be collected by a server from a plurality of devices and used to create an updated trained model aggregating the data; this updated trained model may be sent to some or all of the devices.

Wakeword training

Techniques for using a dynamic wakeword detection threshold are described. A server(s) may receive audio data corresponding to an utterance from a device in response to the device detecting a wakeword using a wakeword detection threshold. The server(s) may then determine the device should use a lower wakeword detection threshold for a duration of time. In addition to sending the device output data responsive to the utterance, the server(s) may send the device an instruction to use the lower wakeword detection threshold for the duration of time. Alternatively, the server(s) may train a machine learning model to determine when the device should use a lower wakeword detection threshold. The server(s) may send the trained machine learned model to the device for use at runtime.

Various examples are directed to methods and systems for object recognition in an image. A computer vision system may receive a patch comprising a plurality of pixels arranged in a grid. The computer vision system may determine a plurality of columns and a plurality of rows in the patch. The plurality of columns may be based at least in part on a column target sum and the plurality of rows may be based at least in part on a row target sum.

Object recognition with attribute-based cells

A multi-orientation text detection method and associated system is disclosed that utilizes orientation-variant glyph features to determine a text line in an image regardless of an orientation of the text line. Glyph features are determined for each glyph in an image with respect to a neighboring glyph. The glyph features are provided to a learned classifier that outputs a glyph pair score for each neighboring glyph pair. Each glyph pair score indicates a likelihood that the corresponding pair of neighboring glyphs form part of a same text line. The glyph pair scores are used to identify candidate text lines, which are then ranked to select a final set of text lines in the image.

Text detection using features associated with neighboring glyph pairs

An approach to computation of kernel descriptors is accelerated using precomputed tables. In one aspect, a fast algorithm for kernel descriptor computation that takes O(1) operations per pixel in each patch, based on pre-computed kernel values. This speeds up the kernel descriptor features under consideration, to levels that are comparable with D-SIFT and color SIFT, and two orders of magnitude faster than STIP and HoG3D. In some examples, kernel descriptors are applied to extract gradient, flow and texture based features for video analysis. In tests of the approach on a large database of internet videos used in the TRECVID MED 2011 evaluations, the flow based kernel descriptors are up to two orders of magnitude faster than STIP and HoG3D, and also produce significant performance improvements. Further, using features from multiple color planes produces small but consistent gains.

Growing community of inventors

Cambridge, MA, United States of America

Shuang Wu