Amazon Technologies, Inc.

Ye He

Felix Xiaomeng Wu

Rongzhou Shen

Manish Dutt Sharma

Rahul Ravu

Philippe Di Cristo

John Dunavent

Jiang Xiang

Techniques for filtering the output of supplemental content are described. When a supplemental output system (e.g., a supplemental content system or notification system) receives supplemental content for output, the supplemental output system sends a user identifier (of the recipient user) and the supplemental content to separately implemented filtering component. The filtering component uses a machine learning (ML) model to determine a topic of the supplemental content. The filtering component determines whether the supplemental content should not be output based on the ML model-determined topic, one or more guardrail policies of the supplemental output system, and user frustration data regarding previously output supplemental content. Use of the ML model to determine the topic prevents a content publisher from surreptitiously associating supplemental content with a specific topic in an effort to bypass topic-based output guardrails.

Proactive supplemental content output

Techniques for performing tests by a speech processing system are described. A speech processing system may apply a hash function to a group of user identifiers, a content provider identifier, and time data to segment the group of user identifiers into non-holdout, holdout control, and holdout treatment groups. A non-holdout group represents users not subject to a test for a duration of time. A holdout control group represents users subject to a test for the duration of time, but which do not receive unresponsive content during the duration of time. A holdout treatment group represents users subject to a test and which receive unresponsive content over the duration of time. User activity of the holdout control and holdout treatment groups may be compared to measure an effectiveness of the unresponsive content. The measured effectiveness may be used to determine a ranking of the content provider at runtime.

Content selection by natural language processing systems

Growing community of inventors

Kirkland, WA, United States of America

Ye He