Amazon Technologies, Inc.

Microsoft Technology Licensing, LLC

Serguei B Stepaniants

Rui Chen

Daniel R Parshall

Michael Luis Collado

Siwei Jia

Kevin Donald Kelly

Brent Russell Smith

Murali Brahmadesam

Sameer R Rajyaguru

Sebastian Kohlmeier

Eric E Schadt

Pek Yee Lum

Robert Andrew Kreek

Shen Li

Tae Yeon Kim

Douglas R Hains

Mengyun Lv

Ajay Kant Singh

Kyle Leon Battisti

Donald Kane

Vignesh Kannappan

Geoffrey Ryan Dworkin

Hancao Li

Sen Zhao

Jeff Brooks

Mia Kilborn

Zhen Liu

Pramod Muralidharan

Richard Bargreen

Kevin McAlister

Ray Chen

Roy Kuraisa

Technologies are disclosed for determining validity of data obtained from an A/B experiment, where the experiment evaluates the desirability of a potential change at a website. The experiment is run for a period of time and based upon an expected allocation of users into the A group (control group) and the B group (e.g., treatment group), along with an actual number of users directed into the two groups, it is determined if a minimal detectable error of allocation has been exceeded and, if it has, the data is deemed to be invalid. If not, the data is deemed to be valid.

Determining the validity of data collected by experiments performed at a network accessible site

Technologies are disclosed for determining network-accessible content changes based on computed models and providing a long term forecast of user interaction at a network accessible site based upon a short term experiment at the site. A forecast model for a period of time is generated based upon historical data of user interactions at the site. An experiment is run for a short term at the site based upon a potential change at the site. Based upon data obtained during the experiment, scores are generated for a control group (no change) and a treatment group (potential change) and compared. If there are statistically significant differences between the control group and the treatment group scores, the long term forecast may be used to forecast what the long term impact of the experiment would be based upon the short term experiment.

Computationally and network bandwidth-efficient technique to determine network-accessible content changes based on computed models

A computer-facilitated service selects, in response to a request to access user interface, an interaction element that can be presented via the user interface. The computer-facilitated service records information involving user interactions with the interaction element presented via the user interface. A model is applied to the recorded information to determine a classification of the user from a set of classifications that comprises human users and automated agents. The computer-facilitated service records an association between the user and the classification.

Detection of automated agents through interaction element presentation

Described herein is a system and method for processing analytics data in a windowed manner. In some embodiments, each user of a number of users is assigned to a group (e.g., a treatment group or a control group). The system may monitor each user to obtain metric data associated with each of the users over the course of an experiment. In some embodiments, the system may identify an indication of a trigger event with respect to each user. For example, the indication of the trigger event may comprise a time at which an event occurred for that user. The system may identify and compile data from the metric data for each of the users with respect to the trigger event. In some embodiments the metric data may be aggregated for each of the users with respect to a window or timespan positioned relative to the identified trigger event.

Windowed approach to analytics processing

Technologies are disclosed for determining a runtime length for an A/B experiment, where the experiment evaluates the desirability of a potential change at a website. The experiment is run for an initial period of time and based upon initial data from the initial period of time, an equation is iteratively solved until a minimal amount of time is determined that that indicates when a statistically significant change will be observed in the data. The experiment is then run for the minimal amount of time.

Determining a run time for experiments performed at a network accessible site

Described are techniques for determining long-term effects of an experimental change to a user experience after the end of the experiment. A control state and a treatment state of a statistical hypothesis experiment may be assigned to first and second client devices, respectively, during an experiment time period. Subsequent to the end of the experiment, presentation of the control state may be discontinued. Result data corresponding to the treatment state may be determined during the experiment time period and for a length of time subsequent to the experiment time period. Result data corresponding to the control state may be determined during the experiment time period, and for a length of time prior to assignment of the control state to the first client device.

Systems for determining long-term effects in statistical hypothesis testing

Systems and methods for determining a functional relationship between pairs of cellular constituents are provided. A plurality of datasets is received. Each dataset represents an experimental condition and comprises measurement data for a plurality of cellular constituents from each of a plurality of organisms. Each respective dataset is represented by correlation coefficients. Each correlation coefficient for a respective dataset in the plurality of datasets represents a correlation between abundance measurement data for a pair of cellular constituents across the dataset. The plurality of correlation coefficients that represents a first dataset in the plurality of datasets is clustered, thereby determining their order. This order is applied to each remaining dataset thereby forming a plurality of correlation matrices. When a conserved area in the plurality of matrices is identified, the functional relationship between the first cellular constituent and the second constituent is determined to be present.

Computer systems and methods for identifying conserved cellular constituent clusters across datasets

Growing community of inventors

Redmond, WA, United States of America

Serguei B Stepaniants