Evaluating statistical significance of test statistics using placebo actions

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for evaluating statistical significance using placebo actions. One method includes receiving exposure data and control data describing exposed and control subjects and determining a test statistic for each of a plurality of subject actions including a desired action and a plurality of placebo actions. The test statistic for an action corresponds to an estimated effect that exposure to the treatment has on a likelihood that a subject will take the action. The method further includes comparing the test statistic for the desired action to the test statistics for the plurality of placebo actions, and determining whether the test statistic for the desired action is statistically significant according to the comparison. Another method includes automatically identifying placebo actions using a taxonomy that classifies subject actions or treatments, and determining test statistics for the placebo actions.