Evaluating content publisher options against benchmark publisher

Abstract

An online system evaluates the quality of a content publisher displaying sponsored content items. To determine a likelihood of conversion actions associated with the sponsored content items, the online system uses information about users and their interactions with sponsored content items featured within the content publisher against interactions with sponsored content items featured within a benchmark system (e.g., online system). By determining a ratio of these interactions, the online system can determine a likelihood of conversion actions for the content publisher. The online system uses this likelihood of conversions to determine a publisher quality score that it uses to normalize third party value contributions toward placing sponsored content items on the content publisher. Thus, third party systems no longer need to be concerned about the intrinsic value of a given content publisher as third party value contributions are normalized based on the content publisher's conversion rates.