Facebook, Inc.

Enming Luo

Emanuel Alexandre Strauss

Daniel Olmedilla De La Calle

Yang Mu

Taiyuan Zhang

A content review system for an online system automatically determines if received content items to be displayed to users violate any policies of the online system. The content review system generates a semantic vector representing the semantic features of a content item, for example, using a neural network. By comparing the semantic vector for the content item with semantic vectors of content items previously determined to violate one or more policies, the content review system determines whether the content item also violates one or more policies. The content review system may also maintain templates corresponding to portions of semantic vectors shared by multiple content items. An analysis of historical content items that conform to the template is performed to determine a probability that received content items that conform to the template violate a policy.

Detecting content items in violation of an online system policy using semantic vectors

Detecting content items in violation of an online system policy using templates based on semantic vectors representing content items

An online system uses a model to detect violations of policies enforced by the online system for content uploaded to the online system by users for viewing by other users. The online system trains the model in multiple stages. To train the model, the online system obtains a set of training content items, with each content item of the set labeled with both a policy violated by the content item and a source of the content item, which acts as a proxy for a sub-category identifying a way in which the content item violated the policy. In the first stage, the online system trains the model using the set of training content items. In a second stage, the model of trained to predict policy violations from content items that are not labeled with a source. For example, the second stage is performed by freezing earlier layers in the model.

Evaluation of content items against policies regulating content presentation by an online system using machine learning

An online system determines whether a test content item violates a policy of the online system. The online system extracts a semantic from the test content item and determines a distance between the extracted semantic vector and the stored semantic vectors for content items that have been labeled to indicate whether they violate a policy. Using a nearest neighbor search, the online system selects a set of the stored semantic vectors and assigns a weight to the selected semantic vectors that is inversely related to the distances. The online system then determines whether the test content item violates a policy using a weighed voting scheme, where the labels of the stored semantic vectors are aggregated based on their associated weights. The online system may first attempt to match the test content with known bad content and terminate the more complex nearest neighbor search if such a match is found.

Policy compliance verification using semantic distance and nearest neighbor search of labeled content

A content review system for an online system automatically determines if received content items to be displayed to users contain text that violates a policy of the online system. The content review system generates a semantic vector representing semantic features of text extracted from the content item, for example, using a neural network. By comparing the semantic vector for the extracted text with stored semantic vectors of extracted text previously determined to violate one or more policies, the content review system determines whether the content item contains text that also violates one or more policies. The content review system also reviews stored semantic vectors previously determined to be unsuitable, in order to remove false positives, as well as unsuitable semantic vectors that are sufficiently similar to known suitable semantic vectors and as such may cause content items having suitable text to be erroneously rejected.

Growing community of inventors

Fremont, CA, United States of America

Enming Luo