Microsoft Technology Licensing, LLC

Google Inc.

Phillip John Isola

Ce Liu

Dilip Krishnan

Chen Wang

Aaron Joseph Maschinot

Aaron Yehuda Sarna

Piotr Teterwak

Prannay Khosla

Yonglong Tian

The present disclosure provides an improved training methodology that enables supervised contrastive learning to be simultaneously performed across multiple positive and negative training examples. In particular, example aspects of the present disclosure are directed to an improved, supervised version of the batch contrastive loss, which has been shown to be very effective at learning powerful representations in the self-supervised setting. Thus, the proposed techniques adapt contrastive learning to the fully supervised setting and also enable learning to occur simultaneously across multiple positive examples.

Supervised contrastive learning with multiple positive examples

The subject disclosure is directed towards layered image understanding by which a layered scene representation is generated for an image. Providing such a scene representation explains a scene being presenting in the image by defining that scene's semantic structure. To generate the layered scene representation, the subject disclosure recognizes objects within the image by combining objects sampled from annotated image data and determining whether that combination is both semantically well-formed and matches the visual appearance of the image. The objects are transformed and then can be used to modify the query image. The subject disclosure models the objects into semantic segments that form a portion of the scene representation.

Layered image understanding

Growing community of inventors

Cambridge, MA, United States of America

Phillip John Isola