Amazon Technologies, Inc.

Davide Modolo

Andrea Olgiati

Joseph Patrick Tighe

Stephen Gould

Hao Chen

Vasant Manohar

Nataliya Shapovalova

Vuong Van Le

Nitin Singhal

Enrica Maria Filippi

Camille Claire Le Men

Yuri Natanzon

Fanyi Xiao

Techniques for training a visual neural network are described. In particular, the training of the visual neural network may be performed using one or more contrastive learning loss functions including one or more of a visual-to-visual contrastive learning function using the visual neural network on positive and negative video clips according to a first loss function, a secondary-to-secondary contrastive learning a secondary neural network on secondary positive and negative information generated from the positive and negative video clips, and a secondary-to-visual contrastive learning according to a third loss function using the visual neural network on positive and negative video clips and using the secondary neural network secondary positive and negative information generated from the positive and negative video clips.

Secondary distillation for self-supervised video representation learning

Technologies are disclosed for efficiently identifying objects in videos using deep neural networks and motion information. Using the disclosed technologies, the amount of time required to identify objects in videos can be greatly reduced. Motion information for a video, such as motion vectors, are extracted during the encoding or decoding of the video. The motion information is used to determine whether there is sufficient motion between frames of the video to warrant performing object detection on the frames. If there is insufficient movement from one frame to a subsequent frame, the subsequent frame will not be processed to identify objects contained therein. In this way, object detection will not be performed on video frames that have changed minimally as compared to a previous frame, thereby reducing the amount of time and the number of processing operations required to identify the objects in the video.

Efficient identification of objects in videos using motion information

People can be tracked across multiple segments of video data, which can correspond to different scenes in a single video file, or multiple video streams or feeds. An instance of video data can be broken up into segments that can each be analyzed to determine faces and bodies represented therein. The bodies can be analyzed across frames of the segment to determine body tracklets that are consistent across the segment. Associations of faces and bodies can be determined based using relative distances and/or spatial relationships. A subsequent clustering of these associations is performed to attempt to determine consistent associations that correspond to unique individuals. Unique identifiers are determined for each person represented in one or more segments of an instance of video data. Such an approach enables individual representations to be correlated across multiple instances.

Person tracking across video instances

A data processing pipeline controller receives a request, from a data iterator associated with a machine learning model, for a data output of a module in the data processing pipeline, wherein each module in the data processing pipeline has an associated cache. The controller determines whether a data output of the module is stored in the associated cache and responsive to the data output being stored in the associated cache, provides the data output from the associated cache to the data iterator without processing data through the module.

Data iterator with automatic caching

Growing community of inventors

Seattle, WA, United States of America

Davide Modolo