Gopro, Inc.

Guanhang Wu

Desmond Chik

Daniel Tse

Systems and method of identifying activities and/or events represented in a video are presented herein. An activity and/or event may be represented in a video by virtue of one or both of an entity moving with a capture device during capture of the video preforming the activity and/or event, or the video portraying one or more entities performing the activity and/or event. Activity types may be characterized by one or more of common movements, equipment, spatial context, and/or other features. Events may be characterized by one or both of individual movements and/or sets of movements that may routinely occur during performance of an activity. The identification of activities and/or events represented in a video may be based on one or more spectrogram representations of sensor output signals of one or more sensors coupled to a capture device.

Systems and methods for identifying activities and/or events represented in a video

An image including a visual capture of a scene may be accessed. The image may be processed through a convolutional neural network. The convolutional neural network may generate a set of two-dimensional feature maps based on the image. The set of two-dimensional feature maps may be processed through a contextual long short-term memory unit. The contextual long short-term memory unit may generate a set of two-dimensional outputs based on the set of two-dimensional feature maps. A set of attention-masks for the image may be generated based on the set of two-dimensional outputs and the set of two-dimensional feature maps. The set of attention-masks may define dimensional portions of the image. The scene may be classified based on the two-dimensional outputs.

Growing community of inventors

Hunan, China

Guanhang Wu