Gopro, Inc.

Daniel Tse

Desmond Chik

Jonathan Wills

Brian G Schunck

Guanhang Wu

Mahlet Getachew

Rajvikram Singh

Systems and method of generating video summaries are presented herein. Information defining a video may be obtained. The video may include a set of frame images. Parameter values for parameters of individual frame images of the video may be determined. Interest weights for the frame images may be determined. An interest curve for the video that characterizes the video by interest weights as a function of progress through the set of frame images may be generated. One or more curve attributes of the interest curve may be identified and one or more interest curve values of the interest curve that correspond to individual curve attributes may be determined. Interest curve values of the interest curve may be compared to threshold curve values. A subset of frame images of the video to include within a video summary of the video may be identified based on the comparison.

Systems and methods for generating a video summary

A system and/or method configured to determine highlight segments. Content files that define content in a content segment set may be obtained. A highlight segment set may be determined from the content segment set. Determining the highlight segment set may include iterating (a)-(c) for multiple iterations. At (a), individual content segments included in the content segment set may be selected as a selected content segment for inclusion in the highlight segment set. At, (b) diversity scores for content segments that are (i) included in the content segment set and (ii) not yet selected for inclusion in the highlight segment set may be determined. At (c), one or more of the content segments may be disqualified for inclusion in the highlight segment set for future iterations based on the diversity scores.

Systems and methods for determining highlight segment sets

Systems and method of identifying activities and/or events represented in a video are presented herein. An activity and/or event may be represented in a video by virtue of one or both of an entity moving with a capture device during capture of the video preforming the activity and/or event, or the video portraying one or more entities performing the activity and/or event. Activity types may be characterized by one or more of common movements, equipment, spatial context, and/or other features. Events may be characterized by one or both of individual movements and/or sets of movements that may routinely occur during performance of an activity. The identification of activities and/or events represented in a video may be based on one or more spectrogram representations of sensor output signals of one or more sensors coupled to a capture device.

Systems and methods for identifying activities and/or events represented in a video

Video information defining video content may be accessed. The video content may include video frames. The video frames may be grouped into similarity groups based on similarity of the video frames. The similarity groups may include a first similarity group including a first video frame and a second video frame. The first video frame may be similar to the second video frame. Representative video frames may be selected from the similarity groups. An individual representative video frame may correspond to an individual similarity group. Representative video frames may include a first representative video frame selected from and corresponding to the first similarity group. The video content may be analyzed based on the representative video frames.

Systems and methods for accelerating video analysis

Video information defining video content may be accessed. The video content may have a progress length and include a capture of an activity and a sub-activity at one or more moments in the progress length. Portions of the video content may be associated individually with values of an interest metric such that the values of the interest metric as a function of progress through the video content form an interest curve for the video content. The activity and the sub-activity captured within the video content may be identified. An activity metric modifier for the interest curve at the one or more moments in the progress length may be determined based on the identification of the activity and the identification of the sub-activity. The interest curve may be modified at the one or more moments based on the activity metric modifier.

Systems and methods for identifying interesting moments within videos

A first command may be received from a client computing platform. The first command may include proxy visual content. The proxy visual content may represent visual content stored on the client computing platform. An identifier may be associated with the proxy visual content. The identifier may be transmitted to the client computing platform. The identifier may be associated with the visual content stored on the client computing platform. Edits to the visual content based upon the proxy visual content may be determined at a remote computing platform. Instructions may be transmitted from the remote computing platform to the client computing platform. The instructions may include the identifier. The instructions may be configured to cause the client computing platform to process the edits on the visual content.

Systems and methods that effectuate transmission of workflow between computing platforms

A system and/or method configured to determine a sample frame order for analyzing a video. The video may have multiple frames ordered in a sequence from a beginning to an end. A first sample frame order for analyzing the video may be determined. Determining the first sample frame order may include determining an initial frame for a first iteration, and determining secondary frames for a second iteration. Determining the initial frame and the secondary frames may be based on a function of frame position in the sequence of frames. The initial frame may be associated with a first sample position, and the secondary frames may be associated with secondary sample positions in the sample frame order. A first feature of the video may be determined based on an analysis of the frames in the video performed on the frames in the first sample frame order.

Systems and methods for determining a sample frame order for analyzing a video

System and/or method configured to determine a sample frame order for analyzing a video. The video may have multiple frames ordered in a sequence from a beginning to an end. Segments of the video may be determined based on an analysis of the frames of the video. A first sample frame order for analyzing the video may be determined based on multiple iterations performed on individual ones of the segments of the video. Determining the first sample frame order may include determining initial frames for a first iteration, and determining secondary frames for a second iteration based on a function of frame position in the sequence of frames. The initial frames and the secondary frames may be associated with sample positions in the sample frame order. A first feature of the video may be determined based on an analysis of the frames performed in the first sample frame order.

Systems and methods for determining a sample frame order for analyzing a video via segments

An image including a visual capture of a scene may be accessed. The image may be processed through a convolutional neural network. The convolutional neural network may generate a set of two-dimensional feature maps based on the image. The set of two-dimensional feature maps may be processed through a contextual long short-term memory unit. The contextual long short-term memory unit may generate a set of two-dimensional outputs based on the set of two-dimensional feature maps. A set of attention-masks for the image may be generated based on the set of two-dimensional outputs and the set of two-dimensional feature maps. The set of attention-masks may define dimensional portions of the image. The scene may be classified based on the two-dimensional outputs.

Systems and methods for classifying activities captured within images

A system and/or method configured to determine highlight segment sets. Content files that define content in content segment sets may be obtained. Individual highlight segment sets of content segments may be determined from the content segment sets. Determining a first highlight segment set of content segments included in the first content segment may include iterating (a)-(c) for multiple iterations. At (a), individual content segments included in the first content segment set may be selected as a selected content segment for inclusion in the first highlight segment set. At, (b) diversity scores for content segments that are (i) included in the first content segment set and (ii) not yet selected for inclusion in the first highlight segment set may be determined. At (c), one or more of the content segments may be disqualified for inclusion in the first highlight segment set for future iterations based on the diversity scores.

Images may be extracted from a video. The images may be grouped into image groups. Numbers and types of classified visuals within the images may be detected. Individual types of classified visuals may correspond to individual classification weights. Image classification weights for the images may be determined based on the numbers and the types of classified visuals and the individual classification weights. Interest weights for the images may be determined based on the image classification weights and the sizes of the image groups to which the individual images belong. An interest curve may be generated based on the interest weights. A retime curve may be generated based on the interest curve. Time lapse images to be included in the time lapse video may be determined based on the images and the retime curve. The time lapse video may be generated based on the time lapse images.

Growing community of inventors

San Mateo, CA, United States of America

Daniel Tse