Gopro, Inc.

Integrated Device Technology, Inc.

Sandeep Doshi

Bruno Cesar Douady-Pleven

Sophana Kok

Adeel Abbas

Sumit Chawla

Bruno César Douady

Video information defining video content to be encoded may be obtained. Scene composition information for the video content may be obtained. The scene composition information may be determined by a convolutional neural network based on visuals represented within the video content. The video content may be encoded based on the scene composition information. The encoding of the video content may generate encoded video information defining the encoded video content.

Systems and methods for encoding videos based on visuals captured within the videos

Systems and methods are disclosed for image signal processing. For example, methods may include receiving, by an image signal processor, one or more input image signals from one or more image sensors; determining a mapping based on the input image signal(s), wherein the mapping includes records that associate image portions of an output image with corresponding image portions of the input image signal(s); sorting the records of the mapping according to an order of the corresponding image portions of the input image signal(s); applying, by the image signal processor, image processing to image portions of the input image signal(s) to determine image portions of one or more processed images in the order; and determining, by the image signal processor, the image portions of the output image based at least in part on the mapping and the corresponding image portions of the processed image(s) in the order.

Warp processing for image capture

Image signal processing including generating image signal processing based encoding hints for motion estimation may include an image signal processor obtaining an input image portion of an input image from the input image signal, generating motion information for the input image portion, processing the input image portion based on the motion information, outputting processed image data, and outputting the motion information as encoding hints, such that the motion information is accessible by an encoder for generating an encoded output bitstream by obtaining the processed image data as source image data, obtaining the motion information, generating prediction data for encoding the source image data based on the motion information, generating encoded image data based on the prediction data, and including the encoded image data in an encoded output bitstream.

Image signal processor for local motion estimation and video codec

Systems and methods for utilizing on-camera sensor information to improve video and/or image encoding quality are discussed herein. Specifically, the systems and methods may utilize on-camera sensor information to efficiently determine whether to encode a particular frame within a set of frames as an intra frame. When captured on video, a particular arrangement of a group of pixels within a frame may comprise a visual representation of an object within the frame. When encoding video footage, motion information characterizing motion of an image capturing device over time may be used to predict displacement of an arrangement of group of pixels between frames. These predictions may be used to efficiently determine whether to encode a particular frame as an intra frame.

Motion estimation through the use of on-camera sensor information

A camera system processes images based on image luminance data. The camera system includes an image sensor, an image pipeline, an encoder and a memory. The image sensor converts light incident upon the image sensor into raw image data. The image pipeline converts raw image data into color-space image data and calculates luminance levels of the color-space image data. The encoder can determine one or more of quantization levels, determining GOP structure or reference frame spacing for the color-space image data based on the luminance levels. The memory stores the color-space image data and the luminance levels.

On-camera image processing based on image luminance data

Image signal processing based encoding hints for bitrate control may include an image signal processor receiving an input image signal, identifying a block of a frame based on the input image signal, generating processed image data corresponding to the block, determining a processed image indication for the block, and outputting the processed image data and the processed image indication as an encoding hint. Image signal processing based encoding hints for bitrate control may include an encoder receiving source image data for a frame, the source image data including the processed image data corresponding to the block, receiving source image indications for the frame, the source image indications including the processed image indication, determining an encoding parameter for encoding the frame based on the processed image indication, generating an encoded frame by encoding the frame using the encoding parameter, and outputting or storing the encoded frame.

Image signal processing based encoding hints for bitrate control

A camera system processes images based on image activity data. The camera system includes an image sensor, an image pipeline, an encoder and a memory. The image sensor converts light incident upon the image sensor into raw image data. The image pipeline converts raw image data into color-space image data and calculates activity variances of the color-space image data. The encoder can determine one or more of quantization levels, block type (Intra vs Inter), determining transform size and type, and determining GOP structure or reference frame spacing for the color-space image data based on the activity variances. The memory stores the color-space image data and the activity variances.

On-camera image processing based on image activity data

Example methods and apparatuses including a picture parallel decoder are described herein. The picture parallel decoder includes an entropy pre-processing unit, an entropy transcoding unit, and a plurality of decoders. The entropy pre-processing unit may be configured to determine dependencies between frames of an encoded bitstream and to determine slice location data within the encoded bitstream based on dependencies between the frames. The entropy transcoding unit may be configured to transcode slices of the encoded bitstream based on the dependencies between frames. The plurality of decoders may be configured to decode two or more of the transcoded slices in parallel.

Growing community of inventors

Sunnyvale, CA, United States of America

Sandeep Doshi