Apple Inc.

Ati International Srl

Haitao Guo

Stephen Olivier Lemay

Ting Chen

Guy Cote

Hao Pan

Thorsten Gernoth

Suk Hwan Lim

D Amnon Silverstein

Sheng Lin

Xiaojin Shi

Feng Tang

Kenneth I Greenebaum

Alexandros Michael Tourapis

Hsi-Jung Wu

David William Singer

Ralph T Brunner

Timothy John Millet

Giovanni M Agnoli

Dazhong Zhang

Yingjun Bai

Elizabeth Caroline Cranfill

Yeping Su

Xiaosong Zhou

Athanasios Leontaris

Jim Chen Chou

Joe S Abuan

Hyeonkuk Jeong

James O Normile

Richard Lawrence Baer

Yan Yang

Roberto Garcia

David Daming Kuo

Xiaohua Yang

Andrew Yanowitz

Andrew Bai

Gregory Kent Wallace

Mitchell Howard Oslick

Richard W Webb

Jay Zipnick

Atulit Kumar

Ian R Fasel

Munehiro Nakazato

Chong Chen

Krishnakanth Rapaka

Vikrant Kasarabada

Yongfang Liang

Linfeng Guo

Sally Fung

Siladitya Bhattacharya

Sumit Chawla

Sorin Constantin Cismas

Xuemei Zhang

Berkat S Tung

Farhan A Baqai

Wei Li

Roberto Garcia, Jr

Douglas Scott Price

Chad Edward Fogg

Onur Canturk Hamsici

Ke Chen

Stefan Eckart

Peter Chou

Michael L Lightstone

Touraj Tajbakhsh

James Battle

Haiyan He

Munehiro Mori

Eslam A Mostafa

Andy Chao Hung

Francesco Iacopino

Ye Cong

Xu Zhao

Xiang Fu

David A Eldred

Vaidyanath Mani

Fabio Ingrao

Munchiro Mori

Shannon L Gardiner

David D Kuo

The present disclosure generally relates to systems and methods for image data processing. In certain embodiments, an image processing pipeline may be configured to receive a frame of the image data having a plurality of pixels acquired using a digital image sensor. The image processing pipeline may then be configured to determine a first plurality of correction factors that may correct each pixel in the plurality of pixels for fixed pattern noise. The first plurality of correction factors may be determined based at least in part on fixed pattern noise statistics that correspond to the frame of the image data. After determining the first plurality of correction factors, the image processing pipeline may be configured to apply the first plurality of correction factors to the plurality of pixels, thereby reducing the fixed pattern noise present in the plurality of pixels.

Systems and methods for reducing fixed pattern noise in image data

In one implementation, a method of generating a bit stream encoding a video stream is performed by a device including one or more processors and non-transitory memory. The method includes decomposing a video stream into a plurality of frequency band video streams. The method includes determining a target bitrate and determining, for each frequency band video stream, a respective frequency band bit rate based on the target bit rate. The method includes encoding each of the plurality of frequency band video streams at its respective frequency band bit rate and transmitting, over a channel, each encoded frequency band video stream.

Multi-band rate control

The present disclosure generally relates to systems and methods for image data processing. In certain embodiments, an image processing pipeline may be configured to receive a frame of the image data having a plurality of pixels acquired using a digital image sensor. The image processing pipeline may then be configured to determine a first plurality of correction factors that may correct each pixel in the plurality of pixels for fixed pattern noise. The first plurality of correction factors may be determined based at least in part on fixed pattern noise statistics that correspond to the frame of the image data. After determining the first plurality of correction factors, the image processing pipeline may be configured to configured to apply the first plurality of correction factors to the plurality of pixels, thereby reducing the fixed pattern noise present in the plurality of pixels.

Systems and method for reducing fixed pattern noise in image data

Intelligent systems are disclosed that respond to user intent and desires based upon activity that may or may not be expressly directed at the intelligent system. In some embodiments, the intelligent system acquires a depth image of a scene surrounding the system. A scene geometry may be extracted from the depth image and elements of the scene may be monitored. In certain embodiments, user activity in the scene is monitored and analyzed to infer user desires or intent with respect to the system. The interpretation of the user's intent as well as the system's response may be affected by the scene geometry surrounding the user and/or the system. In some embodiments, techniques and systems are disclosed for interpreting express user communication, e.g., expressed through hand gesture movements. In some embodiments, such gesture movements may be interpreted based on real-time depth information obtained from, e.g., optical or non-optical type depth sensors.

Multi media computing or entertainment system for responding to user presence and activity

Video processing techniques and pipelines that support capture, distribution, and display of high dynamic range (HDR) image data to both HDR-enabled display devices and display devices that do not support HDR imaging. A sensor pipeline may generate standard dynamic range (SDR) data from HDR data captured by a sensor using tone mapping, for example local tone mapping. Information used to generate the SDR data may be provided to a display pipeline as metadata with the generated SDR data. If a target display does not support HDR imaging, the SDR data may be directly rendered by the display pipeline. If the target display does support HDR imaging, then an inverse mapping technique may be applied to the SDR data according to the metadata to render HDR data for display. Information used in performing color gamut mapping may also be provided in the metadata and used to recover clipped colors for display.

Backward-compatible video capture and distribution

Adaptive video processing for a target display panel may be implemented in or by a server/encoding pipeline. The adaptive video processing methods may obtain and take into account video content and display panel-specific information including display characteristics and environmental conditions (e.g., ambient lighting and viewer location) when processing and encoding video content to be streamed to the target display panel in an ambient setting or environment. The server-side adaptive video processing methods may use this information to adjust one or more video processing functions as applied to the video data to generate video content in the color gamut and dynamic range of the target display panel that is adapted to the display panel characteristics and ambient viewing conditions.

Server-side adaptive video processing

Techniques are disclosed for coding image data adaptively at different levels of downscaling. Such techniques may involve partitioning input data into pixel blocks for coding and performing content analysis on the pixel blocks. The pixel blocks may be input to block coders that operate at different pixel block sizes, which may code the pixel blocks input to them at their respective sizes. Except when a block coder operates at the partitioning size, block coders that operate at different pixel block sizes may perform downscaling of the pixel blocks to match their size with the block coders' respective coding size. A block decoder may invert the coding operations performed by the block coders, decoding coded image data at respective pixel block sizes, then upscaling decoded image data obtained therefrom to a common pixel block size. Image reconstruction may synthesize a resultant image from the decode pixel block data output by the decoders. Quality enhancements may be added by neural network processing systems that have been trained to other segmented images.

Deep quality enhancement of adaptive downscaled coding for image compression

Methods are described for encoding and decoding blocks of image data using intra block copying (IBC). A source block for intra block copying is selected from a source region of a current image that is closer to the current block than a threshold, wherein the source region does not include a portion of the current image that is further from the current block than the threshold.

Techniques for resource conservation during performance of intra block copy prediction searches

A single network encodes and decodes an image captured using a camera on a device. The single network detects if a face is in the image. If a face is detected in the image, the single network determines properties of the face in the image and outputs the properties along with the face detection output. Properties of the face may be determined by sharing the task for face detection. Properties of the face that are output along with the face detection output include the location of the face, the pose of the face, and/or the distance of the face from the camera.

Face detection, pose estimation, and distance from a camera estimation using a single network

An estimate of distance between a user and a camera on a device is used to determine an illumination pattern density used for speckle pattern illumination of the user in subsequent images. The distance may be estimated using an image captured when the user is illuminated with flood infrared illumination. Either a sparse speckle (dot) pattern illumination pattern or a dense speckle pattern illumination pattern is used depending on the distance between the user's face and the camera.

Determining sparse versus dense pattern illumination

Templates used for a facial recognition process for authentication of a user to use a device may be updated by the device as features of the user change over time. Features of the user may gradually change over time due to changes such as facial hair changes, haircuts, gaining/losing weight, and/or aging. Updating the templates used for the facial recognition process may allow the user to continue being authenticated as features of the user change without the need for additional enrollments of the user.

Process for updating templates used in facial recognition

Adaptive video processing for a target display panel may be implemented in or by a decoding/display pipeline associated with the target display panel. The adaptive video processing methods may take into account video content, display characteristics, and environmental conditions including but not limited to ambient lighting and viewer location when processing and rendering video content for a target display panel in an ambient setting or environment. The display-side adaptive video processing methods may use this information to adjust one or more video processing functions as applied to the video data to render video for the target display panel that is adapted to the display panel according to the ambient viewing conditions.

Display-side adaptive video processing

Growing community of inventors

San Jose, CA, United States of America

Haitao Guo