Apple Inc.

Cisco Technology, Inc.

Sony Corporation

Sony Electronics Inc

Jim Chen Chou

Guy Cote

Yun Gong

Hao Pan

Sorin Constantin Cismas

Moinul H Khan

Tobias Eble

Mark P Rygh

Jian Zhou

Shereef Shehata

Sheng Zhang

Arthur Yasheng Zhang

Weichun Ku

Haiyan He

Ling Su

Timothy John Millet

Rohit Puri

Yung-Chin Chen

Joseph J Cheng

Syed Muhammad A Rizvi

Alexandros Michael Tourapis

Avi Bar-Zeev

Marc Albrecht

Geoffrey Grant Stahl

Gurjeet S Saund

Ray Ling Chang

Timothy Robert Oriol

Guy Côté

Justin A Hensley

Krishnakanth Rapaka

Honkai Tam

Shengchang Cai

Munehiro Nakazato

Ye Cong

Hariprasad Puthukkootil Rajagopal

Chenge Li

Ali Jezu Tabatabai

Mahesh B Chappalli

Christopher P Tann

Nicolas P Bonnier

Arthur L Spence

Rui Zhang

Keangpo Ricky Ho

Roy Gideon Moss

Myung-Je Cho

Craig M Okruhlica

Jae Y Park

Ganesh G Yadav

Felix C Fernandes

Jiandong Shen

Cody J White

Randall R Rauwendaal

Jungsun Kim

Hsi-Jung Wu

Yeping Su

Charles Calvin Byers

David William Singer

Chaohao Wang

Khaled Mammou

Athanasios Leontaris

Jean-Pierre Simon Guillou

Hyunwoo Nho

Joseph P Bratt

Paul M Quinn

Fabrice A Robinet

Raffi A Bedikian

Haitao Guo

Aleksandr M Movshovich

Elango Ganesan

Michael Freed

Enkhamgalan Dorjgotov

Mohamed Selim Ben Himane

Koorosh Aflatooni

Paul V Johnson

Sreeraman Anantharaman

Marc A Schaub

Ke Chen

Sachin S Kajarekar

Giovanni Carbone

Xiaohua Yang

Ananth Sankar

Junlin Li

Holly Gerhard

Assaf Menachem

Jaehong Chon

Ido Y Soffair

Pedro Manuel Da Silva Quelhas

William W Sprague

Lingtao Wang

Linfeng Guo

Francesco Iacopino

Tapabrata Biswas

Katharina Buckl

Gaurav S Gulati

Uri Nix

Manching Ko

Guy Cõté

Syed Muhammad Ali Rizvi

Julien Monat Rodier

Embodiments presented herein relate to reducing perceivable image artifacts on an electronic display caused by variances in emission timing of emission groups, and more specifically, to emission techniques that can be used with foveated content, such as dynamically foveated content. For example, an image frame may be divided into multiple foveation regions based on viewing characteristics, such as a viewer's gaze. To improve perceived image quality, a first foveation region may use a higher pixel resolution while a second foveation region may use a lower pixel resolution, which may result in differences in programming time. As such, one or more intra-frame pauses may be used when programming the second foveation region to delay emission timing of the second foveation region. In this way, programming time for the image frame may be more consistent and perceivable image artifacts from frame to frame may be reduced.

Intra-frame pause and delayed emission timing for foveated displays

Methods and systems include neural network-based image processing and blending circuitry to blend an output of the neural network to compensate for potential artifacts from the neural network-based image processing. The neural network(s) apply image processing to image data using one or more neural networks as processed data. Enhance circuitry enhances the image data in a scaling circuitry to generate enhanced data. Blending circuitry receives the processed image data and the enhanced data along with an image plane of the processed data. The blending circuitry also determines whether the image processing using the one or more neural networks has applied a change to the image data greater than a threshold amount. The blending circuitry then, based at least in part in response to the change being greater than the threshold amount and/or edge information of the image data, blends the processed data with the enhanced data.

Neural network-based image processing with artifact compensation

An electronic display may include a display panel comprising a plurality of display pixels, an image source configured to store image data, and image processing circuitry. The image processing circuitry may receive a brightness level of the display panel and receive the image data that may include gray level data for a first display pixel of the plurality of display pixels. The image processing circuitry may convert the gray level data to voltage data based on the brightness level, determine a compensation for the voltage data based on a global voltage compensation value and a local voltage compensation value, and apply the compensation to the voltage data to generate compensated voltage data. The image processing circuitry may compress a range of the compensated voltage data and convert the compensated voltage data into compensated gray level data for the first display pixel.

Micro-OLED sub-pixel uniformity compensation architecture for foveated displays

Systems and methods are provided for using an optical crosstalk compensation (OXTC) block to compensate for optical crosstalk resulted from a combination of viewing angle change across field of view (FoV), color filter (CF) crosstalk, and the OLED various angle color shift (VACS) of a foveated electronic display. One or more two-dimensional (2D) OXTC factor maps are used to determine OXTC factors for input image data of the OXTC block, and the OXTC factors are updated on a per frame basis. Offset values are determined using a parallel architecture and used to determine the OXTC factors. Compensation weights are used to determine weighted OXTC factors to improve processing efficiency. Output image data are obtained by applying the weighted OXTC factors to the input image data.

Optical crosstalk compensation for foveated display

The present disclosure is directed towards image processing circuitry that applies temporal filtering to video image data along motion trajectories in the video image data. The temporal filtering may be applied along motion trajectories in the image data, by filtering source pixels by reference pixel values and the refined motion vectors. The temporal filtering circuitry may fetch source and reference pixel values based on received motion vectors from an encoding pipeline. Additionally, the temporal filtering circuitry may include a motion vector refinement block along with a temporal filtering block, such that the video image data may be filtered based on refined motion vectors and source and reference pixel values.

Systems and methods for motion compensated temporal filtering during image processing operations

An electronic device may include an electronic display to display an image based on processed image data. The electronic device may also include image processing circuitry to generate the processed image data. The image processing circuitry may receive input image data corresponding to an image in a first perspective and warp the input image data from the first perspective to a second perspective, generating warped image data. Additionally, the image processing circuitry may determine one or more occluded regions in the second perspective and determine fill-data corresponding to the occluded regions. The processed image data may be generated by combining the warped image data and the fill-data.

Point-of-view image warp systems and methods

A video encoding system in which pixel data is decomposed into frequency bands prior to encoding. The frequency bands are organized into blocks that are provided to a block-based encoder that encodes the blocks and passes the encoded blocks to a wireless interface that packetizes the blocks for transmittal over a wireless connection. The encoder may categorize the encoded frequency bands into multiple priority levels, and may tag each frequency block with metadata indicating the frequency band represented in the block, the priority of the frequency band, and timing information. The wireless interface may then transmit or drop packets according to the priority levels of the encoded frequency blocks in the packets and/or according to the timing information of the frequency blocks in the packets.

Priority-based video encoding and transmission

A video encoding system encodes source image data corresponding with an image includes a low resolution pipeline that receives the source image data corresponding with a first coding block in the image. The low resolution pipeline includes a low resolution motion estimation block programmed to generate a first downscaled coding block by downscaling resolution of the source image data corresponding with the first coding block. The first downscaled coding block comprises a first downscaled prediction block corresponding with a first prediction block in the first coding block. The low resolution pipeline may also perform several low resolution motion estimation searches to generate motion vector candidates. The video encoding system also includes a main pipeline that receives the source image data and determines encoding parameters to be used to encode the first coding block based at least partially on the motion vector candidates.

Systems and methods for low resolution motion estimation searches

An electronic device includes a wireless transceiver configured to receive content primitives via a wireless communication channel. The electronic device also includes control circuitry control circuitry coupled to the wireless transceiver, and configured to perform content provisioning operations based on the received content primitives, wherein the content provisioning operations comprise generating content image data and transmitting the content image data to the wireless communication channel using the wireless transceiver. In response to a bandwidth condition of the wireless communication channel being less than a threshold, the control circuitry is configured to perform adjusted content provisioning operations that decrease an amount of content image data conveyed by the wireless transceiver to the wireless communication channel.

Electronic device content provisioning adjustments based on wireless communication channel bandwidth condition

A device may include a display for displaying an image frame based on warped image data and image processing circuitry to generate the warped image data by warping input image data to account for one or more distortions associated with displaying the image. The image processing circuitry may include a two-stage cache architecture having an first cache and an second cache and warp the input image data by generating mapping data indicative of a warp between the input image space and the output image space and fetching the input image data to populate the first cache. Warping may also include populating the second cache with a grouping of pixel values from the first cache that are selected according to a sliding window that traverses the first cache based on the mapping data and interpolating between pixel values of the grouping to generate pixel values of the warped image data.

Cache architecture for image warp processing systems and methods

In one implementation, a method includes receiving a warped image representing simulated reality (SR) content (e.g., to be displayed in a display space), the warped image having a plurality of pixels at respective locations uniformly spaced in a grid pattern in a warped space, wherein the plurality of pixels are respectively associated with a plurality of respective pixel values and a plurality of respective scaling factors indicating a plurality of respective resolutions at a plurality of respective locations of the SR content (e.g., in the display space). The method includes processing the warped image in the warped space based on the plurality of respective scaling factors to generate a processed warped image and transmitting the processed warped image.

Dynamic foveated pipeline

An electronic device may include an electronic display to display an image based on processed image data. The electronic device may also include image processing circuitry to determine a hierarchical grid having multiple grid points divided into grid partitions. A first set of grid points associated with a first set of grid partitions may include a first set of mappings to corresponding coordinates of input image data in a source frame. The image processing circuitry may also interpolate between the first set of grid points to determine a second set of grid points of having a second set of mappings to corresponding coordinates of the input image data based on the first set of mappings. The image processing circuitry may also generate the processed image data by applying the first set of mappings and the second set of mappings to the input image data.

Hierarchical grid interpolation systems and methods

A mixed reality system that includes a device and a base station that communicate via a wireless connection The device may include sensors that collect information about the user's environment and about the user. The information collected by the sensors may be transmitted to the base station via the wireless connection. The base station renders frames or slices based at least in part on the sensor information received from the device, encodes the frames or slices, and transmits the compressed frames or slices to the device for decoding and display. The base station may provide more computing power than conventional stand-alone systems, and the wireless connection does not tether the device to the base station as in conventional tethered systems. The system may implement methods and apparatus to maintain a target frame rate through the wireless link and to minimize latency in frame rendering, transmittal, and display.

Video pipeline

A video encoding system in which pixel data is decomposed into frequency bands prior to encoding. The frequency bands for a slice of a frame may be buffered so that complexity statistics may be calculated across the frequency bands prior to encoding. The statistics may then be used by a rate control component in determining quantization parameters for the frequency bands for modulating the rate in the encoder for the current slice. The quantization parameters for the frequency bands may be calculated jointly to optimize the quality of the displayed frames after decoder reconstruction and wavelet synthesis on a receiving device. Information about one or more previously processed frames may be used in combination with the statistics for a current slice in determining the quantization parameters for the current slice.

Rate control for low latency video encoding and transmission

A device may include an electronic display to display an image frame based on blended image data and image processing circuitry to generate the blended image data by combining first image data and second image data via a blend operation. The blend operation may include receiving graphics alpha data indicative of a transparency factor to be applied to the first image data to generate a first layer of the blend operation. The blend operation may also include overlaying the first layer onto a second layer that is based on the second image data. Overlaying the first layer onto the second layer may include adding first pixels values of the first image data that include negative pixel values and are augmented by the transparency factor to second pixel values of the second image data to generate blended pixel values of the blended image data.

Image blending modes systems and methods

Growing community of inventors

San Jose, CA, United States of America

Jim Chen Chou