Samsung Electronics Co., Ltd.

The Texas A&m University System

Korea Advanced Institute of Science and Technology

Aarhus Universitet

Inwoo Ha

Hyong Euk Lee

Minsu Ahn

Seungin Park

Minjung Son

Nahyup Kang

Hyun Sung Chang

Sunghoon Hong

Keechang Lee

Young Hun Sung

Heesae Lee

Kyungboo Jung

Dokyoon Kim

Taehyun Rhee

Xiaolin Wei

Jinxiang Chai

Huajun Liu

Jinwoo Park

Jiyeon Kim

Wonhee Lee

Donghoon Sagong

Hwiryong Jung

Seungeon Kim

Jaeyoon Kim

Junhwan Lee

Jacob Toft Pedersen

Hachisuka Toshiya

Changho Jo

Sungeui Yoon

A processor-implemented method with pose estimation includes: determining depth data by sensing a depth of a target scene comprising planes orthogonal to each other; determining normal vectors corresponding to depth points of the depth data; determining orientation data of a device by comparing the normal vectors to orientation candidates; and determining position data of the device based on distances from the device to the planes.

Method and apparatus with pose estimation

A method and an apparatus with image rendering are provided. A method includes generating a first-rendering result corresponding to input graphic data of a rendering target, wherein the first-rendering result is rendered in units of first tiles by performing first-rendering including a first type of operation using a first processing unit, storing, in memory, the first-rendering result as units of first tiles, generating a second-rendering result of second tiles by, for each first tile of the first-rendering result, performing second-rendering including a second type of operation using a second processing unit, storing, in the memory, the second-rendering result as units of second tiles, and determining a final rendering result based on the second-rendering result.

Method and apparatus with image rendering based on tiles

A method and device with light estimation are provided. A method performed by an electronic device includes generating a reference image based on image data acquired by capturing a reference object and based on a first image signal processing (ISP) setting, generating a background image based on raw image data acquired by capturing a real background in which the reference object is positioned and based on a second ISP setting, estimating light information corresponding to the background image using a light estimation model, rendering a virtual object image corresponding to the light information and the reference object, and training the light estimation model based on a difference between the reference image and the virtual object image.

Method and device for light estimation

A processor-implemented method for light estimation includes: estimating light information corresponding to an input image using a light estimation model; detecting a reference object in the input image; determining object information of the reference object and plane information of a reference plane supporting the reference object; rendering a virtual object corresponding to the reference object based on the light information, the object information, and the plane information; and training the light estimation model by updating the light estimation model based on a result of comparing the reference object and the rendered virtual object.

Method and apparatus for light estimation

A processor-implemented method includes: determining albedo data in a canonical space and depth data in the canonical space based on input image data including an object, using one or more neural network-based extraction models; generating deformed albedo data and deformed depth data by applying a target shape deformation value respectively to the albedo data and the depth data; generating resultant shaded data by performing shading based on the deformed depth data and a target illumination value; generating intermediate image data based on the resultant shaded data and the deformed albedo data; and generating reconstructed image data from the intermediate image data and the deformed depth data based on a target pose value.

Method and apparatus with image processing and reconstructed image generation

A three-dimensional (3D) rendering method includes extracting sample points from a 3D scene, acquiring rendering result information for the sample points by rendering the sample points, and generating a rendering result image corresponding to an entire rendering based on the rendering result information for the sample points and feature information of the 3D scene.

Three-dimensional (3D) rendering method and apparatus

An image processing method is provided. The method includes generating appearance information corresponding to a canonical space defined by a cube including a plurality of regular grids in a first input image by implementing a first neural network, generating geometric information corresponding to a standard space in a second input image by implementing a second neural network, deforming the plurality of regular grids based on the generated geometric information, generating standardized image information by applying the appearance information to the plurality of deformed regular grids, and generating an output image based on the standardized image information.

Image processing method and apparatus

An image processing method and apparatus with an integrated framework of heterogeneous features is provided. The method includes generating first feature frames including first features corresponding to a first density level by sensing a target space; generating second feature frames including second features corresponding to a second density level by sensing the target space; estimating sensor poses corresponding to at least a portion of the generated first feature frames and the generated second feature frames; calculating each of a first reprojection error of the first features and a second reprojection error of the second features based on the estimated sensor poses; and acquiring an optimization result by adjusting at least a portion of the sensor poses, the first features, the second features, reprojection features of the first features, and reprojection features of the second features to decrease an integration error including the first reprojection error and the second reprojection error.

Image processing method and apparatus with integrated optimization framework of heterogeneous features

An method of operating a display apparatus is provided. The operating method includes calculating a confidence of a result obtained by performing light field rendering on each of pixels of a display panel based on positions of a left eye and a right eye, determining a weight kernel of a corresponding pixel based on the confidence, and adjusting a brightness of a pixel corresponding to each of the left eye and the right eye based on the weight kernel.

Display apparatus and operating method of the same

A method and apparatus for processing augmented reality (AR) are disclosed. The method includes determining a compensation parameter to compensate for light attenuation of visual information caused by a display area of an AR device as the visual information corresponding to a target scene is displayed through the display area, generating a background image without the light attenuation by capturing the target scene using a camera of the AR device, generating a compensation image by reducing brightness of the background image using the compensation parameter, generating a virtual object image to be overlaid on the target scene, generating a display image by synthesizing the compensation image and the virtual object image, and displaying the display image in the display area.

Processing method and apparatus with augmented reality

A display apparatus and an operating method thereof obtain at least one first sensor data of a first point in time, receive a first image frame including a first augmented reality (AR) object corresponding to the first point in time, the first AR object generated by a first processor, based on the first sensor data, obtain second sensor data of a second point in time subsequent to the first point in time, correct the first AR object to a second AR object matched to the second point in time, based on the second sensor data, and display a second image frame including the second AR object.

Apparatus and operating method for displaying augmented reality object

A processor-implemented method with crosstalk correction includes: determining a region in which a crosstalk is to occur based on a three-dimensional (3D) position relationship between a position of eyes of a user and a position of a virtual image of a virtual content object; generating a concealer image for correcting the region in which the crosstalk is to occur based on the determined region and the virtual content object; and correcting the crosstalk by combining the virtual content object and the generated concealer image.

Method and apparatus with crosstalk correction

A processor-implemented light source information output method includes: receiving an input image; detecting, using a trained neural network, at least one object in the input image; estimating, using the trained neural network, light source information of a light source corresponding to the at least one object; and outputting the light source information.

Method and apparatus with light source information output

A method of playing back an image in a display device obtains a stereo image corresponding to both eyes of a user including a left eye and a right eye, obtains positions of the eyes determines warping transform information corresponding to the positions of the eyes based on a database (DB) provided in advance based on a curvature of an display plane of the display device, warps the stereo image by applying the warping transform information to the stereo image, determines pixel values of a display panel included in the display device from the warped stereo image by performing light field rendering based on the positions of the eyes, and reproduces the determined pixel values on the display panel.

Method of playing back image on display device and display device

An image processing apparatus includes a first processor configured to obtain, from a color image, an illumination element image and an albedo element image corresponding to the color image, and a second processor configured to divide the illumination element image into a plurality of subelement images each corresponding to the color image.

Growing community of inventors

Seongnam-si, South Korea

Inwoo Ha