Eastman-kodak Company

Intellectual Ventures Fund 83 LLC

Apple Inc.

Sen Wang

Paul James Kane

Rodney L Miller

Ronald S Cok

John N Border

Elena A Fedorovskaya

Kevin Edward Spaulding

Hou Tingbo

Majid Rabbani

Jie Yu

Lawrence B Landry

Lulu He

Tingbo Hou

Rodney Lee Miller

A method for forming a stereoscopic video of a scene from first and second input digital videos captured using respective first and second digital video cameras, wherein the first and second input digital videos include overlapping scene content and overlapping time durations. The method includes determining camera positions for each frame of the first and second input digital videos, and determining first-eye and second-eye viewpoints for each frame of the stereoscopic video. First-eye and second-eye images are formed for each frame of the stereoscopic video responsive to the corresponding video frames in the first and second input digital videos and the associated camera positions.

Forming a stereoscopic video

Controlling a head-mounted display includes providing a head-mounted display, the head-mounted display includes a switchable viewing area that is switched between a transparent viewing state and an information viewing state. The transparent viewing state is transparent with respect to the viewing area and enables a user of the head-mounted display to view the scene outside the head-mounted display in the user's line of sight. The information viewing state is opaque with respect to the viewing area and displays information in the switchable viewing area visible to a user of the head-mounted display. An external environmental state detector provides an external stimulus notification in response to a detected change in the external environment and causes the viewing state to automatically switch in response to the external stimulus notification.

Head-mounted display with environmental state detection

A method for stabilizing an input digital video. Input camera positions are determined for each of the input video frames, and an input camera path is determined representing input camera position as a function of time. A smoothing operation is applied to the input camera path to determine a smoothed camera path, and a corresponding sequence of smoothed camera positions. A stabilized video frame is determined corresponding to each of the smoothed camera positions by: selecting an input video frame having a camera position near to the smoothed camera position; warping the selected input video frame responsive to the input camera position; warping a set of complementary video frames captured from different camera positions than the selected input video frame; and combining the warped input video frame and the warped complementary video frames to form the stabilized video frame.

Method for stabilizing a digital video

A method for determining a three-dimensional model of a scene from a collection of digital images, wherein the collection includes a plurality of digital images captured from a variety of camera positions. A set of the digital images from the collection are selected, wherein each digital image contains overlapping scene content with at least one other digital image in the set of digital images, and wherein the set of digital images overlap to cover a contiguous portion of the scene. Pairs of digital images from the set of digital images to determine a camera position for each digital image. A set of target camera positions is determined to provide a set of target digital images having at least a target level of overlapping scene content. The target digital images are analyzed using a three-dimensional reconstruction process to determine a three-dimensional model of the scene.

3D scene model from collection of images

A head-mounted display apparatus includes a head-mounted display, the head-mounted display including a plurality of independently controllable switchable viewing areas that can each be independently switched between a transparent state and an information state. The transparent state enables a user of the head-mounted display to see the scene outside the head-mounted display through the independently controllable switchable viewing areas. The information state is opaque and displays information in the independently controllable switchable viewing areas visible to a user of the head-mounted display. Circuitry produces a control signal for controlling the states of the independently controllable switchable viewing areas and a controller responsive to the control signal independently switches each of the independently controllable viewing areas between the transparent state and the information state.

Switchable head-mounted display

A method of using an image capture device to identify range information for objects in a scene includes providing an image capture device having an image sensor, at least two coded apertures, and a lens; storing in a memory a set of blur parameters derived from range calibration data for each coded aperture; capturing images of the scene having a plurality of objects using each of the coded apertures; providing a set of deblurred images using the captured images from each coded aperture and each of the blur parameters from the stored set; and using the set of deblurred images to determine the range information for the objects in the scene.

Range measurement using multiple coded apertures

A method of operating a switchable head-mounted display apparatus includes the steps of providing a head-mounted display that includes a switchable viewing area that is switched between a transparent viewing state and an information viewing state, switching the viewing state to an intermediate state from the transparent viewing state or the information viewing state, and switching the intermediate state to the transparent viewing state or the information viewing state.

Switchable head-mounted display transition

A method for forming a stereoscopic image from a main image of a scene captured from a main image viewpoint including one or more foreground objects, together a main image range map and a background image. A first-eye image is determined corresponding to a first-eye viewpoint and a second-eye image is determined corresponding to a second-eye viewpoint. At least one of the first-eye image and the second-eye image is determined by warping the main image to the associated viewpoint, wherein the warped main image includes one or more holes corresponding to scene content that was occluded in the main image; warping the background image to the associated viewpoint; and determining pixel values to fill the one or more holes in the warped main image using pixel values at corresponding pixel locations in the warped background image; and forming a stereoscopic image including the first-eye image and the second-eye image.

Forming a steroscopic image using range map

An image capture device is used to identify object range information, and includes: providing an image capture device, an image sensor, a coded aperture, and a lens; and using the image capture device to capture a digital image of the scene from light passing through the lens and the coded aperture, the scene having a plurality of objects. The method further includes: dividing the digital image into a set of blocks; assigning a point spread function (psf) value to each of the blocks; combining contiguous blocks in accordance with their psf values; producing a set of blur parameters based upon the psf values of the combined blocks and the psf values of the remaining blocks; producing a set of deblurred images based upon the captured image and each of the blur parameters; and using the set of deblurred images to determine the range information for the objects in the scene.

Coded aperture camera with adaptive image processing

A system and method for adjusting perspective and disparity in a stereoscopic image pair using range information includes receiving the stereoscopic image pair representing a scene; identifying range information associated with the stereoscopic image pair and including distances of pixels in the scene from a reference location; generating a cluster map based at least upon an analysis of the range information and the stereoscopic images, the cluster map grouping pixels of the stereoscopic images by their distances from a viewpoint; identifying objects and background in the stereoscopic images based at least upon an analysis of the cluster map and the stereoscopic images; generating a new stereoscopic image pair at least by adjusting perspective and disparity of the object and the background in the stereoscopic image pair, the adjusting occurring based at least upon an analysis of the range information; and storing the new generated stereoscopic image pair in a processor-accessible memory system.

Adjusting perspective and disparity in stereoscopic image pairs

A method for determining a three-dimensional model from three or more images comprising: receiving three or more images, each image being captured from a different viewpoint and including a two-dimensional image together with a corresponding range map; designating a plurality of pairs of received images, each pair including a first image and a second image. For each of the designated pairs a geometric transform is determined by identifying a set of corresponding features in the two-dimensional images; removing any extraneous corresponding features to produce a refined set of corresponding features; and determining the geometrical transformation for transforming three-dimensional coordinates for the first image to three-dimensional coordinates for the second image responsive to three-dimensional coordinates for the refined set of corresponding features. A three-dimensional model is determined responsive to the three or more received images and the geometrical transformations for the designated pairs of received images.

Forming 3D models using multiple images

A method for determining a three-dimensional model from two images comprising: receiving first and second images captured from first and second viewpoints, respectively, each image including a two-dimensional image together with a corresponding range map; identifying a set of corresponding features in the first and second two-dimensional images; removing any extraneous corresponding features in the set of corresponding features responsive to the first and second range maps to produce a refined set of corresponding features; determining a geometrical transform for transforming three-dimensional coordinates for the first image to be consistent three-dimensional coordinates for the second image responsive to three-dimensional coordinates for the refined set of corresponding features, the three-dimensional coordinates comprising two-dimensional pixel coordinates from the corresponding two-dimensional image together with a range coordinate from the corresponding range map; and determining a three-dimensional model responsive to the first image, the second image and the geometrical transform.

Forming 3D models using two images

A method of using an image capture device to identify range information for objects in a scene, includes providing an image capture device having at least one image sensor, a lens and a coded aperture, storing in a memory a set of blur kernels derived from range calibration data for the coded aperture, capturing a first and second image of the scene having a plurality of objects, corresponding to first and second optical magnifications, respectively. The method further includes providing a set of deblurred images using the capture images from each magnification and each of the blur kernels from the stored set, and using the set of deblurred images to determine the range information for the objects in the scene.

Range measurement using a zoom camera

A method for determining a deblurred image, the method implemented at least in part by a data processing system and comprising: receiving a sharp image of a scene captured with a short exposure time; receiving a blurred image of the scene captured with a longer exposure time than the sharp image, wherein the blurred image has a higher level of motion blur and a lower level of image noise than the sharp image; determining a blur kernel responsive to the sharp image and the blurred image; determining one or more reference differential images responsive to the sharp image; determining a deblurred image responsive to the blurred image, the blur kernel and the one or more reference differential images; and storing the deblurred image in a processor-accessible memory system.

Growing community of inventors

Rochester, NY, United States of America

Sen Wang