Lucent Technologies Inc.

John T Edmark

James Robert Ensor

Gianpaolo U Carraro

Although in my co-pending application, so-called &ldquo;pyramidic panel structures&rdquo; are employed to adjust the image's vanishing point(s) in accordance with the movement of the user, such structures have the disadvantage that they work primarily for so-called &ldquo;corridor-like&rdquo; two-dimensional images. I have recognized that, in accordance with the principles of the invention, viewpoint changes may be also dealt with by partitioning the two-dimensional image into polygon partitions, each corresponding to a surface being depicted in the image, and then coupling the vertices of the polygon partitions with the movement of the user so as to limit distortions. In this latter approach, two-dimensional images are not treated as a perspective, corridor-like image, thereby allowing a wider variety of two-dimensional images to be used with the present invention.

Display techniques for three-dimensional virtual reality

A limitation of using two-dimensional images, such as videos or photographs, to represent portions of a three-dimensional world occurs when the user moves within the world and views the world from a location different than from the original context of the two-dimensional image, i.e., from a location different than the image's ideal viewing point (IVP). View changes result in the image not aligning well with the surrounding objects of the three-dimensional world. This limitation is overcome by distorting the two-dimensional image so as to adjust the image's vanishing point(s) in accordance with the movement of the user using a pyramidic panel structure. In this manner, as the user moves away from the ideal viewing point, the distortions act to limit the discontinuities between the two-dimensional image and its surroundings. To minimize the depth profile of the pyramidic panel structure, the structure may be segmented into sections and each section translated towards, or away from, the user's viewpoint. Also, a hierarchical image resolution may be used, with portions of the image near the center or vanishing point having a higher resolution than the portions of the image near its perimeter.

A limitation of modeling three-dimensional objects in a virtual world occurs when a user moves within the world and still wishes a certain target object to remain visible, instead of being occluded by other surfaces in front of the object along the direction the user is viewing the target object. Therefore, in accordance with the principles of the invention, a target object which is to remain visible, regardless of the user's view point, is duplicated and modeled as being located a close distance to the user, but scaled accordingly to make the cloned object appear to be farther away from the viewpoint of the user and co-located with and of the same size and orientation as the target object. In this manner, as the user moves within the world, surfaces located in front of the target object do not occlude the cloned object from the user's view, thus appearing to give that same property to the target object.

Display techniques for object visibility in a three-dimensional virtual world

When there is an inability to convey motion that is occurring in the frames of a video source by a full-frame-rate transmission, a surrogate effect, other than another form of motion, may be used as a compensation technique to better convey motion to a viewer. The surrogate effect employed may be a) fading, b) wiping, c) dissolving, d) blurring, e) enhancing the contrast, f) enhancing one or more colors, g) enhancing the brightness, h) scaling the image, and i) the like. How the surrogate effect is applied to any frame may be a function of one or more video frames. Optionally, more than one effect may be used in combination. Advantageously, a more continuous sense of motion is perceived by a viewer. In one embodiment of the invention, full-frame-rate video is initially available. A controller monitors the ability to transmit or display full-frame-rate video. In the event that it is determined that, ultimately, full-frame-rate video cannot be displayed to a user, the images of the frames that will be displayed are adjusted to incorporate a surrogate effect that helps to better convey the intended motion to the viewer. The surrogate effect applied may be selected as a joint function of the available frame rate and knowledge of the content, e.g., the specific type of motion, intended to be shown to the viewer.

Motion display technique

A limitation of using two-dimensional images, such as videos or photographs, to represent portions of a three-dimensional world occurs when the user moves within the world and views the world from a location different than from the original context of the two-dimensional image, i.e., from a location different than the image's ideal viewing point (IVP). View changes result in the image not aligning well with the surrounding objects of the three-dimensional world. This limitation is overcome by distorting the two-dimensional image so as to adjust the image's vanishing point(s) in accordance with the movement of the user using a pyramidic panel structure. In this manner, as the user moves away from the ideal viewing point, the distortions act to limit the discontinuities between the two-dimensional image and its surroundings. Various pyramidic panel structures may be constructed, including an octahedral panel structure which surrounds the user's view point so as to provide a so-called &ldquo;plenoptic&rdquo; view of the world. Also, to minimize the depth profile of the pyramidic panel structure, the structure may be segmented into sections and each section translated towards, or away from, the user's viewpoint. Also, a hierarchical image resolution may be used, with portions of the image near the center or vanishing point having a higher resolution than portions of the image near its perimeter.

A limitation of a three-dimensional world in which objects in the distance may be represented in only two dimensions as a video on a screen occurs when a computer graphic object represented by computer graphics, e.g., in front of, to the side, above, or below the video screen, undergoes a trajectory that takes it to a location in the world that is not represented as computer graphics, but instead is within the field represented by the video, because such an object would disappear from view by the viewer. This limitation is overcome by having by determining when a computer graphic object undergoes a trajectory that takes it to a location in the world that is not represented as computer graphics, but instead is within the field represented by the video, and representing such an object as computer graphics in front of the screen to which at least one perspective transformation is applied, the perspective transformation being at least a function of the position at which the object is actually located.

A limitation of using two-dimensional images, such as videos or photographs, to represent portions of a three-dimensional world occurs when the user moves within the world and views the world from a location different than from the original context of the two-dimensional image, i.e., from a location different than the image's ideal viewing point (IVP). View changes result in the image not aligning well with the surrounding objects of the three-dimensional world. This limitation is overcome by distorting the two-dimensional image so as to adjust the image's vanishing point(s) in accordance with the movement of the user using a pyramidic panel structure. In this manner, as the user moves away from the ideal viewing point, the distortions act to limit the discontinuities between the two-dimensional image and its surroundings. To minimize the depth profile of the pyramidic panel structure, the structure may be segmented into sections and each section translated towards, or away from, the user's viewpoint.

A limitation of using two-dimensional images, such as videos or photographs, to represent portions of a three-dimensional world occurs when the user moves within the world and views the world from a location different than from the original context of the two-dimensional image, i.e., from a location different than the image's ideal viewing point (IVP). View changes result in the image not aligning well with the surrounding objects of the three-dimensional world. This limitation is overcome by distorting the two-dimensional image so as to adjust the image's vanishing point(s) in accordance with the movement of the user. In this manner, as the user moves away from the ideal viewing point, the distortions act to limit the discontinuities between the two-dimensional image and its surroundings.

Growing community of inventors

New York, NY, United States of America

John T Edmark