Verizon Patent and Licensing Inc.

Michael Lodato

Pai Moodlagiri

Rene Sepulveda

Denny Breitenfeld

Oliver S Castaneda

Mohammad Raheel Khalid

Syed Meeran Kamal

Xavier Hansen

An exemplary virtual reality system includes a management device, a synchronization device, and a gatekeeper device communicatively coupled to the management device and the synchronization device by way of a network. The gatekeeper device is configured to receive, at a particular time, a frame of a surface data frame sequence that the gatekeeper device is responsible for processing. The gatekeeper device is also configured to transmit, to the synchronization device, the particular time at which the gatekeeper device received the frame, and to receive, from the synchronization device, a timeframe during which the gatekeeper device is to transmit the frame. The gatekeeper device is further configured to transmit, to the management device during the timeframe received from the synchronization device, the frame of the surface data frame sequence. Corresponding systems and methods are also disclosed.

Systems and methods for synchronizing surface data management operations for virtual reality

An exemplary virtualized projection generation system receives a first frame sequence that includes frames depicting a real-world scene in accordance with a first set of capture parameters associated with a first view of the real-world scene. The virtualized projection generation system identifies a second set of capture parameters distinct from the first set of capture parameters and associated with a second view of the real-world scene distinct from the first view. Based on the first frame sequence, the virtualized projection generation system renders a second frame sequence that includes frames depicting the real-world scene in accordance with the second set of capture parameters associated with the second view of the real-world scene. Corresponding methods and systems are also disclosed.

Methods and systems for generating a customized view of a real-world scene

An exemplary method includes a virtual reality content rendering system receiving two-dimensional ('2D') color data and depth data captured by a plurality of capture devices disposed at different vantage points in relation to a three-dimensional ('3D') scene, receiving metadata, generating, for each vantage point associated with each respective capture device included in the plurality of capture devices, and based on the metadata and the depth data, a partial 3D mesh projected into a virtual 3D space to produce a partial representation of the 3D scene in the virtual 3D space, and generating, based on the partial 3D meshes projected into the virtual 3D space, and from an arbitrary viewpoint within the virtual 3D space, an image view of the virtual 3D space. The generating of the image view may include blending color samples of overlapping sections off the partial 3D meshes projected into the virtual 3D space.

Methods and systems for rendering virtual reality content based on two-dimensional (2D) captured imagery of a three-dimensional (3D) scene

An exemplary virtual reality provider system generates a surface data projection based on surface data representative of color characteristics or depth characteristics of surfaces within a three-dimensional ('3D') space. The surface data projection is generated for a portion of the 3D space. The virtual reality provider system applies an image transform operation to the surface data projection to transform the surface data projection. Specifically, the image transform operation transforms the surface data projection to account for a level of detail of the surfaces within the 3D space with respect to a particular vantage point within the 3D space. Corresponding methods and systems are also disclosed.

Methods and systems for generating a surface data projection that accounts for level of detail

An exemplary method includes a media content provider system acquiring, from a plurality of capture devices physically disposed at different vantage points in relation to a scene, a distinct set of two-dimensional (2D) color data and depth data (2D data) for each capture device included in the plurality of capture devices, each distinct set of 2D data representing a distinct unmeshed view of the scene as captured from a respective vantage point of a respective capture device included in the plurality of capture devices, acquiring metadata for each distinct set of 2D data, packaging each distinct set of 2D data into a transport stream such that the transport stream includes data representing each distinct unmeshed view of the scene as captured from each respective vantage point of each respective capture device, and providing the metadata and the transport stream for streaming to a media player device.

Methods and systems for using 2D captured imagery of a scene to provide media content

An exemplary virtual reality system provides an experience selection data structure to a media player device. The experience selection data structure comprises a plurality of entries each corresponding to a different virtual reality dataset, and each virtual reality dataset is customized to a different virtual reality experience associated with a different virtual viewpoint positioned at a different location with respect to a three-dimensional (3D) scene. The virtual reality system detects that the media player device selects an entry from the plurality of entries by way of the experience selection data structure. The selected entry corresponds to a particular virtual reality dataset customized to a particular virtual reality experience. Thus, in response to the detecting, the virtual reality system provides the particular virtual reality dataset to the media player device. Corresponding methods and systems, including details regarding corresponding media player devices, are also disclosed.

Methods and systems for customizing virtual reality data

An exemplary virtual reality system accesses metadata describing a plurality of frame sequences that each include two-dimensional color data frames and depth data frames that depict a different view of a three-dimensional (3D) scene. The virtual reality system identifies a set of experience parameters defining a particular virtual reality experience associated with the 3D scene and that is to be experienced by a user using a media player device. Based on the metadata and the set of experience parameters, the virtual reality system selects frame sequences from the plurality of frame sequences for inclusion in a frame sequence subset that is customized to the particular virtual reality experience and that collectively includes data sufficient to allow the media player device to generate a virtual 3D representation of the 3D scene. The virtual reality system also provides the selected frame sequences of the frame sequence subset to the media player device.

An exemplary virtual reality provider system accesses surface data representative of a virtual three-dimensional ('3D') space. Based on the accessed surface data, the system projects a plurality of adjacent surface data slices of the virtual 3D space along an axis in a coordinate system associated with the virtual 3D space. The system selects an image transform operation to be applied to a surface data slice in the plurality of adjacent surface data slices. Specifically, the selected image transform operation is configured to transform the surface data slice to account for level of detail of the surfaces within the virtual 3D space with respect to a particular vantage point. The system also applies the selected image transform operation to the surface data slice to transform the surface data slice to account for the level of detail with respect to the particular vantage point. Corresponding methods and systems are also disclosed.

Methods and systems for generating virtual reality data that accounts for level of detail

An exemplary virtual reality system ('system') generates an experience selection data structure configured to facilitate dynamic selection of different entries included within the experience selection data structure by a media player device as the media player device provides different virtual reality experiences to a user by processing different virtual reality datasets corresponding to different entries that the media player device selects. The system provides the experience selection data structure to the media player device and detects that the media player device selects an entry by way of the experience selection data structure. The entry corresponds to a particular virtual reality dataset that is customized to a particular virtual reality experience. In response to the selection of the entry, the system provides, to the media player device, the particular virtual reality dataset that is customized to the particular virtual reality experience.

An exemplary method includes a virtual reality media provider system acquiring, from a plurality of capture devices physically disposed at different vantage points in relation to a scene, a distinct set of two-dimensional (2D) color data and depth data (2D data) for each capture device included in the plurality of capture devices, each distinct set of 2D data representing a distinct unmeshed view of the scene from a respective vantage point of a respective capture device included in the plurality of capture devices, acquiring metadata for each distinct set of 2D data, packaging each distinct set of 2D data into a transport stream such that the transport stream includes data representing each distinct unmeshed view of the scene from each respective vantage point of each respective capture device, and providing the metadata and the transport stream for streaming to a media player device.

Methods and systems for using 2D captured imagery of a scene to provide virtual reality content

An exemplary method includes a virtual reality content rendering system receiving two-dimensional ('2D') color data and depth data captured by a plurality of capture devices disposed at different vantage points in relation to a three-dimensional ('3D') scene, receiving metadata, generating, for each vantage point associated with each respective capture device included in the plurality of capture devices, and based on the metadata and the depth data, a partial 3D mesh projected into a virtual 3D space to produce a partial representation of the 3D scene in the virtual 3D space, and generating, based on the partial 3D meshes projected into the virtual 3D space, and from an arbitrary viewpoint within the virtual 3D space, an image view of the virtual 3D space. The generating of the image view may comprise accumulating the partial 3D meshes projected into the virtual 3D space.

Methods and systems for rendering virtual reality content based on two-dimensional ('2D') captured imagery of a three-dimensional ('3D') scene

An exemplary virtual reality provider system accesses surface data representative of a virtual three-dimensional ('3D') space of a virtual scene. Based on the accessed surface data, the system orthographically projects a respective plurality of adjacent surface data slices of the virtual 3D space along each of three orthogonal axes in a coordinate system associated with the virtual 3D space. The system generates virtual reality data that represents the virtual 3D space and accounts for level of detail of the surfaces included within the virtual 3D space with respect to a particular vantage point. The system provides this virtual reality data to a media player device associated with the particular vantage point for processing by the media player device to present, to a user, virtual reality content that is based on the virtual 3D space and is tailored to the particular vantage point.

Methods and systems for generating and providing virtual reality data that accounts for level of detail

An exemplary virtualized projection generation system ('system') receives a plurality of captured surface data frame sequences each including color and depth frames depicting a real-world scene in accordance with a respective set of capture parameters included in a plurality of sets of capture parameters associated with different views of the real-world scene. The system identifies an additional set of capture parameters associated with a customized view of the real-world scene distinct from the different captured views. Based on the captured surface data frame sequences and the additional set of capture parameters, the system renders color and depth frames for a virtualized projection of the customized view of the real-world scene. The system then provides a virtualized surface data frame sequence including the rendered color and depth frames for the virtualized projection of the customized view of the real-world scene for inclusion within virtual reality media content.

Methods and systems for generating a virtualized projection of a customized view of a real-world scene for inclusion within virtual reality media content

An exemplary virtual reality system ('system') accesses metadata descriptive of a plurality of surface data frame sequences that each depict a different view of a three-dimensional (3D) scene, and identifies a set of experience parameters descriptive of a particular virtual reality experience providable to a user by a media player device that processes a particular virtual reality dataset that is customized to the particular virtual reality experience. Based on the metadata and the identified set of experience parameters, the system selects surface data frame sequences for inclusion in a frame sequence subset upon which the particular virtual reality dataset is based. The system then includes an entry corresponding to the particular virtual reality dataset within an experience selection data structure configured to facilitate dynamic selection of different entries by the media player device as the media player device provides different virtual reality experiences to the user.

An exemplary method includes a virtual reality media provider system acquiring, from a plurality of capture devices physically disposed at different vantage points in relation to a scene that includes one or more objects, surface data for the one or more objects, generating a color video data stream for the 2D color data and a depth video data stream for the depth data for each capture device included in the plurality of capture devices, packaging the color video data stream and the depth video data stream for each of the capture devices included in the plurality of capture devices into a transport stream, and providing metadata and the transport stream for streaming to a media player device.

Growing community of inventors

New Egypt, NJ, United States of America

Michael Lodato