Verizon Patent and Licensing Inc.

Pai Moodlagiri

Michael Lodato

Rene Sepulveda

Denny Breitenfeld

Mohammad Raheel Khalid

Syed Meeran Kamal

Xavier Hansen

Oliver S Castaneda

Christian Egeler

Sergey Virodov

Vinay Polavarapu

Gang Lin

Adwait Ashish Murudkar

Paul Duree

William Patrick Dildine

Alexander Zilbershtein

Paul Cannon

An exemplary virtual reality system includes a management device, a synchronization device, and a gatekeeper device communicatively coupled to the management device and the synchronization device by way of a network. The gatekeeper device is configured to receive, at a particular time, a frame of a surface data frame sequence that the gatekeeper device is responsible for processing. The gatekeeper device is also configured to transmit, to the synchronization device, the particular time at which the gatekeeper device received the frame, and to receive, from the synchronization device, a timeframe during which the gatekeeper device is to transmit the frame. The gatekeeper device is further configured to transmit, to the management device during the timeframe received from the synchronization device, the frame of the surface data frame sequence. Corresponding systems and methods are also disclosed.

Systems and methods for synchronizing surface data management operations for virtual reality

An exemplary object identification system detects, based on a machine learning model, an object depicted within a video frame. The system identifies, based on the detecting of the object, a class label of the object and a region of interest, within the video frame, of the object. The system identifies, within the region of interest of the object, set of features of the object. The system compares the set of features of the object with a plurality of predefined features within a data store associated with the class label of the object. The system determines, based on the comparing of the set of features of the object with the plurality of predefined features within the data store, whether the object is configured to trigger an augmentation of video content associated with the video frame. Corresponding methods and systems are also disclosed.

Methods and systems for identification and augmentation of video content

An exemplary virtualized projection generation system receives a first frame sequence that includes frames depicting a real-world scene in accordance with a first set of capture parameters associated with a first view of the real-world scene. The virtualized projection generation system identifies a second set of capture parameters distinct from the first set of capture parameters and associated with a second view of the real-world scene distinct from the first view. Based on the first frame sequence, the virtualized projection generation system renders a second frame sequence that includes frames depicting the real-world scene in accordance with the second set of capture parameters associated with the second view of the real-world scene. Corresponding methods and systems are also disclosed.

Methods and systems for generating a customized view of a real-world scene

An exemplary method includes a virtual reality content rendering system receiving two-dimensional ('2D') color data and depth data captured by a plurality of capture devices disposed at different vantage points in relation to a three-dimensional ('3D') scene, receiving metadata, generating, for each vantage point associated with each respective capture device included in the plurality of capture devices, and based on the metadata and the depth data, a partial 3D mesh projected into a virtual 3D space to produce a partial representation of the 3D scene in the virtual 3D space, and generating, based on the partial 3D meshes projected into the virtual 3D space, and from an arbitrary viewpoint within the virtual 3D space, an image view of the virtual 3D space. The generating of the image view may include blending color samples of overlapping sections off the partial 3D meshes projected into the virtual 3D space.

Methods and systems for rendering virtual reality content based on two-dimensional (2D) captured imagery of a three-dimensional (3D) scene

An exemplary method includes a media content provider system acquiring, from a plurality of capture devices physically disposed at different vantage points in relation to a scene, a distinct set of two-dimensional (2D) color data and depth data (2D data) for each capture device included in the plurality of capture devices, each distinct set of 2D data representing a distinct unmeshed view of the scene as captured from a respective vantage point of a respective capture device included in the plurality of capture devices, acquiring metadata for each distinct set of 2D data, packaging each distinct set of 2D data into a transport stream such that the transport stream includes data representing each distinct unmeshed view of the scene as captured from each respective vantage point of each respective capture device, and providing the metadata and the transport stream for streaming to a media player device.

Methods and systems for using 2D captured imagery of a scene to provide media content

An exemplary virtual reality system provides an experience selection data structure to a media player device. The experience selection data structure comprises a plurality of entries each corresponding to a different virtual reality dataset, and each virtual reality dataset is customized to a different virtual reality experience associated with a different virtual viewpoint positioned at a different location with respect to a three-dimensional (3D) scene. The virtual reality system detects that the media player device selects an entry from the plurality of entries by way of the experience selection data structure. The selected entry corresponds to a particular virtual reality dataset customized to a particular virtual reality experience. Thus, in response to the detecting, the virtual reality system provides the particular virtual reality dataset to the media player device. Corresponding methods and systems, including details regarding corresponding media player devices, are also disclosed.

Methods and systems for customizing virtual reality data

An exemplary virtual reality system accesses metadata describing a plurality of frame sequences that each include two-dimensional color data frames and depth data frames that depict a different view of a three-dimensional (3D) scene. The virtual reality system identifies a set of experience parameters defining a particular virtual reality experience associated with the 3D scene and that is to be experienced by a user using a media player device. Based on the metadata and the set of experience parameters, the virtual reality system selects frame sequences from the plurality of frame sequences for inclusion in a frame sequence subset that is customized to the particular virtual reality experience and that collectively includes data sufficient to allow the media player device to generate a virtual 3D representation of the 3D scene. The virtual reality system also provides the selected frame sequences of the frame sequence subset to the media player device.

An exemplary virtual reality system ('system') generates an experience selection data structure configured to facilitate dynamic selection of different entries included within the experience selection data structure by a media player device as the media player device provides different virtual reality experiences to a user by processing different virtual reality datasets corresponding to different entries that the media player device selects. The system provides the experience selection data structure to the media player device and detects that the media player device selects an entry by way of the experience selection data structure. The entry corresponds to a particular virtual reality dataset that is customized to a particular virtual reality experience. In response to the selection of the entry, the system provides, to the media player device, the particular virtual reality dataset that is customized to the particular virtual reality experience.

An exemplary method includes a virtual reality media provider system acquiring, from a plurality of capture devices physically disposed at different vantage points in relation to a scene, a distinct set of two-dimensional (2D) color data and depth data (2D data) for each capture device included in the plurality of capture devices, each distinct set of 2D data representing a distinct unmeshed view of the scene from a respective vantage point of a respective capture device included in the plurality of capture devices, acquiring metadata for each distinct set of 2D data, packaging each distinct set of 2D data into a transport stream such that the transport stream includes data representing each distinct unmeshed view of the scene from each respective vantage point of each respective capture device, and providing the metadata and the transport stream for streaming to a media player device.

Methods and systems for using 2D captured imagery of a scene to provide virtual reality content

An exemplary method includes a virtual reality content rendering system receiving two-dimensional ('2D') color data and depth data captured by a plurality of capture devices disposed at different vantage points in relation to a three-dimensional ('3D') scene, receiving metadata, generating, for each vantage point associated with each respective capture device included in the plurality of capture devices, and based on the metadata and the depth data, a partial 3D mesh projected into a virtual 3D space to produce a partial representation of the 3D scene in the virtual 3D space, and generating, based on the partial 3D meshes projected into the virtual 3D space, and from an arbitrary viewpoint within the virtual 3D space, an image view of the virtual 3D space. The generating of the image view may comprise accumulating the partial 3D meshes projected into the virtual 3D space.

Methods and systems for rendering virtual reality content based on two-dimensional ('2D') captured imagery of a three-dimensional ('3D') scene

An exemplary virtualized projection generation system ('system') receives a plurality of captured surface data frame sequences each including color and depth frames depicting a real-world scene in accordance with a respective set of capture parameters included in a plurality of sets of capture parameters associated with different views of the real-world scene. The system identifies an additional set of capture parameters associated with a customized view of the real-world scene distinct from the different captured views. Based on the captured surface data frame sequences and the additional set of capture parameters, the system renders color and depth frames for a virtualized projection of the customized view of the real-world scene. The system then provides a virtualized surface data frame sequence including the rendered color and depth frames for the virtualized projection of the customized view of the real-world scene for inclusion within virtual reality media content.

Methods and systems for generating a virtualized projection of a customized view of a real-world scene for inclusion within virtual reality media content

An exemplary virtual reality system ('system') accesses metadata descriptive of a plurality of surface data frame sequences that each depict a different view of a three-dimensional (3D) scene, and identifies a set of experience parameters descriptive of a particular virtual reality experience providable to a user by a media player device that processes a particular virtual reality dataset that is customized to the particular virtual reality experience. Based on the metadata and the identified set of experience parameters, the system selects surface data frame sequences for inclusion in a frame sequence subset upon which the particular virtual reality dataset is based. The system then includes an entry corresponding to the particular virtual reality dataset within an experience selection data structure configured to facilitate dynamic selection of different entries by the media player device as the media player device provides different virtual reality experiences to the user.

An exemplary method includes a virtual reality media provider system acquiring, from a plurality of capture devices physically disposed at different vantage points in relation to a scene that includes one or more objects, surface data for the one or more objects, generating a color video data stream for the 2D color data and a depth video data stream for the depth data for each capture device included in the plurality of capture devices, packaging the color video data stream and the depth video data stream for each of the capture devices included in the plurality of capture devices into a transport stream, and providing metadata and the transport stream for streaming to a media player device.

Growing community of inventors

Clinton, NJ, United States of America

Pai Moodlagiri