Verizon Patent and Licensing Inc.

Xavier Hansen

Mohammad Raheel Khalid

Denny Breitenfeld

Syed Meeran Kamal

Christian Egeler

Ali Jaafar

Lama Hewage Ravi Prathapa Chandrasiri

Steven L Smith

Michael Lodato

Pai Moodlagiri

Rene Sepulveda

An exemplary scene customization system detects a selection, by a user of an artificial reality device, of a customization scheme to be applied to an original immersive scene. The system generates a custom immersive scene based on an application of the selected customization scheme to the original immersive scene. This generating of the custom immersive scene includes identifying, within a dynamic volumetric model of the original immersive scene, a particular object that is of a particular object type and is included in the original immersive scene. This generating of the custom immersive scene also includes replacing, within the dynamic volumetric model and in accordance with the applied customization scheme, the particular object with a custom object that is predetermined within the applied customization scheme to correspond to the particular object type. The system also presents the custom immersive scene to the user by way of the artificial reality device.

Methods and systems for customizing a scene for presentation to a user

An exemplary virtual reality media provider system differentiates static objects depicted in two-dimensional video data from dynamic objects depicted in the two-dimensional video data. Based on the differentiating of the static objects from the dynamic objects, the virtual reality media provider system generates dynamic volumetric models of the surfaces of the static objects and the dynamic objects. The virtual reality media provider system updates the dynamic volumetric models of the surfaces of the static objects with a lower regularity or on an as-needed basis, and the virtual reality media provider system separately updates the dynamic volumetric models of the surfaces of the dynamic objects with a higher regularity. The higher regularity is higher than the lower regularity and keeps the dynamic volumetric models of the surfaces of the dynamic objects up-to-date with what is occurring in the two-dimensional video data. Corresponding methods and systems are also disclosed.

Methods and systems for creating a volumetric representation of a real-world event

An exemplary virtual reality media provider system receives two-dimensional ('2D') video data for surfaces of first and second objects located in a natural setting. The 2D video data is captured by first and second capture devices disposed at different positions with respect to the objects. The system distinguishes the first object from the second object by performing a plurality of techniques in combination with one another. The plurality of techniques include determining that the first object is moving in relation to the second object; and determining that, from a vantage point of at least one of the different positions, a representation of the first object captured within the 2D video data does not overlap with a representation of the second object. Based on the received 2D video data and the distinguishing of the first and second objects, the system generates an individually-manipulable volumetric model of the first object.

Methods and systems for distinguishing objects in a natural setting to create an individually-manipulable volumetric model of an object

An exemplary method includes a media content provider system acquiring, from a plurality of capture devices physically disposed at different vantage points in relation to a scene, a distinct set of two-dimensional (2D) color data and depth data (2D data) for each capture device included in the plurality of capture devices, each distinct set of 2D data representing a distinct unmeshed view of the scene as captured from a respective vantage point of a respective capture device included in the plurality of capture devices, acquiring metadata for each distinct set of 2D data, packaging each distinct set of 2D data into a transport stream such that the transport stream includes data representing each distinct unmeshed view of the scene as captured from each respective vantage point of each respective capture device, and providing the metadata and the transport stream for streaming to a media player device.

Methods and systems for using 2D captured imagery of a scene to provide media content

An exemplary method includes a virtual reality media provider system acquiring, from a plurality of capture devices physically disposed at different vantage points in relation to a scene, a distinct set of two-dimensional (2D) color data and depth data (2D data) for each capture device included in the plurality of capture devices, each distinct set of 2D data representing a distinct unmeshed view of the scene from a respective vantage point of a respective capture device included in the plurality of capture devices, acquiring metadata for each distinct set of 2D data, packaging each distinct set of 2D data into a transport stream such that the transport stream includes data representing each distinct unmeshed view of the scene from each respective vantage point of each respective capture device, and providing the metadata and the transport stream for streaming to a media player device.

Methods and systems for using 2D captured imagery of a scene to provide virtual reality content

An exemplary scene customization system detects a selection, by a user, of a customization scheme to be applied to an original immersive scene throughout a scene customization session to be experienced by the user by way of an artificial reality device. The system dynamically generates a custom immersive scene based on an application of the selected customization scheme to the original immersive scene by receiving data representative of the original immersive scene, developing a dynamic volumetric model of the original immersive scene based on the received data representative of the original immersive scene, identifying one or more objects included in the original immersive scene, and replacing the one or more objects with one or more custom objects in accordance with the applied customization scheme. The system also presents the custom immersive scene to the user by way of the artificial reality device throughout the scene customization session.

Methods and systems for dynamically customizing a scene for presentation to a user

An exemplary method includes a media player device ('device') providing a user with an immersive virtual reality experience in accordance with a specification file corresponding to the immersive virtual reality experience. The specification file includes data that defines a plurality of elements included in the immersive virtual reality experience by providing a plurality of links for use by the device in acquiring the plurality of elements while providing the user with the immersive virtual reality experience. The method further includes the device detecting, while the immersive virtual reality experience is being provided to the user, real-world input associated with the user, and integrating the real-world input into the immersive virtual reality experience by updating the specification file to further include data that defines the real-world input as a user-specific element that is specific to the user and that is included in the immersive virtual reality experience.

Methods and systems for representing real-world input as a user-specific element in an immersive virtual reality experience

An exemplary virtual reality media provider system ('system') includes a configuration of synchronous video and depth capture devices disposed at fixed positions at a real-world event. In real time, the video and depth capture devices capture two-dimensional video data and depth data for surfaces of objects at the real-world event. The system generates a real-time volumetric data stream representative of a dynamic volumetric model of the surfaces of the objects at the real-world event in real time based on the captured two-dimensional video data and captured depth data. The dynamic volumetric model of the surfaces of the objects at the real-world event is configured to be used to generate virtual reality media content representative of the real-world event as experienced from a dynamically selectable viewpoint corresponding to an arbitrary location at the real-world event and selected by a user experiencing the real-world event using a media player device.

Methods and systems for creating and providing a real-time volumetric representation of a real-world event

An exemplary virtual reality media provider system ('system') includes a configuration of synchronous video and depth capture devices disposed at fixed positions in a vicinity of a first object located in a natural setting along with one or more additional objects. The video and depth capture devices capture two-dimensional video data and depth data for a surface the first object. The system distinguishes the first object from a second object included in the one or more additional objects located in the natural setting and generates an individually-manipulable volumetric model of the first object. The individually-manipulable volumetric model of the first object is configured to be individually manipulated with respect to an immersive virtual reality world while a user of a media player device is experiencing the immersive virtual reality world using the media player device.

Methods and systems for creating and manipulating an individually-manipulable volumetric model of an object

An exemplary method includes a virtual reality media provider system acquiring, from a plurality of capture devices physically disposed at different vantage points in relation to a scene that includes one or more objects, surface data for the one or more objects, generating a color video data stream for the 2D color data and a depth video data stream for the depth data for each capture device included in the plurality of capture devices, packaging the color video data stream and the depth video data stream for each of the capture devices included in the plurality of capture devices into a transport stream, and providing metadata and the transport stream for streaming to a media player device.

Growing community of inventors

Parsippany, NJ, United States of America

Xavier Hansen