Verizon Patent and Licensing Inc.

Fusar Technologies, Inc.

Steven L Smith

Mohammad Raheel Khalid

Syed Meeran Kamal

Lama Hewage Ravi Prathapa Chandrasiri

Denny Breitenfeld

Ali Jaafar

Christian Egeler

Xavier Hansen

Sergey Virodov

Yongsheng Pan

Jonathan A Globerson

Ryan T Shearman

Daniel R Bersak

Todd H Rushing

An exemplary depth capture system emits a first structured light pattern onto a surface of an object included in a real-world scene using a first structured light emitter included within the depth capture system. The depth capture system also emits, concurrently with the emitting of the first structured light pattern, a second structured light pattern onto the surface of the object using a second structured light emitter included within the depth capture system. The emitting of the second structured light pattern is different than the emitting of the first structured light pattern. The depth capture system detects the first and second structured light patterns using one or more optical sensors included within the depth capture system. Based on the detection of the structured light patterns, the depth capture system generates depth data representative of the surface of the object included in the real-world scene.

Methods and systems for capturing depth data using structured light

An exemplary virtual reality media provider system differentiates static objects depicted in two-dimensional video data from dynamic objects depicted in the two-dimensional video data. Based on the differentiating of the static objects from the dynamic objects, the virtual reality media provider system generates dynamic volumetric models of the surfaces of the static objects and the dynamic objects. The virtual reality media provider system updates the dynamic volumetric models of the surfaces of the static objects with a lower regularity or on an as-needed basis, and the virtual reality media provider system separately updates the dynamic volumetric models of the surfaces of the dynamic objects with a higher regularity. The higher regularity is higher than the lower regularity and keeps the dynamic volumetric models of the surfaces of the dynamic objects up-to-date with what is occurring in the two-dimensional video data. Corresponding methods and systems are also disclosed.

Methods and systems for creating a volumetric representation of a real-world event

An exemplary virtual reality media provider system receives two-dimensional ('2D') video data for surfaces of first and second objects located in a natural setting. The 2D video data is captured by first and second capture devices disposed at different positions with respect to the objects. The system distinguishes the first object from the second object by performing a plurality of techniques in combination with one another. The plurality of techniques include determining that the first object is moving in relation to the second object; and determining that, from a vantage point of at least one of the different positions, a representation of the first object captured within the 2D video data does not overlap with a representation of the second object. Based on the received 2D video data and the distinguishing of the first and second objects, the system generates an individually-manipulable volumetric model of the first object.

Methods and systems for distinguishing objects in a natural setting to create an individually-manipulable volumetric model of an object

An exemplary depth data generation system accesses first and second depth maps of a real-world scene, the depth maps independently captured using first and second depth map capture techniques, respectively. The first and second depth maps include, respectively, first and second depth data points both representative of a same physical point on a surface of an object in the real-world scene. Based on the first and second depth map capture techniques and based on an attribute of the surface of the object, the system assigns a first confidence value to the first depth data point and a second confidence value to the second depth data point. Based on the first and second confidence values, the system converges the first and second depth maps to form a converged depth map of the real-world scene that includes a third depth data point representing the physical point on the surface of the object.

Methods and systems for generating depth data by converging independently-captured depth maps

An exemplary depth scanning system includes first and second depth scanning devices disposed, respectively, at first and second locations on a boundary of a scan zone where segments of the boundary meet to form respective first and second angles that include the scan zone. The first device performs a first sequence of depth scanning operations where the first device detects depth data for surfaces included in the scan zone by sweeping a scan field across the first angle and in which the first device abstains from detecting depth data for surfaces outside the first angle. Concurrently, the second device performs a second sequence of depth scanning operations where the second device detects depth data for the surfaces in the scan zone by sweeping a scan field across the second angle and in which the second device abstains from detecting depth data for surfaces outside the second angle.

Systems and methods for inward-looking depth scanning of a scan zone

An exemplary method includes a media player device ('device') providing a user with an immersive virtual reality experience in accordance with a specification file corresponding to the immersive virtual reality experience. The specification file includes data that defines a plurality of elements included in the immersive virtual reality experience by providing a plurality of links for use by the device in acquiring the plurality of elements while providing the user with the immersive virtual reality experience. The method further includes the device detecting, while the immersive virtual reality experience is being provided to the user, real-world input associated with the user, and integrating the real-world input into the immersive virtual reality experience by updating the specification file to further include data that defines the real-world input as a user-specific element that is specific to the user and that is included in the immersive virtual reality experience.

Methods and systems for representing real-world input as a user-specific element in an immersive virtual reality experience

An exemplary depth scanning system includes first and second depth scanning devices. The first device detects depth data for surfaces included within a real-world scene from a first vantage point by rotating a first scan head in one rotational direction to progressively aim a first scan field to sweep across the surfaces during a scan time period. The second device detects depth data for the surfaces from a second vantage point by rotating a second scan head in phase with the rotation of the first scan head and in the opposite rotational direction so as to progressively aim a second scan field to sweep across the surfaces during the scan time period. The in-phase rotations of the first and second scan heads cause the first scan field to aim at the second vantage point at a same moment when the second scan field aims at the first vantage point.

Systems and methods for inward-looking depth scanning of a real-world scene

An exemplary depth data generation system ('system') accesses a first depth map and a second depth map of surfaces of objects included in a real-world scene. The first and second depth maps are captured independently from one another. The system converges the first and second depth maps into a converged depth map of the surfaces of the objects included in the real-world scene. More specifically, the converging comprises assigning a first confidence value to a first depth data point in the first depth map, assigning a second confidence value to a second depth data point in the second depth map, and generating a third depth data point representing a same particular physical point as the first and second depth data points based on the first and second confidence values and on at least one of the first depth data point and the second depth data point.

An exemplary virtual reality media provider system ('system') includes a configuration of synchronous video and depth capture devices disposed at fixed positions at a real-world event. In real time, the video and depth capture devices capture two-dimensional video data and depth data for surfaces of objects at the real-world event. The system generates a real-time volumetric data stream representative of a dynamic volumetric model of the surfaces of the objects at the real-world event in real time based on the captured two-dimensional video data and captured depth data. The dynamic volumetric model of the surfaces of the objects at the real-world event is configured to be used to generate virtual reality media content representative of the real-world event as experienced from a dynamically selectable viewpoint corresponding to an arbitrary location at the real-world event and selected by a user experiencing the real-world event using a media player device.

Methods and systems for creating and providing a real-time volumetric representation of a real-world event

An exemplary virtual reality media provider system ('system') includes a configuration of synchronous video and depth capture devices disposed at fixed positions in a vicinity of a first object located in a natural setting along with one or more additional objects. The video and depth capture devices capture two-dimensional video data and depth data for a surface the first object. The system distinguishes the first object from a second object included in the one or more additional objects located in the natural setting and generates an individually-manipulable volumetric model of the first object. The individually-manipulable volumetric model of the first object is configured to be individually manipulated with respect to an immersive virtual reality world while a user of a media player device is experiencing the immersive virtual reality world using the media player device.

Methods and systems for creating and manipulating an individually-manipulable volumetric model of an object

An exemplary depth capture system ('system') emits, from a first fixed position with respect to a real-world scene and within a first frequency band, a first structured light pattern onto surfaces of objects included in a real-world scene. The system also emits, from a second fixed position with respect to the real-world scene and within a second frequency band, a second structured light pattern onto the surfaces of the objects. The system detects the first and second structured light patterns using one or more optical sensors by way of first and second optical filters, respectively. The first and second optical filters are each configured to only pass one of the structured light patterns and to block the other. Based on the detection of the structured light patterns, the system generates depth data representative of the surfaces of the objects included in the real-world scene.

Methods and systems for capturing depth data using frequency-segregated structured light

One variation of a helmet system includes: a shell defining an interior volume and an aperture proximal an anterior end of the shell; a primary visor coupled to the shell and transiently arranged over the aperture; a secondary visor adjacent the aperture, including an elongated translucent member suspended from the shell, and defining a first discrete reflective region on a right side of the elongated translucent member and a second discrete reflective region on a left side of the elongated translucent member; a camera coupled to the shell and defining a field of view extending outwardly from a posterior end of the shell; and a projection system projecting an image onto the first discrete reflective region and onto the second discrete reflective region, the image comprising a subregion of a video frame recorded by the camera, the secondary visor reflecting the image toward the interior volume.

Growing community of inventors

Putnam Valley, NY, United States of America

Steven L Smith