Verizon Patent and Licensing Inc.

Liang Luo

Elena Dotsenko

Tom Hsi Hao Shang

Vidhya Seran

Oliver S Castaneda

Denny Breitenfeld

Timothy Atchley

Igor Gomon

Lonnie Souder

An illustrative camera calibration system accesses a set of images captured by one or more cameras at a scene. The set of images may each depict a same element of image content present at the scene, but may depict the element of image content differently in each image so as to show an apparent movement of the element from one image to another. The camera calibration system applies a structure-from-motion algorithm to the set of images to generate calibration parameters for the one or more cameras based on the apparent movement of the element of image content shown in the set of images. Additionally, the camera calibration system provides the calibration parameters for the one or more cameras to a 3D modeling system configured to model the scene based on the set of images. Corresponding methods and systems are also disclosed.

Methods and systems for camera calibration based on apparent movement of image content at a scene

An illustrative content server system associated with a server domain may transmit, to a client device associated with a client domain, a data stream representing media content. The media content is represented at a first quality level for presentation within a presentation environment as the data stream is transmitted. Based on environment data captured in the client domain to represent real-time conditions of the presentation environment as the media content is presented, the content server system may determine an engagement parameter representing a real-time engagement level of a user with respect to the media content. Based on this engagement parameter, the content server system may adjust, in real time during the transmitting of the data stream and presentation of the media content, the data stream to represent the media content at a second quality level that is different from the first quality level. Corresponding methods and systems are also disclosed.

Bandwidth management methods and systems based on client-side environment data

An illustrative volumetric content production system may access first capture data and second capture data. These first capture data may represent an entirety of a capture target and may be captured by a fixed camera system including a first plurality of image capture devices having respective fixed viewpoints with respect to the capture target. The second capture data may represent a portion of the capture target less than the entirety of the capture target and may be captured by a dynamic camera system including a second plurality of image capture devices having respective dynamic viewpoints with respect to the capture target. Based on the first and second capture data, the volumetric content production system may generate a volumetric representation of an object included in the portion of the capture target in accordance with principles described herein. Corresponding methods and systems are also disclosed.

Methods and systems for producing volumetric content using combined fixed and dynamic camera systems

An illustrative audiovisual content rendering system generates a light dataset configured to model light energy at a particular location within a simulated 3D environment for a video frame time. The audiovisual content rendering system also generates an audio dataset configured to model acoustic energy at the particular location within the simulated 3D environment for an audio frame time concurrent with the video frame time. The audiovisual content rendering system stores the light dataset and the audio dataset together within a voxel-based data structure. More particularly, the light dataset and the audio dataset are stored together within a particular voxel of the voxel-based data structure that corresponds to the particular location within the simulated 3D environment. Corresponding methods and systems are also disclosed.

Methods and systems for unified rendering of light and sound content for a simulated 3D environment

An illustrative image processing system determines that an object depicted in an image is an instance of an object type for which a machine learning model is available to the image processing system. In response to the determining that the object is the instance of the object type, the image processing system obtains pose data generated based on the machine learning model to represent how objects of the object type are capable of being posed. The image processing system generates a volumetric representation of the object in an estimated pose. The estimated pose is estimated independently of depth data for the object based on the image and the pose data. Corresponding methods and systems are also disclosed.

Methods and systems for volumetric modeling independent of depth data

An illustrative scene capture system determines a set of two-dimensional (2D) feature pairs each representing a respective correspondence between particular features depicted in both a first intensity image from a first vantage point and a second intensity image from a second vantage point. Based on the set of 2D feature pairs, the system determines a set of candidate three-dimensional (3D) feature pairs for a first depth image from the first vantage point and a second depth image from the second vantage point. The system selects a subset of selected 3D feature pairs from the set of candidate 3D feature pairs in a manner configured to minimize an error associated with a transformation between the first depth image and the second depth image. Based on the subset of selected 3D feature pairs, the system manages calibration parameters for surface data capture devices that captured the intensity and depth images.

Methods and systems for calibrating surface data capture devices

An illustrative image processing system identifies an object depicted in an image captured by a camera. The image processing system locates, within the image, a set of calibration points corresponding to features of the object. Based on the set of calibration points, the image processing system performs a calibration operation with respect to the camera. Additionally, the image processing system generates model data based on the image. The model data is representative of a model of the object depicted in the image. Corresponding methods and systems are also disclosed.

Methods and systems for intra-capture camera calibration

An illustrative image processing system determines calibration parameters for a set of cameras including a first camera configured to capture a scene from a first vantage point and a second camera configured to capture the scene from a second vantage point. The image processing system obtains pose data for an object included in the scene and depicted by first and second images captured, respectively, by the first and second cameras. The pose data is representative of how the object is capable of being posed. Based on the calibration parameters, the pose data, and the first and second images, the image processing system estimates a pose of the object in the scene independently of depth data for the object. The image processing system also generates model data of the scene that includes a volumetric representation of the object in the estimated pose. Corresponding methods and systems are also disclosed.

An exemplary image simulation system accesses an image depicting imagery captured at a capture resolution from a capture vantage point. The image simulation system partitions the image into a plurality of simulated images, and determines a respective virtual resolution and a respective virtual vantage point for each of the simulated images in the plurality of simulated images. Each virtual resolution determined by the image simulation system is different from the capture resolution and each virtual vantage point determined by the image simulation system is different from the capture vantage point. The image simulation system represents each simulated image of the plurality of simulated images as a distinct image captured at the respective virtual resolution and from the respective virtual vantage point that have been determined for the simulated image. Corresponding methods and systems are also disclosed, including methods and systems for combining captured images to form a single simulated image.

Methods and systems for simulating image capture in an extended reality system

An exemplary virtual reality system determines a first confidence metric representing an objective degree to which a pre-modeled object is recognized within a first captured image depicting a scene from a first vantage point. The virtual reality system also determines a second confidence metric representing an objective degree to which the pre-modeled object is recognized within a second captured image depicting the scene from a second vantage point distinct from the first vantage point. The virtual reality system then generates a rendered image that includes a depiction of the pre-modeled object from a third vantage point distinct from the first and second vantage points. The depiction of the pre-modeled object is generated based on data from the first and second captured images, the data prioritized for the depiction of the pre-modeled object according to relative magnitudes of the first and second confidence metrics. Corresponding methods and systems are also disclosed.

Methods and systems for representing a pre-modeled object within virtual reality data

An exemplary virtual reality system accesses first and second image datasets representative of first and second captured images depicting a real-world scene from first and second vantage points. The system recognizes a pre-modeled object within both the first and second captured images, and determines first and second confidence metrics representative of objective degrees to which the system accurately recognizes the pre-modeled object within the first and second captured images, respectively. The system further generates, a third image dataset representative of a rendered image based on the first and second image datasets. The rendered image includes a depiction of the pre-modeled object within the real-world scene from a third vantage point, and the generating comprises prioritizing, based on a determination that the second confidence metric is greater than the first confidence metric, the second image dataset over the first image dataset for the depiction of the pre-modeled object.

An exemplary volumetric frame capture controller ('controller') transmits first and second commands to first and second three-dimensional ('3D') capture devices disposed, respectively, at first and second positions with respect to a real-world scene. The first and second commands are to capture, respectively, first and second 3D sub-frames representative of the real-world scene from first and second vantage points associated with the first and second positions. Based on the first and second commands, the controller receives the first and second 3D sub-frames and, in response, determines that a first timestamp associated with a capture of the first 3D sub-frame is less than a predefined time threshold from a second timestamp associated with a capture of the second 3D sub-frame. In response to this determination, the controller provides the first and second 3D sub-frames for use in forming a volumetric frame of the real-world scene. Corresponding methods and systems are also disclosed.

Growing community of inventors

Piscataway Township, NJ, United States of America

Liang Luo