The University of North Carolina at Chapel Hill

Urc Ventures, Inc.

Honda Motor Co., Ltd.

Cisco Technology, Inc.

Eth Zurich

Eidgenossische Technische Hochschule Zurich

Everypoint, Inc.

Jan-Michael Frahm

Charles A Erignac

David Bradley Boardman

Srinivas Kapaganty

Jared Scott Heinly

Henry Fuchs

Brian Sanderson Clipp

David Paul Perra

Stephen Murray Pizer

Rui Wang

Enrique Dunn

Julian Gary Rosenman

Ruibin Ma

James True Price

Sarah Kelly McGill

Ben Semerjian

J William Mauchly

Gregory Francis Welch

Madhav V Marathe

David Robert Gallup

Marc Andre Leon Pollefeys

Dinesh Manocha

Jongwoo Lim

Marc Pollefeys

Ravish Mehra

Anselmo A Lastra

Mingsong Dou

Johannes Lutz Schoenberger

Akash Abhijit Bapat

Qingyu Zhao

Anthony James Jacobson

Alok Namdeo Meshram

Miao Fan

Hongsheng Yang

Nate Michael Dierk

Charles Erignac

Methods, systems, and computer readable media for deriving a three-dimensional (3D) textured surface from endoscopic video are disclosed. According to one method for deriving a 3D textured surface from endoscopic video, the method comprises: performing video frame preprocessing to identify a plurality of video frames of an endoscopic video, wherein the video frame preprocessing includes informative frame selection, specularity removal, and key-frame selection; generating, using a neural network or a shape-from-motion-and-shading (SfMS) approach, a 3D textured surface from the plurality of video frames; and optionally registering the 3D textured surface to at least one CT image.

Methods, systems, and computer readable media for deriving a three-dimensional (3D) textured surface from endoscopic video

Methods, systems, and computer readable media for deriving a three-dimensional (3D) surface from colonoscopic video are disclosed. According to one method for deriving a 3D surface from colonoscopic video, the method comprises: performing video frame preprocessing to identify a plurality of keyframes of a colonoscopic video, wherein the video frame preprocessing includes informative frame selection and keyframe selection; generating, using a recurrent neural network and direct sparse odometry, camera poses and depth maps for the keyframes; and fusing, using SurfelMeshing and the camera poses, the depth maps into a three-dimensional (3D) surface of a colon portion, wherein the 3D surface indicates at least one region of the colon portion that was not visualized.

Methods, systems, and computer readable media for three-dimensional (3D) reconstruction of colonoscopic surfaces for determining missing regions

Methods, systems, and computer readable media for image overlap detection. An example method includes identifying, by one or more computers, a collection of images; streaming, by the one or more computers, each image from the collection of images so that, in one or a limited number of passes through the collection of images, each image is loaded only once from an input source and each image is discarded after a processing time for the image is exceeded; and during the streaming, for each image in at least a first subset of the images in the collection, determining whether the image overlaps with at least one other image in the at least a first subset of the images.

Methods, systems, and computer readable media for image overlap detection

Methods, systems, and computer-readable media for utilizing radial distortion to estimate a pose configuration are disclosed. According to one aspect, the method includes receiving, from each of a plurality of camera devices, an input pixel row of a radially distorted image and conducting a row comparison between each of the input pixel rows and a respectively associated synthesized pixel row. The method further includes approximating, for each row comparison, a span of a curve in an image space with a plurality of segments and computing, for each of the plurality of segments, a constraint. The method also includes utilizing the constraints to estimate a pose configuration.

Methods, systems, and computer-readable media for utilizing radial distortion to estimate a pose configuration

Techniques are described for analyzing images acquired via mobile devices in various ways, including to estimate measurements for one or more attributes of one or more objects in the images and/or to perform automated verification of such attribute measurements. For example, the described techniques may be used to measure the volume of a stockpile of material or other large object, based on images acquired via a mobile device that is moved around some or all of the object. The calculation of object volume and/or other determined object information may include generating and manipulating one or more computer models of the object from selected images. In addition, further automated verification activities may be performed for such computer model(s) and resulting object attribute measurements, such as based on analyzing one or more types of information that reflect accuracy and/or completeness of the computer model(s).

Verifying object measurements determined from mobile device images

Methods, systems, and computer readable media for unified scene acquisition and pose tracking in a wearable display are disclosed. According to one aspect, a system for unified scene acquisition and pose tracking in a wearable display includes a wearable frame configured to be worn by a user. Mounted on the frame are: at least one sensor for acquiring scene information for a real scene proximate to the user, the scene information including images and depth information; a pose tracker for estimating the user's head pose based on the acquired scene information; a rendering unit for generating a virtual reality (VR) image based on the acquired scene information and estimated head pose; and at least one display for displaying to the user a combination of the generated VR image and the scene proximate to the user.

Methods, systems, and computer readable media for unified scene acquisition and pose tracking in a wearable display

Methods, systems, and computer readable media for performing image compression are disclosed. According to one exemplary method, the method includes identifying a canonical image set from a plurality of images uploaded to or existing on a cloud computing and/or a storage environment. The method also includes computing an image representation for each image in the canonical image set. The method further includes receiving a first image. The method also includes identifying, using the image representations for the canonical image set, one or more reference images that are visually similar to the first image. The method further includes compressing the first image using the one or more reference images.

Methods, systems, and computer readable media for performing image compression

Techniques are described for analyzing images acquired via mobile devices in various ways, including to estimate measurements for one or more attributes of one or more objects in the images, as well as determine changes over time in objects and their measurements based on images acquired at different times. For example, the described techniques may be used to measure the volume of a stockpile of material or other large object, based on images acquired via a mobile device that moves around some or all of the object. The calculation of object volume and/or other determined object information may include generating and manipulating one or more computer models of the object from selected images, and determining changes may include comparing different models for different times. In addition, further automated activities may include displaying, presenting or otherwise providing information about some or all of the determined information.

Determining changes in object structure over time using mobile device images

A system for leveraging user gaze in a user monitoring subregion selection system includes a first camera configured to capture an image of a scene. A pattern generator is configured to generate and project a pattern onto a surface of at least one of a user's eyes. A sensor is configured to obtain an image of the pattern reflected from at least one of the user's eyes. In an alternate implementation, the pattern generator may be omitted, and the sensor may be a stereo user facing camera. A gaze estimation and scene mapping module is configured to estimate a gaze direction of the user using the image captured by the user facing sensor and to map the estimated gaze direction to the image of the scene based on a location of an object of interest within the scene. A subregion selection module is configured to select a subregion of the image of the scene based on the mapped user gaze direction.

Methods, systems, and computer readable media for leveraging user gaze in user monitoring subregion selection systems

Methods, systems, and computer readable media for utilizing adaptive rectangular decomposition (ARD) to perform head-related transfer function (HRTF) simulations are disclosed herein. According to one method, the method includes obtaining a mesh model representative of head and ear geometry of a listener entity and segmenting a simulation domain of the mesh model into a plurality of partitions. The method further includes conducting an ARD simulation on the plurality of partitions to generate simulated sound pressure signals within each of the plurality of partitions and processing the simulated sound pressure signals to generate at least one HRTF that is customized for the listener entity.

Methods, systems, and computer readable media for utilizing adaptive rectangular decomposition (ARD) to generate head-related transfer functions

Methods, systems, and computer readable media for low latency stabilization for head-worn displays are disclosed. According to one aspect, the subject matter described herein includes a system for low latency stabilization of a head-worn display. The system includes a low latency pose tracker having one or more rolling-shutter cameras that capture a 2D image by exposing each row of a frame at a later point in time than the previous row and that output image data row by row, and a tracking module for receiving image data row by row and using that data to generate a local appearance manifold. The generated manifold is used to track camera movements, which are used to produce a pose estimate.

Low latency stabilization for head-worn displays

Techniques are described for analyzing images acquired via mobile devices in various ways, including to estimate measurements for one or more attributes of one or more objects in the images. For example, the described techniques may be used to measure the volume of a stockpile of material or other large object, based on images acquired via a mobile device that is carried by a human user as he or she passes around some or all of the object. During the acquisition of a series of digital images of an object of interest, various types of user feedback may be provided to a human user operator of the mobile device, and particular images may be selected for further analysis in various manners. Furthermore, the calculation of object volume and/or other determined object information may include generating and manipulating a computer model or other representation of the object from selected images.

Determining object volume from mobile device images

Methods of generating a three dimensional representation of an object in a reference plane from a depth map including distances from a reference point to pixels in an image of the object taken from a reference point. Weights are assigned to respective voxels in a three dimensional grid along rays extending from the reference point through the pixels in the image based on the distances in the depth map from the reference point to the respective pixels, and a height map including an array of height values in the reference plane is formed based on the assigned weights. An n-layer height map may be constructed by generating a probabilistic occupancy grid for the voxels and forming an n-dimensional height map comprising an array of layer height values in the reference plane based on the probabilistic occupancy grid.

Systems and methods that generate height map models for efficient three dimensional reconstruction from depth information

Growing community of inventors

Chapel Hill, NC, United States of America

Jan-Michael Frahm