International Business Machines Corporation

Csr Technology Inc.

Technion Research & Development Foundation Limited

Technion R&d Foundation Ltd.

Yochay Tzur

Asaf Tzadok

Joseph Shtok

Udi Barzelay

Ophir Azulai

Dudi Vakrat

Ayellet Tal

Mattias Marder

Oded Dubovsky

Leonid Karlinsky

Uzi Shvadron

Adi Raz Goldfarb

Benjamin M Cohen

Eyal Mush

Embodiments provide 3D coordinates for points in a scene that are observed to be in the correct physical position in a series of images. A method may comprise obtaining a plurality of images including a base image having at least one annotated point corresponding to a point of an object shown in the base image, and a plurality of side images showing the object from different viewpoints than the base image, wherein the plurality of side images are given with the camera poses relative to the base image, extracting from at least some of the side images, image patches showing the annotated point, wherein a plurality of sets of image patches are extracted, wherein a set of image patches is extracted at a plurality of corresponding candidate depth values, classifying each set as having a corresponding candidate depth value that is correct or incorrect, and outputting a correct depth value.

Point depth estimation from a set of 3D-registered images

A system and method for resolving an ambiguity between similar objects in an image is disclosed. A three dimensional representation of a room is generated, and objects in the room are identified from an image of the room are identified. A determination is made that at least two objects are visually similar, and a position of the two objects is ambiguous. At least one question based on the determined ambiguity is programmatically generated based on information known about the room, and is phrased such that the ambiguity can be resolved by an answer to the question. Based on the answer received one of the objects is selected. At least one property of the selected object is modified based upon the selection of one of the at least two objects.

Object instance ambiguity resolution

Automatically generating augmented reality (AR) content by constructing a three-dimensional (3D) model of an object-including scene using images recorded during a remotely-guided AR session from a camera position defined relative to first 3D axes, the model including camera positions defined relative to second 3D axes, registering the first axes with the second axes by matching a trajectory derived from the image camera positions to a trajectory derived from the model's camera positions for determining a session-to-model transform, translating, using the transform, positions of points of interest (POIs) indicated on the object during the session, to corresponding POI positions on the object within the model, where the session POI positions are defined relative to the first axes and the model POI positions are defined relative to the second axes, and generating a content package including the model, model POI positions, and POI annotations provided during the session.

Automatic generation of content for autonomic augmented reality applications

Receiving a recording of a remotely-guided augmented reality (AR) session which includes: images of a scene, captured by a camera of a local user; position and orientation data of the camera; and annotations generated by a remote user at points-of-interest (POIs) in a three-dimensional (3D) representation of the scene. Automatically generating a summary of the session, by: projecting the annotations to matching locations in some of the prominent images, based on the POIs of the annotations and on the position and orientation data of the camera and including, in the summary: the prominent images, including those of the prominent images having the projected annotations.

Automatic summarization of remotely-guided augmented reality sessions

Embodiments of the present systems and methods may provide techniques that provide automatic, reliable performance of the point cloud cleanup task. Embodiments may provide the capability to progressively learn the object-background segmentation from tracking sessions in augmented reality (AR) applications. The 3D point cloud may be used for tracking in AR applications, by matching points from the cloud to regions in the live video. A 3D point that has many matches is more likely to be part of the object, while a 3D point that rarely has matches is more likely to be part of the background. Advantages of this approach include that no manual work is needed to do the segmentation, and the results may be constantly improved over time, as the object is being tracked in multiple environments.

Progressive 3D point cloud segmentation into object and background from tracking sessions

Embodiments may provide a fusion of Visual-Inertial-Odometry with Object Tracking for physically anchored Augmented Reality content presentation. For example, a method of generating augmented reality content may comprise receiving a video stream, comprising a plurality of frames, the video stream generated using a camera that is moving about a scene, determining a VIO camera pose for each frame in the video stream in a VIO coordinate system using visual inertial odometry, determining a OT camera pose for at least some of the frames of the video stream in a OT coordinate system using object tracking, determining a relation between the VIO coordinate system and the OT coordinate system and generating augmented reality content that is placed in the video stream of the scene at a location based on the determined relation between the VIO coordinate system and the OT coordinate system.

Fusion of visual-inertial-odometry and object tracker for physically anchored augmented reality

A method of constructing hierarchical representation models of compound 3D physical objects, comprising receiving a plurality of image sets depicting a compound 3D physical object constructed of a static component having a single representation state and dynamic component(s) having multiple representation states, each pair of the image sets corresponds depicts a respective dynamic component in one of its representation states, generating a pair of point cloud structures from each pair of image sets, analyzing the point cloud structures pair to identify common region(s) associated with the static component and region(s) of difference associated with the dynamic component, generating a hierarchical 3D representation model comprising a high level model of the static component generated from the common region(s) with a surface outline of the region(s) of difference associated with multiple lower level models constructed for the representation states of the dynamic component and outputting the hierarchical 3D representation model.

Construction of an efficient representation for a three-dimensional (3D) compound object from raw video data

Technology for matching images (for example, video images, still images) of an identical infrastructure object (for example, a tower component of a tower supporting power lines) for purposes of comparing the infrastructure object to itself at different points in time to detect a potential anomaly and the potential need for maintenance of the infrastructure object. In some embodiments, this matching of images is done using creation of a three dimensional (#D) computer model of the infrastructure object and by tagging captured images with location on the 3D model across multiple videos taken at different points in time.

Identifying temporal changes of industrial objects by matching images

Technology for matching images (for example, video images, still images) of an identical infrastructure object (for example, a tower component of a tower supporting power lines) for purposes of comparing the infrastructure object to itself at different points in time to detect a potential anomaly and the potential need for maintenance of the infrastructure object. In some embodiments, this matching of images is done using creation of a three dimensional (# D) computer model of the infrastructure object and by tagging captured images with location on the 3D model across multiple videos taken at different points in time.

There is provided a method of computing a camera pose of a digital image, comprising: computing query-regions of a digital image, each query-region maps to training image region(s) of training image(s) by a 2D translation and/or a 2D scaling, each training image associated with a reference camera pose, each query-region associated with a center point and a computed weighted mask that weights the query-region pixels according to computed correlations with the corresponding training image region, mapping cloud points corresponding to pixels of matched training image region(s) to corresponding images pixels of the matched query-regions according to a statistically significant correlation requirement between the center point of the query-region and the matched training image region, and according to the computed weight mask, and computing the camera pose according to an aggregation of the camera poses, and the mapped cloud points and corresponding image pixels of the matched query-regions.

Systems and methods for determining a camera pose of an image

A system for identifying specific instances of objects in a three-dimensional (3D) scene, comprising: a camera for capturing an image of multiple objects at a site; at least one processor executable to: use a location and orientation of the camera to create a 3D model of the site including multiple instances of objects expected to be in proximity to the camera, and generate multiple candidate clusters each representing a different projection of the 3D model, detect at least two objects in the image, and determine a spatial configuration for each detected object; and match the detected image objects to one of the multiple candidate cluster using the spatial configurations, associate the detected objects with the expected object instances of the matched cluster, and retrieve information of one of the detected objects that is stored with the associated expected object instance; and a head-wearable display configured to display the information.

Object instance identification using three-dimensional spatial configuration

A method of training an object identification system and identifying three dimensional objects using semantic segments includes receiving, into a non-volatile memory, an input file containing a geometric description of a three dimensional object having one or more semantic segments and one or more annotations for each of the one or more semantic segments, receiving, into the non-volatile memory one or more training images of the three dimensional object, identifying, through a processor, the one or more segments in the one or more training images, computing, through a training module, one or more descriptors to the one or more segments, and generating an output file representing a machine vision of the three dimensional object.

System, method and computer program product for training a three dimensional object indentification system and identifying three dimensional objects using semantic segments

A method for photogrammetric texture mapping using casual images is provided. The method may include the following steps: estimating, for each vertex of at least a portion of a three dimensional (3D) mesh representing a model, projection parameters associated with a virtual camera that is unique for each vertex; mapping pixels from a two dimensional (2D) image to the vertices, such that each mapping of a pixel is based on the estimated respective virtual camera parameters; and texturing the portion of the mesh with corresponding mapped pixels wherein vertices on the textured portion are selected such that they are visible from a specified viewpoint associated with the 3D mesh.

Photogrammetric texture mapping using casual images

A system, method, and computer program product for automatically detecting and correcting the 'purple fringing' effect, typically due to axial chromatic aberration in imaging devices, are disclosed and claimed. A chromaticity score is computed, denoting the amount of false color related to a purple fringing artifact. A locality score is computed, denoting the similarity of the purple fringing region to a shape of a narrow ridge, which is typical for purple fringing artifacts. A saturation score is also computed, denoting the proximity of a pixel to saturated pixels. These scores are then combined into a detection score, denoting pixels having strong indications they share properties common to pixels of purple fringing artifacts. The detected pixels are then correspondingly corrected, e.g. by chroma suppression. The scoring and correction may be performed over combinations of image resolutions, e.g. an original version and potentially numerous downscaled versions.

Growing community of inventors

Haifa, Israel

Yochay Tzur