Fyusion, Inc.

Stefan Johannes Josef Holzer

Radu Bogdan Rusu

Stephen David Miller

Alexander Jay Bruen Trevor

Matteo Munaro

Abhishek Kar

Pavel Hanchar

Krunal Ketan Chande

Martin Saelzle

Pantelis Kalogiros

Chris Beall

George Haber

Yuheng Ren

Martin Markus Hubert Wawro

Aidas Liaudanskas

Vladimir Roumenov Glavtchev

Keith Martin

Rodrigo Ortiz-Cayon

Mike Penz

Dave Morrison

Julius Santiago

Matthias Reso

Nico Gregor Sebastian Blodow

Rodrigo Ortiz Cayon

Shuichi Tsutsumi

Milos Vlaski

Michelle Jung-Ah Ho

Ioannis Spanos

David Klein

Santi Arano

Yusuke Tomoto

Wook Yeon Hwang

Johan Nordin

Ben Mildenhall

Nick Stetco

Jai Chaudhry

Luke Parham

Luo Yi Tan

Gabriel Popa

Martin Josef Nikolaus Saelzle

Srinivas Rao

Bojana Dumeljic

Michelle Ho

Ferry Tanu

Blake McConnell

Endre Ajandi

Nishant Rai

Raul Dronca

Tobias Bertel

Christian Richardt

Ashley Wakefield

Josh Faust

Santiago Arano Perez

Tanvir Parhar

A plurality of images of an object and a background may be captured. Three-dimensional representations of the background and the object may be generated based on the captured images. A synthetic image of the object and the background may be rendered. The synthetic image may depict a two-dimensional view of the object having a novel viewpoint different from the viewpoints of the captured images. A corrected synthetic image may be generated. The corrected synthetic image may be stored on a storage medium.

Automatically generating synthetic images from novel viewpoints

Various embodiments of the present disclosure relate generally to systems and methods for drone-based systems and methods for capturing a multi- media representation of an entity. In some embodiments, the multi-media representation is digital, or multi-view, or interactive, and/or the combinations thereof. According to particular embodiments, a drone having a camera to is controlled or operated to obtain a plurality of images having location information. The plurality of images, including at least a portion of overlapping subject matter, are fused to form multi-view interactive digital media representations.

Drone based capture of multi-view interactive digital media

Various embodiments of the present invention relate generally to systems and methods for analyzing and manipulating images and video. In particular, a multi-view interactive digital media representation can be generated from live images captured from a camera. The live images can include an object. An angular view of the object captured in the live images can be estimated using sensor data from an inertial measurement unit. The determined angular views can be used to select from among the live images. The multi-view interactive digital media representation can include a plurality of images where each of the plurality of images includes the object from a different camera view. When the plurality of images is output to a display, the object can appear to undergo a 3-D rotation through the determined angular view where the 3-D rotation of the object is generated without a 3-D polygon model of the object.

Snapshots at predefined intervals or angles

Provided are mechanisms and processes for scene-aware selection of filters and effects for visual digital media content. In one example, a digital media item is analyzed with a processor to identify one or more characteristics associated with the digital media item, where the characteristics include a physical object represented in the digital media item. Based on the identified characteristics, a digital media modification is selected from a plurality of digital media modifications for application to the digital media item. The digital media modification may then be provided for presentation in a user interface for selection by a user.

Scene-aware selection of filters and effects for visual digital media content

Various embodiments of the present disclosure relate generally to systems and methods for generating multi-view interactive digital media representations in a virtual reality environment. According to particular embodiments, a plurality of images is fused into a first content model and a first context model, both of which include multi-view interactive digital media representations of objects. Next, a virtual reality environment is generated using the first content model and the first context model. The virtual reality environment includes a first layer and a second layer. The user can navigate through and within the virtual reality environment to switch between multiple viewpoints of the content model via corresponding physical movements. The first layer includes the first content model and the second layer includes a second content model and wherein selection of the first layer provides access to the second layer with the second content model.

Virtual reality environment based manipulation of multi-layered multi-view interactive digital media representations

A background scenery portion may be identified in each of a plurality of image sets of an object, where each image set includes images captured simultaneously from different cameras. A correspondence between the image sets may determined, where the correspondence tracks control points associated with the object and present in multiple images. A multi-view interactive digital media representation of the object that is navigable in one or more dimensions and that includes the image sets may be generated and stored.

Mobile multi-camera multi-view capture

One or more two-dimensional images of a three-dimensional object may be analyzed to estimate a three-dimensional mesh representing the object and a mapping of the two-dimensional images to the three-dimensional mesh. Initially, a correspondence may be determined between the images and a UV representation of a three-dimensional template mesh by training a neural network. Then, the three-dimensional template mesh may be deformed to determine the representation of the object. The process may involve a reprojection loss cycle in which points from the images are mapped onto the UV representation, then onto the three-dimensional template mesh, and then back onto the two-dimensional images.

Weak multi-view supervision for surface mapping estimation

Various embodiments describe systems and processes for capturing and generating multi-view interactive digital media representations (MIDMRs). In one aspect, a method for automatically generating a MIDMR comprises obtaining a first MIDMR and a second MIDMR. The first MIDMR includes a convex or concave motion capture using a recording device and is a general object MIDMR. The second MIDMR is a specific feature MIDMR. The first and second MIDMRs may be obtained using different capture motions. A third MIDMR is generated from the first and second MIDMRs, and is a combined embedded MIDMR. The combined embedded MIDMR may comprise the second MIDMR being embedded in the first MIDMR, forming an embedded second MIDMR. The third MIDMR may include a general view in which the first MIDMR is displayed for interactive viewing by a user on a user device. The embedded second MIDMR may not be viewable in the general view.

System and method for generating combined embedded multi-view interactive digital media representations

A live camera feed may be analyzed to determine the identify of an object, and augmented reality overlay data may be determined based on that identity. The overlay data may include one or more tags that are each associated with a respective location on the object. The live camera feed may be presented on a display screen with the tags being positioned as the respective location.

Live in-camera overlays

Inventory at a remote location may be verified by transmitting a security key associated with uniquely identifying object identification information from a verification server to a client machine at the remote location. The security key may then be used to generate a multi-view interactive digital media representation (MVIDMR) of the object that includes a plurality of images captured from different viewpoints. The MVIDMR may then be transmitted to the verification server.

Multiview interactive digital media representation inventory verification

Images of an object may be captured via a camera at a mobile computing device at different viewpoints. The images may be used to identify components of the object and to identify damage estimates estimating damage to some or all of the components. Capture coverage levels corresponding with the components may be determined, and then recording guidance may be provided for capturing additional images to increase the capture coverage levels.

Multi-view visual data damage detection

Images of an object may be captured at a computing device. Each of the images may be captured from a respective viewpoint based on image capture configuration information identifying one or more parameter values. A multiview image digital media representation of the object may be generated that includes some or all of the images of the object and that is navigable in one or more dimensions.

Damage detection from multi-view visual data

Damage to an object such as a vehicle may be detected and presented based at least in part on image data. In some configurations, image data may be detected by causing the object to pass through a gate or portal on which cameras are located. Alternatively, or additionally, image data may be selected by a user operating a camera and moving around the object. The cameras may capture image data, which may be combined and analyzed to detect damage. Some or all of the image data and/or analysis of the image data may be presented in a viewer, which may allow a user to perform actions such as navigating around the object in a virtual environment, identifying and viewing areas of the object where damage has been detected, and accessing the results of the analysis.

Multi-view interactive digital media representation viewer

A tag characterizing a portion of a multi-view interactive digital media representation (MVIDMR) may be determined by applying a grammar to natural language data. The MVIDMR may include images of an object and may be navigable in one or more dimensions. An object model location for the tag identifying a location within a three-dimensional object model may be determined by applying the grammar to the natural language data. The tag may then be applied to the MVIDMR by associating it with two or more of the images at positions determined based on the object model location.

Natural language understanding for visual tagging

According to particular embodiments, one process includes retrieving a multi-view interactive digital media representation that includes numerous images fused together into content and context models. The process next includes retrieving and processing audio data to be integrated into the multi-view interactive digital media representation. A first segment of audio data may be associated with a first position in the multi-view interactive digital media representation. In other examples, a first segment of audio data may be associated with a visual position or the location of a camera in the multi-view interactive digital media representation. The audio data may be played in coordination with the multi-view interactive digital media representation based on a user's navigation through the multi-view interactive digital media representation, where the first segment is played when the first position or first visual position is reached.

Integration of audio into a multi-view interactive digital media representation

Growing community of inventors

San Mateo, CA, United States of America

Stefan Johannes Josef Holzer