Fyusion, Inc.

Radu Bogdan Rusu

Stefan Johannes Josef Holzer

Stephen David Miller

Alexander Jay Bruen Trevor

Abhishek Kar

Matteo Munaro

Pavel Hanchar

Krunal Ketan Chande

Pantelis Kalogiros

Chris Beall

Martin Saelzle

George Haber

Yuheng Ren

Vladimir Roumenov Glavtchev

Martin Markus Hubert Wawro

Keith Martin

Mike Penz

Dave Morrison

Matthias Reso

Shuichi Tsutsumi

Michelle Jung-Ah Ho

David Klein

Julius Santiago

Nico Gregor Sebastian Blodow

Ioannis Spanos

Aidas Liaudanskas

Rodrigo Ortiz Cayon

Santi Arano

Yusuke Tomoto

Ben Mildenhall

Jai Chaudhry

Luke Parham

Luo Yi Tan

Gabriel Popa

Martin Josef Nikolaus Saelzle

Bojana Dumeljic

Rodrigo Ortiz-Cayon

Milos Vlaski

Michelle Ho

Ferry Tanu

Endre Ajandi

Ashley Wakefield

Josh Faust

Santiago Arano Perez

Provided are mechanisms and processes for scene-aware selection of filters and effects for visual digital media content. In one example, a digital media item is analyzed with a processor to identify one or more characteristics associated with the digital media item, where the characteristics include a physical object represented in the digital media item. Based on the identified characteristics, a digital media modification is selected from a plurality of digital media modifications for application to the digital media item. The digital media modification may then be provided for presentation in a user interface for selection by a user.

Scene-aware selection of filters and effects for visual digital media content

Various embodiments of the present disclosure relate generally to systems and methods for generating multi-view interactive digital media representations in a virtual reality environment. According to particular embodiments, a plurality of images is fused into a first content model and a first context model, both of which include multi-view interactive digital media representations of objects. Next, a virtual reality environment is generated using the first content model and the first context model. The virtual reality environment includes a first layer and a second layer. The user can navigate through and within the virtual reality environment to switch between multiple viewpoints of the content model via corresponding physical movements. The first layer includes the first content model and the second layer includes a second content model and wherein selection of the first layer provides access to the second layer with the second content model.

Virtual reality environment based manipulation of multi-layered multi-view interactive digital media representations

A background scenery portion may be identified in each of a plurality of image sets of an object, where each image set includes images captured simultaneously from different cameras. A correspondence between the image sets may determined, where the correspondence tracks control points associated with the object and present in multiple images. A multi-view interactive digital media representation of the object that is navigable in one or more dimensions and that includes the image sets may be generated and stored.

Mobile multi-camera multi-view capture

Various embodiments describe systems and processes for capturing and generating multi-view interactive digital media representations (MIDMRs). In one aspect, a method for automatically generating a MIDMR comprises obtaining a first MIDMR and a second MIDMR. The first MIDMR includes a convex or concave motion capture using a recording device and is a general object MIDMR. The second MIDMR is a specific feature MIDMR. The first and second MIDMRs may be obtained using different capture motions. A third MIDMR is generated from the first and second MIDMRs, and is a combined embedded MIDMR. The combined embedded MIDMR may comprise the second MIDMR being embedded in the first MIDMR, forming an embedded second MIDMR. The third MIDMR may include a general view in which the first MIDMR is displayed for interactive viewing by a user on a user device. The embedded second MIDMR may not be viewable in the general view.

System and method for generating combined embedded multi-view interactive digital media representations

A live camera feed may be analyzed to determine the identify of an object, and augmented reality overlay data may be determined based on that identity. The overlay data may include one or more tags that are each associated with a respective location on the object. The live camera feed may be presented on a display screen with the tags being positioned as the respective location.

Live in-camera overlays

Inventory at a remote location may be verified by transmitting a security key associated with uniquely identifying object identification information from a verification server to a client machine at the remote location. The security key may then be used to generate a multi-view interactive digital media representation (MVIDMR) of the object that includes a plurality of images captured from different viewpoints. The MVIDMR may then be transmitted to the verification server.

Multiview interactive digital media representation inventory verification

Images of an object may be captured at a computing device. Each of the images may be captured from a respective viewpoint based on image capture configuration information identifying one or more parameter values. A multiview image digital media representation of the object may be generated that includes some or all of the images of the object and that is navigable in one or more dimensions.

Damage detection from multi-view visual data

A tag characterizing a portion of a multi-view interactive digital media representation (MVIDMR) may be determined by applying a grammar to natural language data. The MVIDMR may include images of an object and may be navigable in one or more dimensions. An object model location for the tag identifying a location within a three-dimensional object model may be determined by applying the grammar to the natural language data. The tag may then be applied to the MVIDMR by associating it with two or more of the images at positions determined based on the object model location.

Natural language understanding for visual tagging

According to particular embodiments, one process includes retrieving a multi-view interactive digital media representation that includes numerous images fused together into content and context models. The process next includes retrieving and processing audio data to be integrated into the multi-view interactive digital media representation. A first segment of audio data may be associated with a first position in the multi-view interactive digital media representation. In other examples, a first segment of audio data may be associated with a visual position or the location of a camera in the multi-view interactive digital media representation. The audio data may be played in coordination with the multi-view interactive digital media representation based on a user's navigation through the multi-view interactive digital media representation, where the first segment is played when the first position or first visual position is reached.

Integration of audio into a multi-view interactive digital media representation

The pose of an object may be estimated based on fiducial points identified in a visual representation of the object. Each fiducial point may correspond with a component of the object, and may be associated with a first location in an image of the object and a second location in a 3D coordinate pace. A 3D skeleton of the object may be determined by connecting the locations in the 3D space, and the object's pose may be determined based on the 3D skeleton.

Object pose estimation in visual data

Mappings are determined between viewpoints of an object and an object model representing the object. Each mapping identifies a location on the object model corresponding with a portion of the object captured in one of the viewpoints. Tags for the object model are created based on the mappings, where each tag links one of the viewpoints to one of the locations on the object model. A user interface that includes the object model and the tags is provided for presentation on a display screen in a user interface. One of the viewpoints is presented in the user interface when the corresponding tag is selected in the object model.

Structuring visual data

Various embodiments of the present invention relate generally to systems and methods for artificially rendering images using viewpoint interpolation and extrapolation. According to particular embodiments, a method includes moving a set of control points perpendicular to a trajectory between a first frame and a second frame, where the first frame includes a first image captured from a first location and the second frame includes a second image captured from a second location. The set of control points is associated with a layer and each control point is moved based on an associated depth of the control point. The method also includes generating an artificially rendered image corresponding to a third location outside of the trajectory by extrapolating individual control points using the set of control points for the third location and extrapolating pixel locations using the individual control points.

Artificially rendering images using viewpoint interpolation and extrapolation

A plurality of images may be analyzed to determine an object model. The object model may have a plurality of components, and each of the images may correspond with one or more of the components. Component condition information may be determined for one or more of the components based on the images. The component condition information may indicate damage incurred by the object portion corresponding with the component.

A multi-view interactive digital media representation (MVIDMR) of an object can be generated from live images of an object captured from a camera. Selectable tags can be placed at locations on the object in the MVIDMR. When the selectable tags are selected, media content can be output which shows details of the object at location where the selectable tag is placed. A machine learning algorithm can be used to automatically recognize landmarks on the object in the frames of the MVIDMR and a structure from motion calculation can be used to determine 3-D positions associated with the landmarks. A 3-D skeleton associated with the object can be assembled from the 3-D positions and projected into the frames associated with the MVIDMR. The 3-D skeleton can be used to determine the selectable tag locations in the frames of the MVIDMR of the object.

Method and apparatus for 3-D auto tagging

Growing community of inventors

San Francisco, CA, United States of America

Radu Bogdan Rusu