Fyusion, Inc.

Other

Stephen David Miller

Stefan Johannes Josef Holzer

Radu Bogdan Rusu

Alexander Jay Bruen Trevor

Martin Saelzle

Pantelis Kalogiros

Krunal Ketan Chande

George Haber

Martin Markus Hubert Wawro

Mike Penz

Shuichi Tsutsumi

David Klein

Michelle Jung-Ah Ho

Matteo Munaro

Chris Beall

Matthias Reso

Nico Gregor Sebastian Blodow

Yusuke Tomoto

Luo Yi Tan

Yuheng Ren

Keith Martin

Dave Morrison

Julius Santiago

Jai Chaudhry

Luke Parham

Bojana Dumeljic

Michael E Casey

Abhishek Kar

Pavel Hanchar

Ronald Nutt

Charles C Watson

James J Hamill

Various embodiments of the present disclosure relate generally to systems and methods for drone-based systems and methods for capturing a multi- media representation of an entity. In some embodiments, the multi-media representation is digital, or multi-view, or interactive, and/or the combinations thereof. According to particular embodiments, a drone having a camera to is controlled or operated to obtain a plurality of images having location information. The plurality of images, including at least a portion of overlapping subject matter, are fused to form multi-view interactive digital media representations.

Drone based capture of multi-view interactive digital media

Various embodiments of the present disclosure relate generally to systems and methods for generating multi-view interactive digital media representations in a virtual reality environment. According to particular embodiments, a plurality of images is fused into a first content model and a first context model, both of which include multi-view interactive digital media representations of objects. Next, a virtual reality environment is generated using the first content model and the first context model. The virtual reality environment includes a first layer and a second layer. The user can navigate through and within the virtual reality environment to switch between multiple viewpoints of the content model via corresponding physical movements. The first layer includes the first content model and the second layer includes a second content model and wherein selection of the first layer provides access to the second layer with the second content model.

Virtual reality environment based manipulation of multi-layered multi-view interactive digital media representations

Various embodiments of the present invention relate generally to systems and methods for artificially rendering images using viewpoint interpolation and extrapolation. According to particular embodiments, a method includes moving a set of control points perpendicular to a trajectory between a first frame and a second frame, where the first frame includes a first image captured from a first location and the second frame includes a second image captured from a second location. The set of control points is associated with a layer and each control point is moved based on an associated depth of the control point. The method also includes generating an artificially rendered image corresponding to a third location outside of the trajectory by extrapolating individual control points using the set of control points for the third location and extrapolating pixel locations using the individual control points.

Artificially rendering images using viewpoint interpolation and extrapolation

Various embodiments of the present disclosure relate generally to systems and methods for drone-based systems and methods for capturing a multi-media representation of an entity. In some embodiments, the multi-media representation is digital, or multi-view, or interactive, and/or the combinations thereof. According to particular embodiments, a drone having a camera is controlled or operated to obtain a plurality of images having location information. The plurality of images, including at least a portion of overlapping subject matter, are fused to form multi-view interactive digital media representations.

Provided are mechanisms and processes for inserting a visual element into a multi-view digital media representation (MVIDMR). In one example, a process includes analyzing an MVIDMR to determine if there is an appropriate location to insert a visual element. Once a location is found, the type of visual element appropriate for the location is determined, where the type of visual element includes either a three-dimensional object to be inserted in the MVIDMR or a two-dimensional image to be inserted as or projected onto a background or object in the MVIDMR. A visual element that is appropriate for the location is then retrieved and inserted into the MVIDMR, such that the visual element is integrated into the MVIDMR and navigable by a user.

Modification of multi-view interactive digital media representation

Various embodiments of the present invention relate generally to mechanisms and processes relating to artificially rendering images using viewpoint interpolation and extrapolation. According to particular embodiments, a method includes applying a transform to estimate a path outside the trajectory between a first frame and a second frame, where the first frame includes a first image captured from a first location and the second frame includes a second image captured from a second location. The process also includes generating an artificially rendered image corresponding to a third location positioned on the path. The artificially rendered image is generated by interpolating a transformation from the first location to the third location and from the third location to the second location, gathering image information from the first frame and the second frame by transferring first image information from the first frame to the third frame and second image information from the second frame to the third frame, and combining the first image information and the second image information.

Various embodiments of the present invention relate generally to systems and processes for transforming a style of video data. In one embodiment, a neural network is used to interpolate native video data received from a camera system on a mobile device in real-time. The interpolation converts the live native video data into a particular style. For example, the style can be associated with a particular artist or a particular theme. The stylized video data can viewed on a display of the mobile device in a manner similar to which native live video data is output to the display. Thus, the stylized video data, which is viewed on the display, is consistent with a current position and orientation of the camera system on the display.

Live style transfer on a mobile device

Provided are mechanisms and processes for visual feature tagging in multi-view interactive digital media representations (MIDMRs). In one example, a process includes receiving a visual feature tagging request that includes an MIDMR of an object to be searched, where the MIDMR includes spatial information, scale information, and different viewpoint images of the object. A visual feature in the MIDMR is identified, and visual feature correspondence information is created that links information identifying the visual feature with locations in the viewpoint images. At least one image associated with the MIDMR is transmitted in response to the feature tagging request.

Visual feature tagging in multi-view interactive digital media representations

Various embodiments disclosed herein relate to systems and methods for analyzing and manipulating images and video. Methods as disclosed herein may include retrieving, using a processor, a multi-view interactive digital media representation (MIDMR) from a storage location, the MIDMR including a content model and a context model, the content model characterizing an object, and the context model characterizing scenery surrounding the object. The methods may also include receiving, using the processor, at least one dynamic content input associated with the retrieved MIDMR, the dynamic content input being received while a user is interacting with the MIDMR. The methods may further include implementing, using the processor, one or more modifications associated with the MIDMR based, at least in part, on the received at least one dynamic content input, the one or more modifications modifying a presentation and functionality of the MIDMR.

Dynamic content modification of image and video based multi-view interactive digital media representations

Various embodiments of the present invention relate generally to systems and methods for analyzing and manipulating images and video. According to particular embodiments, the spatial relationship between multiple images and video is analyzed together with location information data, for purposes of creating a representation referred to herein as a surround view for presentation on a device. A visual guide can provided for capturing the multiple images used in the surround view. The visual guide can be a synthetic object that is rendered in real-time into the images output to a display of an image capture device. The visual guide can help user keep the image capture device moving along a desired trajectory.

Live augmented reality guides

Drone based capture of a multi-view interactive digital media

Growing community of inventors

San Francisco, CA, United States of America

Stephen David Miller