Fyusion, Inc.

Videostitch Inc.

Cresta Technology Corporation

George Haber

Stefan Johannes Josef Holzer

Radu Bogdan Rusu

Stephen David Miller

Pantelis Kalogiros

Alexander Fink

Nicolas Burtey

Stephane Valente

Keith Martin

Dave Morrison

Vladimir Roumenov Glavtchev

Mihai H Murgulescu

Gabriel Popa

Abhishek Kar

Pavel Hanchar

Martin Saelzle

Stéphane Valente

Mike Penz

Shuichi Tsutsumi

Ferry Tanu

Various embodiments describe systems and processes for capturing and generating multi-view interactive digital media representations (MIDMRs). In one aspect, a method for automatically generating a MIDMR comprises obtaining a first MIDMR and a second MIDMR. The first MIDMR includes a convex or concave motion capture using a recording device and is a general object MIDMR. The second MIDMR is a specific feature MIDMR. The first and second MIDMRs may be obtained using different capture motions. A third MIDMR is generated from the first and second MIDMRs, and is a combined embedded MIDMR. The combined embedded MIDMR may comprise the second MIDMR being embedded in the first MIDMR, forming an embedded second MIDMR. The third MIDMR may include a general view in which the first MIDMR is displayed for interactive viewing by a user on a user device. The embedded second MIDMR may not be viewable in the general view.

System and method for generating combined embedded multi-view interactive digital media representations

Provided are mechanisms and processes for inserting a visual element into a multi-view digital media representation (MVIDMR). In one example, a process includes analyzing an MVIDMR to determine if there is an appropriate location to insert a visual element. Once a location is found, the type of visual element appropriate for the location is determined, where the type of visual element includes either a three-dimensional object to be inserted in the MVIDMR or a two-dimensional image to be inserted as or projected onto a background or object in the MVIDMR. A visual element that is appropriate for the location is then retrieved and inserted into the MVIDMR, such that the visual element is integrated into the MVIDMR and navigable by a user.

Modification of multi-view interactive digital media representation

Various embodiments of the present invention relate generally to systems and processes for transforming a style of video data. In one embodiment, a neural network is used to interpolate native video data received from a camera system on a mobile device in real-time. The interpolation converts the live native video data into a particular style. For example, the style can be associated with a particular artist or a particular theme. The stylized video data can viewed on a display of the mobile device in a manner similar to which native live video data is output to the display. Thus, the stylized video data, which is viewed on the display, is consistent with a current position and orientation of the camera system on the display.

Live style transfer on a mobile device

Various embodiments disclosed herein relate to systems and methods for analyzing and manipulating images and video. Methods as disclosed herein may include retrieving, using a processor, a multi-view interactive digital media representation (MIDMR) from a storage location, the MIDMR including a content model and a context model, the content model characterizing an object, and the context model characterizing scenery surrounding the object. The methods may also include receiving, using the processor, at least one dynamic content input associated with the retrieved MIDMR, the dynamic content input being received while a user is interacting with the MIDMR. The methods may further include implementing, using the processor, one or more modifications associated with the MIDMR based, at least in part, on the received at least one dynamic content input, the one or more modifications modifying a presentation and functionality of the MIDMR.

Dynamic content modification of image and video based multi-view interactive digital media representations

Described are systems and processes for generating multi-view interactive digital media representations (MIDMR) for display on a user device. In one aspect, a mobile device is provided which comprises a display, one or more processors, memory, and one or more programs stored in memory. The one or more programs comprise instructions for locking the mobile device, and providing a lock screen on the display in a lock mode upon receiving user input for accessing the mobile device. The lock screen may display a dynamic MIDMR that dynamically changes without user input, which provides an interactive three-dimensional representation of an object that is responsive to user interaction with the mobile device. The dynamic MIDMR displayed is selected based on predetermined criteria, and may change based on a predetermined algorithm that includes weighted predetermined criteria factors as variables and recalculates the algorithm value to determine changes to the dynamic MIDMR.

Dynamic multi-view interactive digital media representation lock screen

Various embodiments describe systems and processes for capturing and generating multi-view interactive digital media representations for display on a user device. In one aspect, a mobile device is provided which comprises a display, one or more processors, memory, and one or more programs stored in the memory. The one or more programs comprise instructions for locking the mobile device, and providing a lock screen on the display in a lock mode upon receiving user input for accessing the mobile device. The lock screen may display a multi-view interactive digital media representation (MIDMR) which provides an interactive three-dimensional representation of an object that is responsive to user interaction with the mobile device. The MIDMR may respond to spatial and movement sensors in the mobile device. The mobile device may be unlocked for use upon receiving user identification input, which may include maneuvering the MIDMR in a predetermined pattern.

Multi-view interactive digital media representation lock screen

Various embodiments of the present invention relate generally to systems and methods for analyzing and manipulating images and video. According to particular embodiments, the spatial relationship between multiple images and video is analyzed together with location information data, for purposes of creating a representation referred to herein as a multi-view interactive digital media representation for presentation on a device. Once a multi-view interactive digital media representation is generated, a user can provide navigational inputs, such via tilting of the device, which alter the presentation state of the multi-view interactive digital media representation. The navigational inputs can be analyzed to determine metrics which indicate a user's interest in the multi-view interactive digital media representation.

Tilts as a measure of user engagement for multiview interactive digital media representations

Tilts as a measure of user engagement for multiview digital media representations

An apparatus comprising a plurality of image modules and a plurality of processors. The image modules may each comprise (i) a sensor configured to generate images and (ii) a lens mounted to the sensor. The processors may each be configured to (A) receive the images from a subset of the plurality of image modules and (B) generate a plurality of video streams. Each one of the video streams may be generated by one of the processors in response to the images received from one of the image modules. The subset of the plurality of image modules may comprise at least two distinct image modules of the plurality of image modules. The lenses may be arranged to allow the images to provide coverage for a spherical field of view of a scene surrounding the apparatus.

Omnidirectional camera with multiple processors and/or multiple sensors connected to each processor

A system comprising a camera and a computing device. The camera may comprise (a) a plurality of capture devices configured to capture images of an environment surrounding the camera to provide a spherical field of view and (b) a first interface. The computing device may comprise (a) a processor and (b) a second interface. The camera may be configured to encode a plurality of video streams based on the captured images. The first interface may be configured to transfer the plurality of video streams to the second interface. The processor may perform a stitching operation on the plurality of video streams to generate a single video signal. The stitching operation may be performed on the plurality of video streams in real time as the plurality of video streams are transferred. The single video signal may be configured to represent an omnidirectional view based on the environment surrounding the camera.

Growing community of inventors

Los Altos Hills, CA, United States of America

George Haber