Pixar

Magic Leap, Inc.

Pixar Animation Studios

Michael Kass

Anthony David DeRose

Andrew Witkin

David E Baraff

Warren Trezevant

Justin M Solomon

John R Anderson

Rick Sayre

Igor Mordatch

Apurva Shah

Douglas Bertram Lee

Maxwell E Planck

Forrester Cole

Michael Boyle Johnson

Hayley Iben

Najeeb Tarazi

Tien Gia Truong

Yu-Chung Chen

Pierre Benard

James Hegarty

Yasmin Khan

Davide Pesare

An augmented reality system and method of operating the same is provided. An end user is allowed to visualize direct light from a three-dimensional scene in an ambient environment, generating virtual image data from the point of the view of the end user. A region of spatial overlap between a real object in the three-dimensional scene and a virtual object in the virtual image data is determined. Color characteristics of the real object in the overlap region are determined. A perceived contrast between the real object and the virtual object in the overlap region is decreased based on the determined color characteristics of the real object in the overlap region. The virtual image data is displayed as a virtual image after the perceived contrast between the real object and the virtual object has been decreased that, along with the visualized direct light, creates a three-dimensional augmented scene.

Direct light compensation technique for augmented reality system

User interface display layouts are provided that draw a user's attention to a specific element or elements by de-emphasizing the surrounding content, but without removing the de-emphasized content from the interface. This ability to maintain the whole presentable layout with visibility layers and without layout changes provides a useful navigation experience for the user as it is clear where the user's attention should go and yet the surrounding content is still subtly there, constantly reminding the user of the other available content. De-emphasis of certain content items is achieved by modifying display characteristics of those content items relative to a base display level, for example by lowering saturation, lowering opacity, and/or de-focusing (as if the user is looking through a camera) and modification can be done variably. Driven by a relevancy score, each content item in a display layout can be de-emphasized more or less depending on which content is more meaningful to the user's filtering actions.

Variable de-emphasis of displayed content based on relevance score

A virtual image generation system and method of operating same are provided. A left synthetic image and a right synthetic image of a three-dimensional scene are rendered respectively from a first left focal center and a first right focal center relative to a first viewpoint. The first left and first right focal centers are spaced from each other a distance greater than the inter-ocular distance of an end user. The synthetic image and the right synthetic image are warped respectively to a second left focal center and a second right focal center relative to a second viewpoint different from the first viewpoint. The second left and right focal centers are spaced from each other a distance equal to the inter-ocular distance of the end user. A frame is constructed from the left and right warped synthetic images, and displayed to the end user.

Wide baseline stereo for low-latency rendering

Described are improved approaches to implement color sequential displays that can mitigate problems with conventional display technologies. Color-breakup is mitigated by modifying the original color channels and adding one or more additional color channels derived from the original ones.

Color sequential display

An image review system presents an image to a reviewer and records reviewer feedback, including reviewer position indicators relating to portions of the image and reviewer commentary data, which can be audio, visual, textual, etc. The reviewer position indicators and reviewer commentary data each have a time component, corresponding to when the reviewer indicated a position on the image and when the reviewer provided the commentary, respectively. During playback, the viewer can select a subset of the image of interest and get a playback of the reviewer commentary data that corresponds to the times represented by the reviewer position indicators that fall within the selected subset of the image. The reviewer position indicators can be collectively indicated by a heat map that, when laid over the image, indicates which portions of the image are commented on and how much time is spent there.

Subselection of portions of an image review sequence using spatial or other selectors

The disclosure provides an approach for stylizing animations to synthesize example textures. In one embodiment, a synthesis application down-samples input and style buffers. To obtain a sequence of offset fields, each of which takes pixels in the output stylized frame to corresponding pixels in the stylized example image, the synthesis application may optimize each frame of the animation at level l&#x2212;1, then advect the results of a previous frame to a next frame using velocity fields. After having processed the entire animation sequence forward through time, a similar sweep may be performed backwards. Then, the resulting offset fields may be up-sampled to level l and used as the starting point for optimization at that finer level of detail. This process may be repeated until returning to the original sampling, which yields the final output.

Stylizing animation by example

An animation processor comprises a processor, an instruction memory, a working memory, inputs and outputs, and an input for receiving an input animation sequence comprising a plurality of time-sequenced frames, storage for a style image pair, comprising an input style image and an output style image, wherein differences between the input style image and an output style image convey a style for drawings and other visual elements, logic for generating set of time-varying orientation fields, logic for rendering velocity fields for the sequences, and an output for outputting an output animation sequence that takes on the style specified by the style image pair, with the output animation sequence having temporal coherence.

Generating animation using image analogies to animate according to particular styles

A method for generating two-dimensional image space noise. The method includes receiving a first frame of a rendered animation sequence; receiving a second frame of the rendered animation sequence; and generating a frame of noise that is associated with at least one object moving between the first frame and the second frame and is to be applied to the second frame, where generating the frame of noise comprises applying, for each pixel in the second frame, a causal filter based on a depth associated with the pixel and a velocity associated with the pixel.

Coherent noise for non-photorealistic rendering

A model is associated with a deep pose. When the model is changed from an attractor pose to a current pose, the current pose and the attractor pose are compared with the deep pose. If any portion of the current pose is more similar to the deep pose than the attractor pose, then the attractor pose is updated. A portion of the attractor pose may be set to the corresponding portion of the current pose. The attractor pose may be modified by a function. Pose attributes of each pose degrees of freedom for the attractor pose, the current pose, and the deep pose may be evaluated to potentially modify all or a portion of the attractor pose. The attractor pose and pose constraints are used to determine a pose of the model, for example by an optimization process based on the attractor pose while satisfying pose constraints.

Inverse kinematic melting for posing models

An edge-preserving diffusion filter maintains the sharp edges in images while smoothing out image noise. An edge-preserving diffusion filter applies an edge-preserving smoothing filter to an image to form a filtered image. The modified image is then blurred by a blurring filter to form a blurred image. The modified image and the blurred image are blended together to form an output image based on an error metric associated with each pixel. The edge-preserving diffusion filter may be utilized to perform a multilevel decomposition of the image. The edge-preserving diffusion filter may be applied to an unfiltered image to produce a base image. The difference between the unfiltered image and the base image defines a detail image. The detail image may be used as the input for recursively generating additional levels of detail. The multilevel decomposition may utilize filter kernels associated with different contrast levels for each iteration.

Selective diffusion of filtered edges in images

An animation creator authors a transition between two different pre-authored motions of a degree of freedom of an object such as an animatronic or virtual character. The author selects an arbitrary position to begin the transition and the transition ends at the position where a subsequent state of a degree of freedom of the object begins. The selected position is arbitrary because an initial state of the degree of freedom of the object is not known until the complete motion of the object is determined. The transition is transformed to begin where the initial state ends and to end where the subsequent state begins. The transition is transformed to retain the general shape of the authored transition and to provide the same quality of motion as the pre-authored motions. The transformed transition may be scaled in time such that limits of velocity and/or acceleration for the object are not violated.

Authored motion transitions

Local histogram and local histogram-based functions can be determined by generating offset-kernel images based on domain-shifted tonal filter kernels. The offset-kernel images can be reused for multiple image locations and/or local neighborhood sizes, shapes, and weights. A neighborhood filter representing the desired local neighborhood size, shape, and frequency domain characteristics is applied to each of the offset-kernel images. Neighborhood filters may include a temporal dimension for evaluating neighborhoods in space and time. Neighborhood filtered offset-kernel images' values represent samples of local histogram or local histogram-based function corresponding with the domains of their associated domain-shifted tonal filter kernels. Arbitrary functions may be used as tonal filter kernels. A histogram kernel may be used to sample values of local histogram functions. A tonal filter kernel that is a derivative or integral of another tonal filter kernel may be used to sample a derivative or integral, respectively, of a function.

Smoothed local histogram filters for computer graphics

Local histogram-based filters may create non-photorealistic computer graphics effects. An edge-preserving smoothing filter processes a source image to generate a smoothed image. An orientation filter processes the source image to generate a smoothed orientation image. At least one of the orientation filter and the edge-preserving smoothing filter include a local histogram-based filter. The non-photorealistic effect filter may generate a fine detail image from the smoothed image and the source image. The fine detail image may be diffused according to the smoothed orientation image to generate a brush stroke image. The brush stroke image is combined with a base image to create a non-photorealistic image. Local histogram-based filters include a closest mode, a local median, and a dominant mode filter. Local histogram-based filters are determined by convolving a neighborhood filter with one or more offset-kernel images, each based on the application of a shifted tonal filter kernel to the source image.

Painterly filtering

An input device for controlling an object includes a joystick and a modal switch. A user may use the modal switch to select a subset of degrees of freedom of the object. The joystick may then be used to control a change over time of the selected subset, where the change over time is functionally depend on both a motion of the joystick and a state of the selected subset. A method for controlling an object via the input device is also provided. The method includes receiving inputs indicating a selection by the modal switch of a subset of degrees of freedom of the object, and a motion of the joystick. A configuration of the selected subset is then caused to be changed based on the motion of the joystick and a state of the selected subset.

Growing community of inventors

Berkeley, CA, United States of America

Michael Kass