Apple Inc.

Youspace, Inc.

Humane, Inc.

Ralph T Brunner

Peter N Graffagnino

John Harper

Mark Zimmer

Nikhil Bhogal

David Hayward

James David Batson

Haroon S Sheikh

Imran A Chaudhri

Frank Doepke

Jeremy B Jones

Timothy David Cherna

Andrew R Barnes

John Owen Louch

Haitao Guo

Rudolph Van Der Merwe

Eric Steven Peyton

John Philip Stoddard

Kok S Chen

Ali T Ozer

Howard Aron Miller

Christopher J Hynes

Samuel Noble

Assana Fard

Hsin-Yi Chien

Michael Chin

Monique Relova

Po-Jui Chen

Vivi Brunner

Guy Fullerton

Charles G Fleming

Scott James Forstall

Guy Cote

Suk Hwan Lim

Randy Ubillos

Ting Chen

Paul M Hubel

Jianping Zhou

Farhan A Baqai

Marius Tico

Patrick G Gates

Jean-Pierre Ciudad

Rolf Toft

Claus Molgaard

Chendi Zhang

Bethany Bongiorno

Michael Stochosky

Anita Nariani Schulze

Alexander Lindskog

Lilynaz Hashemi

Benjamin M Olson

Jeremy Spencer Jones

Yanir Nulman

Nathan Lord

Wangju Tsai

Yichen Pan

Andrew Emmett Seligman

Elon Sharton-Bierig

Sierra Justine Gaston

Anita Nariani-Schulze

Pater Graffagnino

Kurt Knight

Monique Perez Relova

Kristin Webster

Britt Nelson

John S Harper

Peter M Graffagnino

Mark Alan Zimmer

Systems, methods, devices and non-transitory, computer-readable storage mediums are disclosed for gesture recognition for a wearable multimedia device using real-time data streams. In an embodiment, a method comprises: detecting a trigger event from one or more real-time data streams running on a wearable multimedia device; taking one or more data snapshots of the one or more real-time data streams; inferring user intent from the one or more data snapshots; and selecting a service or preparing content for the user based on the inferred user intent. In an embodiment, a hand and finger pointing direction is determined from a depth image, a 2D bounding box for the hand/finger is projected into a 2D image space and compared to bounding boxes for identified/labeled objects in the 2D image space to identify an object that the hand is holding or the finger is pointing toward.

Gesture recognition for wearable multimedia device using real-time data streams

Techniques are disclosed for creating scaled images with super resolution using neighborhood patches of pixels to provide higher resolution than traditional interpolation techniques. Also disclosed are techniques for creating super field-of-view (FOV) images of a scene created from previously captured and stored images of the scene that are stitched together with a current image of the scene, to generate an image of the scene that extends beyond the fixed FOV of the camera.

Super resolution/super field of view (FOV) digital photography

A graphics animation and compositing operations framework has a layer tree for interfacing with the application and a render tree for interfacing with a render engine. Layers in the layer tree can be content, windows, views, video, images, etc., for an application's user interface. The application commits state changes of the layers of the layer tree. The application does not need to include explicit code for animating the changes to the layers. Instead, after a synchronization threshold has been met, an animation is determined for animating the change in state by the framework which can define a set of predetermined animations based on motion, visibility and transition. The determined animation is explicitly applied to the affected layers in the render tree. A render engine renders from the render tree into a frame buffer, synchronized with the display. Portions of the render tree changing relative to prior versions can be tracked.

Frameworks for graphics animation and compositing operations

At least certain embodiments described herein provide a continuous autofocus mechanism for an image capturing device. The continuous autofocus mechanism can perform an autofocus scan for a lens of the image capturing device and obtain focus scores associated with the autofocus scan. The continuous autofocus mechanism can determine an acceptable band of focus scores based on the obtained focus scores. Next, the continuous autofocus mechanism can determine whether a current focus score is within the acceptable band of focus scores. A refocus scan may be performed if the current focus score is outside of the acceptable band of focus scores.

Continuous autofocus mechanisms for image capturing devices

A graphics animation and compositing operations framework has a layer tree for interfacing with the application and a render tree for interfacing with a render engine. Layers in the layer tree can be content, windows, views, video, images, text, media or other type of objects for an application's user interface. The application commits state changes of the layers of the layer tree. The application does not need to include explicit code for animating the changes to the layers. Instead, after a synchronization threshold has been met, an animation is determined for animating the change in state by the framework which can define a set of predetermined animations based on motion, visibility and transition. The determined animation is explicitly applied to the affected layers in the render tree. A render engine renders from the render tree into a frame buffer, synchronized with the display. Portions of the render tree changing relative to prior versions can be tracked to improve resource management.

Framework for graphics animation and compositing operations

Methods, devices, and systems for continuous image capturing are described herein. In one embodiment, a method includes continuously capturing a sequence of images with an image capturing device. The method may further include storing a predetermined number of the sequence of images in a buffer. The method may further include receiving a user request to capture an image. In response to the user request, the method may further include automatically selecting one of the buffered images based on an exposure time of one of the buffered images. The sequence of images is captured prior to or concurrently with receiving the user request.

Image capturing device having continuous image capture

A user interface can have one or more spaces presented therein. A space is a grouping of one or more program windows in relation to windows of other application programs, such that the program(s) of only a single space is visible when the space is active. A view can be generated of all spaces and their contents.

User interface spaces

Various of the disclosed embodiments provide Human Computer Interfaces (HCI) that incorporate depth sensors at multiple positions and orientations. The depth sensors may be used in conjunction with a display screen to permit users to interact dynamically with the system, e.g., via gestures. Calibration methods for orienting depth values between sensors are also presented. The calibration methods may generate both rotation and translation transformations that can be used to determine the location of a depth value acquired in one sensor from the perspective of another sensor. The calibration process may itself include visual feedback to direct a user assisting with the calibration. In some embodiments, floor estimation techniques may be used alone or in conjunction with the calibration process to facilitate data processing and gesture identification.

Calibration systems and methods for depth-based interfaces with disparate fields of view

Disclosed is a system for producing images including techniques for reducing the memory and processing power required for such operations. The system provides techniques for programmatically representing a graphics problem. The system further provides techniques for reducing and optimizing graphics problems for rendering with consideration of the system resources, such as the availability of a compatible GPU.

System for optimizing graphics operations

Various of the disclosed embodiments present systems and methods for distinguishing portions of a virtual model associated with a clothing article from portions of the virtual model not associated with the clothing article. Some embodiments facilitate quick and effective separation by employing a feature vector structure conducive to separation by a linear classifier. Such efficient separation may be especially beneficial in applications requiring the rapid scanning of large quantities of clothing while retaining high-fidelity representations of the clothing's geometry. Some embodiments further accommodate artist participation in the filtering process as well as scanning of articles from a variety of orientations and with a variety of supporting structures.

Object separation for scanned assets

Several methods for operating a built-in digital camera of a portable, handheld electronic device are described. In one embodiment, the device receives a user selection (e.g., tap, tap and hold, gesture) of a region displayed on the display screen (e.g., touch sensitive screen). A touch to focus mode may then be initiated in response to the user selection and exposure and focus parameters determined and adjusted. Then, an automatic scene detection mechanism can determine whether a scene has changed. If the scene has changed, then the touch to focus mode ends and a default automatic focus mode initiates. This mode sets a new exposure metering area and focus area prior to determining and adjusting exposure and focus parameters, respectively.

Image capturing device with touch screen for adjusting camera settings

Human Computer Interfaces (HCI) may allow a user to interact with a computer via a variety of mechanisms, such as hand, head, and body gestures. Various of the disclosed embodiments allow information captured from a depth camera on an HCI system to be used to recognize such gestures. Particularly, by training a classifier using vectors having both base and extended components, more accurate classification results may be subsequently obtained. The base vector may include a leaf-based assessment of the classification results from a forest for a given depth value candidate pixel. The extended vector may include additional information, such as the leaf-based assessment of the classification results for one or more pixels related to the candidate pixel. Various embodiments employ this improved structure with various optimization methods and structure to provide more efficient in-situ operation.

Depth-value classification using forests

Human Computer Interfaces (HCI) may allow a user to interact with a computer via a variety of mechanisms, such as hand, head, and body gestures. Various of the disclosed embodiments allow information captured from a depth camera on an HCI system to be used to recognize such gestures. Particularly, the HCI system's depth sensor may capture depth frames of the user's movements over time. To discern gestures from these movements, the system may group portions of the user's anatomy represented by the depth data into classes. 'Features' which reflect distinguishing features of the user's anatomy may be used to accomplish this classification. Some embodiments provide improved systems and methods for generating and/or selecting these features. Features prepared by various of the disclosed embodiments may be less susceptible to overfitting training data and may more quickly distinguish portions of the user's anatomy.

Depth-based feature systems for classification applications

Various of the disclosed embodiments present depth-based user interaction systems. By anticipating various installation constraints and factors into the interface's design, various of the disclosed embodiments facilitate benefits such as complementary depth fields of view and display orientation flexibility. Some embodiments include a frame housing for the depth sensors. Within the housing, depth sensors may be affixed to a mount, such that each sensor's field of view is at a disparate angle. These disparate angles may facilitate gesture recognitions that might otherwise be difficult or impossible to achieve. When mounted in connection with a modular unit, the housing may provide a versatile means for integrating multiple module units into a composite interface.

Interactive systems for depth-based input

Growing community of inventors

Santa Clara, CA, United States of America

Ralph T Brunner