Amazon Technologies, Inc.

Adobe, Inc.

Geoffrey Heller

Chang Yuan

Daniel Bibireata

Richard Howard Suplee, Iii

Paul Aksenti Savastinuk

Louis LeRoi LeGrand, Iii

Francislav Petrov Penov

Jim Oommen Thomas

Kah Kuen Fu

Leo Benedict Baldwin

Aaron Michael Donsbach

Kenneth Mark Karakotsios

Volodymyr V Ivanchenko

Timothy Youngjin Sohn

Oleg Rybakov

Ross David Roessler

Michael Lee Sandige

Kritarth Jain

David P Simons

Sharadh Ramaswamy

Wei Li

Jakub Fiser

Christopher S A MacGregor

Honghong Peng

Charles P Vaughn

Neil David Cooper

Laura Varnum Finney

Christopher John Lish

Michal Marek Kozlowski

Andrew Bartlett Leonard

Saurabh Verma

Certain embodiments involve automatically detecting video frames that depict visemes and that are usable for generating an animatable puppet. For example, a computing device accesses video frames depicting a person performing gestures usable for generating a layered puppet, including a viseme gesture corresponding to a target sound or phoneme. The computing device determines that audio data including the target sound or phoneme aligns with a particular video frame from the video frames that depicts the person performing the viseme gesture. The computing device creates, from the video frames, a puppet animation of the gestures, including an animation of the viseme corresponding to the target sound or phoneme that is generated from the particular video frame. The computing device outputs the puppet animation to a presentation device.

Automatic viseme detection for generating animatable puppet

A mobile device for projecting images on a surface and for maintaining a position of the image on the surface. The device including an image rendering module to scan the surface and to modify the image based on data obtained about a three dimensional nature of the surface. The device further including an optical image stabilization to adjust an alignment of the projector based on orientation data of the device, an electronic image stabilization module to shift the image within a projection area based on the orientation data and an image-tracking module to shift the image within the projection area based on changes in a relative positions of the image and a target.

Self stabilizing projector

Various approaches for providing textual information to an application, system, or service are disclosed. In particular, various embodiments enable a user to capture an image with a camera of a portable computing device. The computing device is capable of taking the image and processing it to recognize, identify, and/or isolate the text in order to forward the text to an application or function. The application or function can then utilize the text to perform an action in substantially real-time. The text may include an email, phone number, URL, an address, and the like and the application or function may be dialing the phone number, navigating to the URL, opening an address book to save contact information, displaying a map to show the address, and so on.

Text recognition driven functionality

A user can capture an image of an object, using a computing device, to obtain information about that object. If a specular highlight (or other saturated region) is detected, the device can attempt to determine a location of a light source associated with the highlight. The device can then provide instructions as to a direction to move in order to reduce the presence of the specular highlight in subsequent images. Multiple images of the object can be captured and analyzed to generate a three-dimensional reconstruction of the environment, whereby a position of the light source can be determined. In other embodiments, movement of the specular reflections in response to movement of the device is used to determine a direction of the light source. In other embodiments, an image of the user is captured to determine the position of the light source based on shadows or reflections on the user.

Image capture guidance to reduce specular reflection effects

A tracking architecture is provided that enables data for gestures and head positions to be provided to both native and non-native clients on a computing device. A pipeline component can obtain the raw image data and sensor data and synchronize that data to be processed to determine, for example, location and/or motion data that may correspond to device input. The data can be processed by separate components, such as an event publisher and an event provider, each capable of filtering the location, motion, and/or raw sensor data to generate a set of event data. The event data then can be published to registered listeners or provided in response to polling requests. Head coordinates, gesture data, and other such information can be passed through one or more interface layers enabling the data to be processed by a non-native client on the device.

Architectures for processing of head tracking on a mobile device

A computing device can detect first and second device motions within a first time period. The first and second device motions may be separated by a second time period. The computing device can determine that the second time period is within a specified time range. The computing device can determine that first and second motion metrics for the first and second device motions, respectively, are likely indicative of first and second taps, respectively, on a surface of the computing device. The computing device can determine that a first coordinate frame of the computing device associated with the first tap is within a range of angular degrees from a second coordinate frame of the computing device associated with the second tap. The computing device can further determine the first tap and the second tap as part of a double tap gesture.

Detecting tap-based user input on a mobile device based on motion sensor data

A device configured with a user interface (UI) that changes based on a position of a user determines the position of the user through multiple data sources including camera based head tracking and output from motion sensors such as a gyroscope. Each data source may output its own estimated head position. The device may apply a reliability weight to the head position determined by each data source. A composite head position is then determined from the weighted position. The composite position is then used to render the UI.

Trust shifting for user position detection

A device may identify a user among multiple individuals detected by comparing a physiological condition, such as a heart rate, of the multiple detected individuals as detected by a camera of the device with a physiological condition of a user of the device using different sensors. A heart rate may be detected by a camera by monitoring blood flow to an individual's face. A heart rate may be detected by a motion sensor by monitoring vibrations of the device that are in an expected frequency range. If a heart rate of a user matches a heart rate of an individual seen by a camera, that individual may be determined to be the user of the device. The position of the individual may be used to then render a user interface.

Detecting a primary user of a device

Various embodiments utilize two-dimensional ('2D') and three-dimensional ('3D') object features for purposes such as object recognition and/or image matching. For example, a user can capture an image (e.g., still images or video) of an object and can receive information about items that are determined to match the object. For example, the image can be analyzed to detect visual features (e.g., corners, edges, etc.) of the object and the detected visual features can be combined to generate a combined visual feature vector which can be used for object recognition, image matching, or other such purposes. Other approaches utilize the image to generate a 3D model of the object represented in the image, which can be used to determine at least one object or types of objects that match the object represented in the image.

Object recognition for three-dimensional bodies

A device may recognize a tilt gesture when a device rotates about an axis and then back again. The gesture may be recognized using a state machine. Recognition of the gesture may be performed based on a context of a device, where the specific movement of the device during a tilt gesture may change based on the context. The tilt gesture may be confirmed using a classifier trained on features describing the gesture and the context.

Tilt gesture detection

A user attempting to obtain information about an object can capture image information including a view of that object, and the image information can be used with a matching or identification process to provide information about that type of object to the user. In order to narrow the search space to a specific category, and thus improve the accuracy of the results and the speed at which results can be obtained, the user can be guided to capture image information with an appropriate orientation. An outline or other graphical guide can be displayed over image information captured by a computing device, in order to guide the user in capturing the object from an appropriate direction and with an appropriate scale for the type of matching and/or information used for the matching. Such an approach enables three-dimensional objects to be analyzed using conventional two-dimensional identification algorithms, among other such processes.

User-guided object identification

The accuracy of a text recognition process can be improved using a set of semantic boosting rules, as may be contained in a sequence or other such arrangement. When text is output from a text recognition process, that text can have alternatives and confidence values for different characters or portions of the string. In order to improve the accuracy, this data can be processed using the organized rules, where rules are applied as long as any preconditions for that rule are satisfied, and each rule has the ability to modify the confidence values or modify one or more of the alternatives. When a result it produced with a minimum confidence level, or all applicable rules have been applied, the result can be provided as a refined text output of the recognition process.

Semantic boosting rules for improving text recognition

Various embodiments provide an image pre-processing method to accelerate image computing tasks. For example, an image is obtained from a camera of a portable computing device. Salient features of the image are identified by a number of techniques, such as identifying feature points, identifying gradients within the image, and other techniques and methods discussed herein. Based at least in part on the identified salient features, a saliency map of the image is computed. The saliency map includes a saliency value for each of at least a portion of the plurality of pixels. In this example pixels above a threshold saliency value are processed with an optical character recognition engine.

Image preprocessing for character recognition

Various embodiments utilize geometric hashing to automatically recognize and track and an object. For example, a user can capture an image of a product, or other object, and a point detection algorithm can identify particular features of the product and designate feature points of the product within the captured image. Hash values are then determined for each feature point by determining a basis for the image and determining the location of each feature point relative to that basis. A hash table including the identified hash values is then created and compared to hash values for either a stored product when performing object recognition or from a reference image when performing object tracking.

Growing community of inventors

Seattle, WA, United States of America

Geoffrey Heller