Amazon Technologies, Inc.

Isaac Scott Noble

Kenneth Mark Karakotsios

Dong Zhou

Matthew Paul Bell

Guenael Thomas Strutt

Leo Benedict Baldwin

Volodymyr V Ivanchenko

Stephen Michael Polansky

Jason Robert Weber

Bradley James Bozarth

Mingjing Huang

Edwin Joseph Selker

Yuzo Watanabe

Jeffrey P Bezos

David Wayne Stafford

Steven Ka Cheung Moy

Dennis R Hodge

Gabrielle M Halberg

Gregory Michael Hart

Howard Look

Sharadh Ramaswamy

Jerry Weiming Kuo

David Allen Fotland

Gunnar Hovden

Ryan H Cassidy

Christopher S A MacGregor

Dmitri Khoklov

Approaches are described which enable a computing device (e.g., mobile phone, tablet computer) to display alternate views or layers of information within a window on the display screen when a user's finger (or other object) is detected to be within a particular range of the display screen of the device. For example, a device displaying a road map view on the display screen may detect a user's finger near the screen and, in response to detecting the finger, render a small window that shows a portion of a satellite view of the map proximate to the location of the user's finger. As the user's finger moves laterally above the screen, the window can follow the location of the user's finger and display the satellite views of the various portions of the map over which the user's finger passes.

Approaches for displaying alternate views of information

A user can make a symbol with their hand, or other such gesture, at a distance from a computing device that can be captured by at least one imaging element of the device. The captured information can be analyzed to attempt to determine the location of distinguishing features of the symbol in the image information. The image information is then compared to hand gesture information stored in, for example, a library of hand gestures for the user. Upon identifying a match, an input to an application executing on the computing device is provided when the image information contains information matching at least one hand gesture with at least a minimum level of certainty. The hand gesture could include a single 'static' gesture, such as a specific letter in sign language, for example, or include two or more 'static' gestures. The gesture could also include motion, such as hand movement.

Gesture recognition for device input

The accuracy of an image matching and/or object identification process can be improved by utilizing a BCM network-based process that maintains higher order relationships between features in an image. A dataset of images can be converted to floating point vectors and then processed using a BCM-based approach. The resulting vectors can be stored as an image library for purposes of matching subsequently received images. When a match is located for a query image, for example, information associated with the matching image can be provided in order to help identify one or more objects in the received query image.

Architectures for object recognition

A user interacting with an electronic device can receive suggestions for applications or services that can help the user with a specific task. The user might enter information that can be processed semantically to determine one or more actions, objects, or other types of information useful in discovering related applications or services, which might be local or remote to that device. One or more of these discovered applications or services is selected to suggest to the user based on any number of criteria, such as user behavior, preferences, location, time of day, etc. Using such an approach, a device can utilize a semantic process to attempt to infer an action or intent relating to a simple or complex task, and can discover and suggest applications or services that can assist with that task even where the user is unaware of, or not looking for, such an application or service.

Dynamic content discoverability

Systems and approaches are provided for presenting incoming notifications to a computing device based on a determined context of the computing device. Various sensors of the device can be used to determine the context of the device with respect to a user, the state of the device itself, or the context of the device with respect to the environment in which the device is situated. The user can then be informed of received notifications in a manner likely to get the user's attention while not being overly obtrusive to the user or others within the vicinity of the user.

Context-aware notifications

A user of a computing device can be authenticated using image information captured by at least one camera of the computing device. In addition to analyzing the image information using a facial recognition algorithm, for example, variations in color of a portion of the captured image information corresponding to a user's face can be monitored over a period of time. The variations can be analyzed to determine whether the captured image information likely corresponds to an actual human user instead of a representation (e.g., photo) of a human user, such as where the chroma variations in at least a red channel occur with an oscillation frequency and amplitude consistent with changes due to a pulse or heartbeat.

User authentication through image analysis

A user can use a finger, or other such object, to provide input to a computing device. The finger can be positioned and/or oriented at a distance from the device, and the device can determine an input that the user is attempting to provide based on a position, motion, or gesture associated with the finger. One or more cameras can capture image information, which can be analyzed to attempt to determine the location and/or orientation of the finger. If the finger is at least partially outside a field of view of the camera(s), the device can use another sensor (e.g., EMF) to determine one or more potential locations of the fingertip. The image information then can be used to determine which of the potential locations actually corresponds to the fingertip by analyzing that portion of the finger or hand represented in the image information.

Fingertip location determinations for gesture input

A user attempting to obtain information about an object can capture image information including a view of that object, and the image information can be used with a matching or identification process to provide information about that type of object to the user. Information about the orientation of the camera and/or device used to capture the image can be provided in order to limit an initial search space for the matching or identification process. In some embodiments, images can be selected for matching based at least in part upon having a view matching the orientation of the camera or device. In other embodiments, images of objects corresponding to the orientation can be selected. Such a process can increase the average speed and efficiency in locating matching images. If a match cannot be found in the initial space, images of other views and categories can be analyzed as well.

Orientation-assisted object recognition

A low distraction interface can be provided for an electronic device by monitoring information such as a gaze direction of a user. A device can be configured to only display information to a user when the user is looking substantially toward the device, and can be configured to present that information in a way that is of minimal distraction to others nearby. The user can control aspects of the display by looking away from the device, continuing to look at the device for a period of time, or otherwise providing input without physically interacting with the device. Such an approach can be beneficial for settings such as business meetings where a user might want to obtain information from the device but does not want to appear inconsiderate by picking up and checking information on the device.

Low distraction interfaces

Image information displayed on an electronic device can be adjusted based at least in part upon a relative position of a viewer with respect to a device. In some embodiments, image stabilization can be provided such that an image remains substantially consistent from the point of view of the viewer, not the display element of the device. The image can be stretched, rotated, compressed, or otherwise manipulated based at least in part upon the relative viewing position. Similarly, the viewer can move relative to the device to obtain different views, but views that are consistent with the viewer looking at an object, for example, through a piece of glass. The device can overlay information on the image that will adjust with the adjusted image. Three-dimensional modeling and display can be used to offset parallax and focus point effects.

Viewer tracking image display

An electronic device can attempt to determine a gaze direction of a user, which can be used to determine an object of interest to the user. Determining the gaze direction helps to reduce the search space and reduce processing requirements for identifying the object. Image information can be captured that includes the object of interest, which then can be analyzed to recognize the type of object. Upon recognizing the object, the user can be provided with information about the object, which in some cases can depend at least in part upon a current context or location of the object. If the object is a networked device, the user can potentially be provided with input options for controlling the device.

Gaze assisted object recognition

The amount of power and processing needed to process gesture input for a computing device can be reduced by utilizing a separate gesture sensor. The gesture sensor can have a form factor similar to that of conventional cameras to reduce costs by being able to utilize readily available low cost parts, but can have a lower resolution and adjustable virtual shutter such that fast motions can be captured and/or recognized by the device. In some devices, a subset of the pixels of the gesture sensor can be used as a motion detector, enabling the gesture sensor to run in a low power state unless there is likely gesture input to process. Further, at least some of the processing and circuitry can be included with the gesture sensor such that various functionality can be performed without accessing a central processor or system bus, thus further reducing power consumption.

Camera based sensor for motion detection

An electronic device can utilize one or more sensors and/or imaging elements to determine the relative position of at least one light source relative to the device. In various embodiments, the intensity of light can be determined in various directions around the device. By determining the relative intensity around the device, an approximate direction of one or more light sources can be determined. Utilizing the relative position of each light source, the electronic device can properly light or shade a graphical object to be rendered by the device, or otherwise process image information captured by the device.

Intensity modeling for rendering realistic images

A user can use a finger, or other such object, to provide input to a computing device. The finger does not have to contact the device, but can be positioned and/or oriented in such a way that the device can determine an input that the user is attempting to provide, such as an element or icon that the user is intended to select. One or more cameras can capture image information, which can be analyzed to attempt to determine the location and/or orientation of the finger. If the finger is at least partially outside a field of view of the camera(s), the device can use a sensor (e.g., EMF) to attempt to determine a location of at least a portion of the finger, which can be used with the image information to determine the location and/or orientation of the finger. Other estimation processes can be used as well.

Growing community of inventors

Soquel, CA, United States of America

Isaac Scott Noble