Amazon Technologies, Inc.

Jim Oommen Thomas

Cheng-Hao Kuo

Paul Aksenti Savastinuk

William Evan Welbourne

Ross David Roessler

Yinfei Yang

Sharadh Ramaswamy

Kah Kuen Fu

Amit Kumar Agrawal

Ambrish Tyagi

Geoffrey Heller

Stephen Vincent Mangiat

Tsz Ho Yu

Tianyang Ma

Sisil Sanjeev Mehta

Chang Yuan

Oleg Rybakov

Jingjing Zheng

Matthew Alan Townsend

Tarun Yohann Morton

Michael Lee Sandige

Deon Poncini

Junwu Luo

Geoff Hunter Donaldson

Che-Chun Su

Songyao Jiang

Christopher John Lish

Lu Xia

Ning Zhou

Jiajia Luo

Shuang Gao

Jingyi Zhang

Rakesh Ramesh

Anuja Shantaram Dawane

Yuyin Sun

Peng Lei

A stylus provides input via a touchscreen comprising a touch sensor and a display. Latency between placement of a stylus tip and corresponding presentation of visual indicia on the display is reduced or eliminated by determining a predicted path of the stylus tip during a stroke. Visual indicia is presented on the display, based on the predicted path. Inputs from the touch sensor may include hover events associated with detection of the tip while not in contact with the touchscreen and touch events associated with presence of the tip on the touchscreen. A machine learning network may be trained to determine the predicted path. A portion of the network may be trained to accept dynamic-length sequences of events and generate fixed length sequences, reducing subsequent network complexity. A hand may be detected and used to determine the predicted path. The end of a stroke may be predicted, reducing overshoot.

Stylus input latency compensation system

An autonomous mobile device (AMD) may perform various tasks during operation. Some tasks, such as delivering a message to a particular user, may involve the AMD identifying the particular user. The AMD includes a camera to acquire an image, and image-based authentication techniques are used to determine a user's identity. A user may move within in a physical space, and the space may contain various obstructions which may occlude images. The AMD may move within the space to obtain a vantage point from which an image of the face of the user is obtained which is suitable for image-based authentication. In some situations, the AMD may present an attention signal, such as playing a sound from a speaker or flashing a light, to encourage the user to look at the AMD, providing an image for use in image-based authentication.

System to facilitate user authentication by autonomous mobile device

A device capable of autonomous motion may process image data determined by one or more cameras to determine one or more properties of objects represented in the image data. The device may determine that two or more computer vision components correspond to a particular property. A first computer vision component may process the image data to determine first output data, and the second computer vision component may process the first output data to determine second output data corresponding to the property.

Autonomously motile device with computer vision

An autonomous mobile device (AMD) or other device may perform various tasks during operation. The AMD includes a camera to acquire an image. Some tasks, such as presenting information on a display screen or a video call, may involve the AMD determining whether a user is engaged with the AMD. The AMD may move a component, such as the camera or the display screen, to provide a best experience for an engaged user. Images from the camera are processed to determine attributes of the user, such as yaw of the face of the user, pitch of the face of the user, distance from the camera, and so forth. Based on the values of these attributes, a user engagement score is determined. The score may be used to select a particular user from many users in the image, or to otherwise facilitate operation of the AMD.

System to determine user engagement with autonomous mobile device

Devices, systems and methods are disclosed for improving facial recognition and/or speaker recognition models by using results obtained from one model to assist in generating results from the other model. For example, a device may perform facial recognition for image data to identify users and may use the results of the facial recognition to assist in speaker recognition for corresponding audio data. Alternatively or additionally, the device may perform speaker recognition for audio data to identify users and may use the results of the speaker recognition to assist in facial recognition for corresponding image data. As a result, the device may identify users in video data that are not included in the facial recognition model and may identify users in audio data that are not included in the speaker recognition model. The facial recognition and/or speaker recognition models may be updated during run-time and/or offline using post-processed data.

User identification based on voice and face

A video capture device may include multiple cameras that simultaneously capture video data. The video capture device and/or one or more remote computing resources may stitch the video data captured by the multiple cameras to generate stitched video data that corresponds to 360&#xb0; video. The remote computing resources may apply one or more algorithms to the stitched video data to identify one or more frames that depict content that is likely to be of interest to a user. The video capture device and/or the remote computing resources may generate one or more images from the one or more frames, and may send the one or more images to the user.

Panoramic image generation from video

A viewing device, such as a virtual reality headset, allows a user to view a panoramic scene captured by one or more video capture devices that may include multiple cameras that simultaneously capture 360&#xb0; video data. The viewing device may display the panoramic scene in real time and change the display in response to moving the viewing device and/or changing perspectives by switching to video data being captured by a different video capture device within the environment. Moreover, multiple video capture devices located within an environment can be used to create a three-dimensional representation of the environment that allows a user to explore the three-dimensional space while viewing the environment in real time.

Remote immersive user experience from panoramic video

A video capture device may include multiple cameras that simultaneously capture video data. The video capture device and/or one or more remote computing resources may stitch the video data captured by the multiple cameras to generate stitched video data that corresponds to 360&#xb0; video. The remote computing resources may apply one or more algorithms to the stitched video data to adjust the color characteristics of the stitched video data, such as lighting, exposure, white balance contrast, and saturation. The remote computing resources may further smooth the transition between the video data captured by the multiple cameras to reduce artifacts such as abrupt changes in color as a result of the individual cameras of the video capture device having different video capture settings. The video capture device and/or the remote computing resources may generate a panoramic video that may include up to a 360&#xb0; field of view.

Color adjustment of stitched panoramic video

Various embodiments enable a primary user to be identified and tracked using stereo association and multiple tracking algorithms. For example, a face detection algorithm can be run on each image captured by a respective camera independently. Stereo association can be performed to match faces between cameras. If the faces are matched and a primary user is determined, a face pair is created and used as the first data point in memory for initializing object tracking. Further, features of a user's face can be extracted and the change in position of these features between images can determine what tracking method will be used for that particular frame.

Primary user selection for head tracking

Devices, systems and methods are disclosed for identifying content in video data and creating content-based zooming and panning effects to emphasize the content. Contents may be detected and analyzed in the video data using computer vision, machine learning algorithms or specified through a user interface. Panning and zooming controls may be associated with the contents, panning or zooming based on a location and size of content within the video data. The device may determine a number of pixels associated with content and may frame the content to be a certain percentage of the edited video data, such as a close-up shot where a subject is displayed as 50% of the viewing frame. The device may identify an event of interest, may determine multiple frames associated with the event of interest and may pan and zoom between the multiple frames based on a size/location of the content within the multiple frames.

Content-based zooming and panning for video curation

Object detection and tracking

A device configured with a user interface (UI) that changes based on a position of a user determines the position of the user through multiple data sources including camera based head tracking and output from motion sensors such as a gyroscope. Each data source may output its own estimated head position. The device may apply a reliability weight to the head position determined by each data source. A composite head position is then determined from the weighted position. The composite position is then used to render the UI.

Trust shifting for user position detection

Various embodiments utilize two-dimensional ('2D') and three-dimensional ('3D') object features for purposes such as object recognition and/or image matching. For example, a user can capture an image (e.g., still images or video) of an object and can receive information about items that are determined to match the object. For example, the image can be analyzed to detect visual features (e.g., corners, edges, etc.) of the object and the detected visual features can be combined to generate a combined visual feature vector which can be used for object recognition, image matching, or other such purposes. Other approaches utilize the image to generate a 3D model of the object represented in the image, which can be used to determine at least one object or types of objects that match the object represented in the image.

Object recognition for three-dimensional bodies

Object identification through stereo association

Growing community of inventors

Seattle, WA, United States of America

Jim Oommen Thomas