Amazon Technologies, Inc.

Omron Corporation

Omron Scientific Technologies, Inc.

Ambrish Tyagi

Amit Kumar Agrawal

Rohith Mysore Vijaya Kumar

Yadunandana Nagaraja Rao

Apoorv Chaudhri

Stephen Vincent Mangiat

Suresh Bholabhai Lakhani

Sharadh Ramaswamy

Matthew Alan Townsend

John Drinkard

Eduard Oks

Siddhartha Chandra

Shashank Tripathi

Jim Oommen Thomas

Kah Kuen Fu

Cheng-Hao Kuo

James Rehg

Tianyang Ma

Sisil Sanjeev Mehta

David Allen Fotland

Koichi Kinoshita

Yoshiharu Tani

Abhishek Singh

Visesh Chari

Ilya Vladimirovich Brailovskiy

Timothy Thomas Gray

Wei Li

Prakash Ramu

Visesh Uday Kumar Chari

Takeshi Shoji

Cyrus Fazeli

Nicholas Jialei Su

Viveka Kulharia

Siddhant Ranade

Described are systems and methods directed to a body change journey that utilizes collected two-dimensional ('2D') body images of a body of a user, generation and presentation of a personalized three-dimensional (“3D”) body model of the body of the user based on those 2D body images, the generation and presentation of different predicted personalized 3D body models of the body of the user at different body measurements (e.g., different body fat percentages, different muscle mass amounts, different body weights, etc.), and the generation of a body change journey that guides the user from their current body model to a selected target body model that is represented by a target personalized 3D body model of the body of the user.

Personalized three-dimensional body models and body change journey

Devices, systems and methods are disclosed for improving story assembly and video summarization. For example, video clips may be received and a theme may be determined from the received video clips based on annotation data or other characteristics of the received video data. Individual moments may be extracted from the video clips, based on the selected theme and the annotation data. The moments may be ranked based on a priority metric corresponding to content determined to be desirable for purposes of video summarization. Select moments may be chosen based on the priority metric and a structure may be determined based on the selected theme. Finally, a video summarization may be generated using the selected theme and the structure, the video summarization including the select moments.

Video ingestion and clip creation

Techniques are generally described for object detection in image data. First image data comprising a three-dimensional model representing an object may be received. First background image data comprising a first plurality of pixel values may be received. A first feature vector representing the three-dimensional model may be generated. A second feature vector representing the first plurality of pixel values of the first background image data may be generated. A first machine learning model may generate a transformed representation of the three-dimensional model using the first feature vector. First foreground image data comprising a two-dimensional representation of the transformed representation of the three-dimensional model may be generated. A frame of composite image data may be generated by combining the first foreground image data with the first background image data.

Generation of synthetic image data using three-dimensional models

Devices and techniques are generally described for estimating three-dimensional pose data. In some examples, a first machine learning network may generate first three-dimensional (3D) data representing input 2D data. In various examples, a first 2D projection of the first 3D data may be generated. A determination may be made that the first 2D projection conforms to a distribution of natural 2D data. A second machine learning network may generate parameters of a 3D model based at least in part on the input 2D data and based at least in part on the first 3D data. In some examples, second 3D data may be generated using the parameters of the 3D model.

Three-dimensional pose estimation

Devices and techniques are generally described for weakly-supervised object segmentation in image data. In various examples, a first frame of image data may be received. The first frame may include a first bounding box surrounding a first set of pixels, wherein first subset of pixels of the first set of pixels represent a first object of a first class and wherein second subset of pixels of the first set of pixels represent background image data. Cross-entropy loss may be determined for the first set of pixels. In some examples, a spatial attention map may be determined for the first set of pixels. In further examples, parameters of a convolutional neural network may be determined by modulating the cross-entropy loss for the first set of pixels using the spatial attention map. The convolutional neural network may be used to generate a segmentation map.

Segmentation using attention-weighted loss and discriminative feature learning

Techniques are generally described for object detection in image data. First image data comprising a first plurality of pixel values representing an object and a second plurality of pixel values representing a background may be received. First foreground image data and first background image data may be generated from the first image data. A first feature vector representing the first plurality of pixel values may be generated. A second feature vector representing a first plurality of pixel values of second background image data may be generated. A first machine learning model may determine a first operation to perform on the first foreground image data. A transformed representation of the first foreground image data may be generated by performing the first operation on the first foreground image data. Composite image data may be generated by compositing the transformed representation of the first foreground image data with the second background image data.

Generation of synthetic image data for computer vision models

A system and method for generating preview data from video data and using the preview data to select portions of the video data or determine an order with which to upload the video data. The system may sample video data to generate sampled video data and may identify portions of the sampled video data having complexity metrics exceeding a threshold. The system may upload a first portion of the video data corresponding to the identified portions while omitting a second portion of the video data. The system may determine an order with which to upload portions of the video data based on a complexity of the video data. Therefore, portions of the video data that may require additional processing after being uploaded may be prioritized and uploaded first. As a result, a latency between the video data being uploaded and a video summarization being received is reduced.

Preview streaming of video data

Devices, systems and methods are disclosed for reducing a perceived latency associated with uploading and annotating video data. For example, video data may be divided into video sections that are uploaded individually so that the video sections may be annotated as they are received. This reduces a latency associated with the annotation process, as a portion of the video data is annotated before an entirety of the video data is uploaded. In addition, the annotation data may be used to generate a master clip table and extract individual video clips from the video data.

A system and method for selecting portions of video data from preview video data is provided. The system may extract image features from the preview video data and discard video frames associated with poor image quality based on the image features. The system may determine similarity scores between individual video frames and corresponding transition costs and may identify transition points in the preview video data based on the similarity scores and/or transition costs. The system may select portions of the video data for further processing based on the transition points and the image features. By selecting portions of the video data, the system may reduce a bandwidth consumption, processing burden and/or latency associated with uploading the video data or performing further processing.

Frame selection of video data

According to one aspect of the teachings herein, a method and apparatus detect mechanical misalignments in a machine vision system during run-time operation of the machine vision system, and compensate image processing based on the detected misalignments, unless the detected misalignments are excessive. Excessive misalignments may be detected by determining a worst-case error based on them. If the worst-case error exceeds a defined limit, the machine vision system transitions to a fault state. The fault state may include disrupting operation of a hazardous machine or performing one or more other fault-state operations. Among the detected misalignments are internal misalignments within individual cameras used for imaging, and relative misalignments between cameras. The method and apparatus may further perform run-time verifications of focus and transition the machine vision system to a fault state responsive to detecting insufficient focal quality.

Method and apparatus for detecting and mitigating mechanical misalignments in an optical system

Various embodiments enable a primary user to be identified and tracked using stereo association and multiple tracking algorithms. For example, a face detection algorithm can be run on each image captured by a respective camera independently. Stereo association can be performed to match faces between cameras. If the faces are matched and a primary user is determined, a face pair is created and used as the first data point in memory for initializing object tracking. Further, features of a user's face can be extracted and the change in position of these features between images can determine what tracking method will be used for that particular frame.

Primary user selection for head tracking

The techniques described herein may identify images that likely depict one or more items by comparing features of the items to features of different regions-of-interest (ROIs) of the images. For instance, some of the images may include a user, and the techniques may define multiple regions within the image corresponding to different portions of the user. The techniques may then use a trained convolutional neural network or any other type of trained classifier to determine, for each region of the image, whether the region depicts a particular item. If so, the techniques may designate the corresponding image as depicting the item and may output an indication that the image depicts the item. The techniques may perform this process for multiple images, outputting an indication of each image deemed to depict the particular item.

Identifying items in images using regions-of-interest

Growing community of inventors

Palo Alto, CA, United States of America

Ambrish Tyagi