Amazon Technologies, Inc.

Ricoh Company, Ltd.

Beijing Voyager Technology Co., Ltd.

Meta Platforms, Inc.

Yue Liu

Avnish Sikka

Kurt Knodt

Hitoshi Sekine

David Paul Ramos

Emilie Noelle McConville

Qingfeng Yu

Matthew Joseph Cole

Yao-Tian Wang

Ronald Marc Conescu

Sonjeev Jahagirdar

Utkarsh Prateek

Arindam Mandal

Ning Yao

Senthil K Selvaraj

James David Meyers

Alain Regnier

Pradeep Natarajan

Zhenning Xiao

Loc T Le

Matthew Daniel Hart

Robert Ramsey Flenniken

Shah Samir Pravinchandra

Xing Liu

Alexander J Kolmykov-Zotov

Amit Singh Chhetri

Ashish Kumar Agrawal

Shiv Naga Prasad Vitaladevuni

Mashhour Solh

Oleg Rybakov

Daniel N Miller

Lifen Tian

Ramya Gopalan

Bhavesh Anil Doshi

Yuwen Wu

Yi Ding

Vijay Kamarshi

Sowmya Gopalan

Ramy Sammy Sadek

Rahul Agrawal

Shuang Wu

Thibaud Senechal

Soiba Mohammad

Robert Warren Sjoberg

David A Williams

Ahmed Aly

Bin Li

Laura Varnum Finney

Gordon Lee

Denis Savenkov

Arlen R Dean

Ankur Datta

Tong Wang

Michael Vanne

Shuyan Dong

Zhichu Lu

Daniel Miller

Arlen Dean

Pooja Sethi

Han Wang

Praveen Jayakumar

Bryan Taylor

Quan Wang

Daniel Makoto Wilenson

Paul Matthew Bombach

Helder Jesus Ramalho

Daniel Makoto Willenson

Cautious driving and cautious driving speed determination for an autonomous vehicle is responsive to receiving a non-yield backup prediction for the vehicle regarding a traffic participant in a region of interest in a road network surrounding the vehicle, the non-yield backup prediction including a non-yield probability value for the traffic participant not yielding to the vehicle. Driving information, including speed, for other traffic participants within the region of interest is obtained from a sensor system, and an average speed of the other traffic participants is determined. A driving system determines a cautious driving speed for the vehicle by calculating a reverse probability value, which is a reverse percentage of the non-yield probability value relative to a maximum value for it, and multiplying the average speed of the other traffic participants by the reverse probability value. The driving system controls the vehicle to reduce its speed to the cautious driving speed.

Speed generation in cautious driving for autonomous vehicles

Devices and techniques are generally described for determining named entity recognition tags. In various examples, first input data representing a natural language input may be determined. In some examples, a first machine learned model may determine first data comprising a first encoded representation of the first input data. In various examples, second data representing a grouping of text of the first input data may be determined based at least in part on the first data. In some examples, first entity data may be determined by searching a memory layer using the second data. In at least some examples, the first entity data and the first data may be combined to generate third data. In various examples, output data comprising a predicted named entity recognition tag may be generated for the grouping of text based at least in part on the third data.

Natural language processing

A system may use multiple speech interface devices to interact with a user by speech. All or a portion of the speech interface devices may detect a user utterance and may initiate speech processing to determine a meaning or intent of the utterance. Within the speech processing, arbitration is employed to select one of the multiple speech interface devices to respond to the user utterance. Arbitration may be based in part on metadata that directly or indirectly indicates the proximity of the user to the devices, and the device that is deemed to be nearest the user may be selected to respond to the user utterance.

Device selection for providing a response

In one embodiment, a method includes receiving a user request to automatically debug a natural-language understanding (NLU) model, accessing a plurality of predicted semantic representations generated by the NLU model, wherein the plurality of predicted semantic representations are associated with a plurality of dialog sessions, respectively, wherein each dialog session is between a user and an assistant xbot associated with the NLU model, generating a plurality of expected semantic representations associated with the plurality of dialog sessions based on an auto-correction model, wherein the auto-correction model is learned from dialog training samples generated based on active learning, identifying incorrect semantic representations of the predicted semantic representations based on a comparison between the predicted semantic representations and the expected semantic representations, and automatically correcting the incorrect semantic representations by replacing them with respective expected semantic representations generated by the auto-correction model.

Continuous learning for natural-language understanding models for assistant systems

A computer-implemented method for controlling a vehicle comprises: receiving tracking data associated with a surrounding environment of the vehicle; detecting, based upon the tracking data, an object in the surrounding environment of the vehicle; determining a location of the object; determining, based on navigation assistance data, whether the location of the object is at least partially within a classified area in the surrounding environment; and configuring a control system of the vehicle to: initiate, based upon determining that the location of the object is not at least partially within the classified area, a first collision avoidance response procedure for responding to the object; and initiate, based upon determining that the location of the object is at least partially within the classified area, a second collision avoidance response procedure for responding to the object, the second collision avoidance response procedure different from the first collision avoidance response procedure.

Dynamically modifying collision avoidance response procedure in autonomous vehicles

Techniques for processing complex natural language inputs are described. A complex natural language input may be semantically tagged and parsed to identify individual clauses in the complex natural language input. An execution graph may be generated to represent the clauses and their dependencies. Nodes of the execution graph may be processed using NLU processing and/or a knowledge graph or other information storage and retrieval techniques, and results of such processing may be used to update clause variables with specific entities in the execution graph.

Complex natural language processing

A number of images of an environment may be obtained over time by a number of sensors. The images may include an input surface and an input object, such as a hand or stylus, that may be utilized by a user to provide an input via the input surface. In some cases, the input may be provided without the input object touching the input surface. The location of the input object with respect to the input surface over time and a speed of the input object over time may be used to determine whether an input has been indicated by the input object without contacting the input surface.

Detection of an input object on or near a surface

A system for recognizing objects and/or text in image data may use context data to perform object/text recognition. The system may also use context data when determining potential functions to execute in response to recognizing the object/text. Context data may be gathered based on device sensor data, user profile data such as the behavior of a user or the behavior of those in a user's social network, or other factors. Recognition processing and/or function selection may be configured to account for context data when operating to improve output results.

Item recognition using context data

Systems, methods and devices for automatically cropping images taken by an electronic device in order to determine the identity of a product contained in the image are described herein. A number of different techniques may be applied to perform the automatic cropping, including a focus sweep technique in which a first image is analyzed for the presence of a human being and then a second image is taken in a plane closer to the camera than the first image. The two frames are analyzed and a resultant image is provided that avoids the regions in which the human being is present to focus on the product. In other embodiments, a motion vector calculation is made between two images in which an individual is holding a product. The motion vectors related to the human are removed and a bounding box is calculated to reduce the size of the image to include a higher percentage of the product, such that the product can be more easily identified.

Systems and methods for identifying objects

A system and method of determining a tilt angle of a portable computing device using a sensor indicating gravitational pull on the device; determining the tilt angle of a camera of the device; identifying a tilt angle range from a plurality of predetermined tilt angle ranges; determining a first focal length setting using a first array that associates the tilt angle range with the first focal length setting; determining an adjustment increment using a second array that associates the adjustment increment with the tilt angle range; and determining a second focal length setting of the camera using the adjustment increment according to an autofocus scan range algorithm. A portable computing device including a processor; a camera; and a memory device including instructions operable to be executed by the processor to perform a set of actions, enabling the portable computing device to perform the method.

Determining camera auto-focus settings

A system and method of determining a tilt angle of a portable computing device using sensor data; identifying a tilt angle from a plurality of predetermined tilt angle ranges; determining focal length settings for image capture devices of the portable computing device using the tilt angle, adjustment increments, and autofocus scan range algorithms. A portable computing device including a processor; a first image capture device on a first side of the portable computing device, and a second image capture device on the second side of the portable computing device, the second side located opposite of the first side; and a memory device including instructions operable to be executed by the processor to perform a set of actions, enabling the portable computing device to perform the method.

In a speech-based system, a wake word or other trigger expression is used to preface user speech that is intended as a command. The system receives multiple directional audio signals, each of which emphasizes sound from a different direction. The signals are monitored and analyzed to detect the directions of interfering audio sources such as televisions or other types of electronic audio players. One of the directional signals having the strongest presence of speech is selected to be monitored for the trigger expression. If the directional signal corresponds to the direction of an interfering audio source, a more strict standard is used to detect the trigger expression. In addition, the directional audio signal having the second strongest presence of speech may also be monitored to detect the trigger expression.

Mitigating effects of electronic audio sources in expression detection

A system that selects image frames from a video feed for recognition of objects (such as physical objects, text characters, or the like) within the image frames. The individual frames are selected using robust historical metrics that compare individual metrics of the particular image (such as focus, motion, intensity, etc.) to similar metrics of previous image frames in the video feed. The system will select the image frame for object recognition if the image frame is relatively high quality, that is the image frame is suitable for a later object recognition processing.

Image selection and recognition processing from a video feed

A system to select video frames for optical character recognition (OCR) based on feature metrics associated with blur and sharpness. A device captures a video frame including text characters. An edge detection filter is applied to the frame to determine gradient features in perpendicular directions. An 'edge map' is created from the gradient features, and points along edges in the edge map are identified. Edge transition widths are determined at each of the edge points based in local intensity minimum and maximum on opposite sides of the respective edge point in the frame. Sharper edges have smaller edge transition widths than blurry images. Statistics are determined from the edge transition widths, and the statistics are processed by a trained classifier to determine if the frame is or is not sufficiently sharp for text processing.

Sharpness-based frame selection for OCR

Growing community of inventors

San Jose, CA, United States of America

Yue Liu