Toyota Motor Engineering & Manufacturing North America, Inc.

Waymo LLC

Toyota Research Institute, Inc.

Toyota Jidosha Kabushiki Kaisha

Tiffany L Chen

Rajiv Dayal

Douglas Alan Moore

Joseph M A Djugash

Kenichi Yamamoto

Patrick K Ching

Chen Wu

Yasuhiro Ota

Alper Ayvaci

Christopher P Lee

Zhinan Xu

Lo Po Tsui

Sean Rafferty

Changchang Wu

Justin Thorsen

Brandon D Northcutt

Guy Rosman

Suhas E Chelian

Stephanie Paepcke

Simon A I Stent

Daniel J Brooks

Robert F Tow

Jonathan Decastro

Shabnam Hakimi

Youenn Colin

Emily Sarah Sumner

Deepak Edakkattil Gopinath

Joseph Ma Djugash

Allison V L Thackston

Nagisa Adachi

Aspects of the disclosure provide for automatically generating labels for sensor data. For instance, first sensor data, for a vehicle may be identified. This first sensor data may have been captured by a first sensor of the vehicle at a first location during a first point in time and may be associated with a first label for an object. Second sensor data for the vehicle may be identified. The second sensor data may have been captured by a second sensor of the vehicle at a second location at a second point in time outside of the first point in time. The second location is different from the first location. A determination may be made as to whether the object is a static object. Based on the determination that the object is a static object, the first label may be used to automatically generate a second label for the second sensor data.

Automatic labeling of objects in sensor data

Systems and methods for learning and managing robot user interfaces are disclosed herein. One embodiment generates, based on input data including information about past interactions of a particular user with a robot and with existing HMIs of the robot, a latent space using one or more encoder neural networks, wherein the latent space is a reduced-dimensionality representation of learned behavior and characteristics of the particular user, and uses the latent space as input to train a decoder neural network associated with (1) a new HMI distinct from the existing HMIs or (2) a particular HMI among the existing HMIs to alter operation of the particular HMI. The trained first decoder neural network is deployed in the robot to control, at least in part, operation of the new HMI or the particular HMI in accordance with the learned behavior and characteristics of the particular user.

Systems and methods for learning and managing robot user interfaces

A vision-assist device may include at least one image sensor for generating image data corresponding to an environment, a user input device for receiving user input regarding one or more physical characteristics of a user, and a processor. The processor may be programmed to receive the image data from the at least one image sensor, receive the user input from the user input device, and adjust an alignment of the at least one image sensor based on the received image data and the user input. Methods for aligning an image sensor are also provided.

Aligning vision-assist device cameras based on physical characteristics of a user

Aspects of the disclosure provide for automatically generating labels for sensor data. For instance, first sensor data for a vehicle may be identified. This first sensor data may have been captured by a first sensor of the vehicle at a first location during a first point in time and may be associated with a first label for an object. Second sensor data for the vehicle may be identified. The second sensor data may have been captured by a second sensor of the vehicle at a second location at a second point in time outside of the first point in time. The second location is different from the first location. A determination may be made as to whether the object is a static object. Based on the determination that the object is a static object, the first label may be used to automatically generate a second label for the second sensor data.

A robot system includes a holding compartment configured to transport one or more items and an electronic control unit having a processor and a non-transitory computer readable memory including a machine-readable instruction set. The robot system further includes a camera for capturing image data of an environment of the holding compartment and a robotic arm each communicatively coupled to the electronic control unit. The machine-readable instruction set causes the processor to receive image data of the environment of the holding compartment from the camera, determine a set of handling instructions for an item collected by the robotic arm, determine a location within the holding compartment for placing the item collected by the robotic arm based on the image data of the environment of the holding compartment and the set of handling instructions, and manipulate the robotic arm to place the item within the holding compartment at the determined location.

User assisting robot for shopping applications

A wearable computing device includes a first side portion and a second side portion that partially extend across a shoulder and rest on a front of a user. The device also includes a neck portion connected to the first side portion and the second side portion. The neck portion includes an outer edge and an inner edge. The neck portion is curved from the first side portion to the second side portion to extend around a portion of a circumference of the neck of the user. The neck portion is also curved from the outer edge to the inner edge to follow a curvature of a spine of the user. The wearable computing device also includes an input device and a mobile processor designed to determine output data based on input data. The wearable computing device also includes an output device designed to output the output data.

Wearable computing device having a curved back to reduce pressure on vertebrae

A device for providing guidance or feedback to a user. The device includes a camera configured to detect image data indicating a user performance of an activity. The system includes a guidance unit connected to the camera. The guidance unit is configured to identify the activity based on image processing of the image data or an identification of the activity from the user. The guidance unit is also configured to determine a criteria associated with the activity. The guidance unit is also configured to determine a user performance of the activity based on the image data. The guidance unit is also configured to determine feedback based on a comparison of the criteria and the user performance of the activity, the feedback indicating an improvement or suggestion for the user. The system also includes an output unit connected to the guidance unit, the output unit configured to output the feedback.

System and method for providing guidance or feedback to a user

Systems and methods of receiving and detecting dimensional aspects of a malleable target object are disclosed. A method of receiving dimensional aspects of a malleable target object includes forming the malleable target object from a malleable material including a magnetic component that generates a magnetic field having particular dimensional aspects that correspond to dimensional aspects of the target object, placing the malleable target object within a sensed area such that a sensor senses the particular dimensional aspects of the magnetic field and generates sensed data corresponding to the particular dimensional aspects of the magnetic field, and receiving, from a computing device, the three dimensional computer rendering of the malleable target object. The computing device generates the three dimensional computer rendering from the sensed data and outputs the three dimensional computer rendering such that the three dimensional computer rendering corresponds to the dimensional aspects of the malleable target object.

Systems and methods for receiving and detecting dimensional aspects of a malleable target object

Methods, systems, and apparatus for controlling air filtration, circulation, climate control and/or other adjustable components. The air control system includes an air filtration or circulation unit for filtering or circulating air. The air control system includes a navigation unit for providing at least one of a navigational map information or environment information including a current location of the vehicle. The air control system includes an electronic control unit connected to the air filtration or circulation unit. The electronic control unit is configured to determine an air control event based on at least one of static air quality information or dynamic air quality information. The electronic control unit is configured to determine that the air control event is within a range of the current location and is approaching. The electronic control unit is configured to activate the air filtration or circulation unit before the air control event reaches the vehicle.

Automatic air quality monitoring and improvement systems

A robot includes a camera for detecting image data in an area, a memory for storing an object location of objects, an input device for receiving user input, and a robot controller. The robot controller can determine that an object is a desired object to track based on user input or previously detected image data indicating that the desired object has been previously manipulated. The robot controller can also determine an identifier of the desired object based on at least one of a comparison of the image data to a database of objects and identifiers or an identifier received via the input device. The robot controller can also determine a current location of the desired object based on the image data and update the object location of the desired object to include the current location of the desired object.

Inventory robot

Blind aid device

Vision-assist systems and methods include use of processor(s) and a communicatively coupled feedback device and environmental sensing device upon which generated feedback is based. Machine readable instructions stored on a memory module cause the system to perform the following when executed by the processor(s): record a first metric corresponding to a level of reliance by a user on the feedback device at a first time; record a second metric corresponding to the level of reliance by the user on the feedback device at a second time after the first time; and record an awarded amount of reliance points when the second metric is less than the first metric, indicative of a decrease on a reliance by the user on the vision-assist system. The awarded amount of reliance points corresponds to a percentage by which the second metric decreases compared to the first metric.

Vision-assist systems for orientation and mobility training

A wearable computing device includes an input device, a global positioning system (GPS) sensor, an inertial measurement unit (IMU), a camera, and a memory that is designed to store a database of landmarks and corresponding locations. The device includes a mobile processor that is designed to receive first user input indicating that the user is traveling from a starting location to a destination location and second user input indicating that new landmarks are present. The processor is designed to determine locations of the new landmarks based on the current location, the inertial measurement data, and/or the detected image data. The processor is designed to update the database to include the new landmarks and corresponding locations. The processor is designed to update the database to include an ordered list of landmarks including the new landmarks in order of their appearance from the starting location to the destination location.

Wearable system for providing walking directions

A system and method for assisting a visually impaired user with navigating an environment. The system and method include determining a first navigational path to a first target destination and a second navigational path to a second target destination and providing the user with directional information in the form a first audio output associated with the first target destination and a second audio output associated with the second target destination. The system and method vary an audio characteristic of the first audio output in relation to the current orientation of the user with respect to the first navigational path and vary an audio characteristic of the second audio output in relation to the current orientation of the user with respect to the second navigational path. The system and method control the audio output unit to simultaneously output the first audio output and the second audio output.

Vision-assist systems and methods for assisting visually impaired users with navigating an environment using simultaneous audio outputs

A system for providing information to a user includes a haptic display including a plurality of sensors positioned thereon, a map positioned on the haptic display and including points of interest where each point of interest is positioned adjacent to or directly on at least one of the plurality of sensors, and a memory for storing a plurality of breadcrumb files where each of the plurality of breadcrumb files corresponds to at least one of the points of interest and contains information related to at least one of an environment, an object, an obstacle, a path or a location. The system includes a processor for determining which point of interest has been selected based on contact with at least one of the points of interest and an output device configured to provide the information contained in the breadcrumb file corresponding to the selected point of interest to the user.

Growing community of inventors

Santa Clara, CA, United States of America

Tiffany L Chen