X Development LLC

Matthieu Guilbert

Craig Latimer

Laura Stoia

Emily B Cooper

David Deephanphongs

Thomas Buschmann

Yuanzheng Gong

Methods, apparatus, systems, and computer-readable media for assigning a real-time clock domain timestamp to sensor frames from a sensor component that operates in a non-real-time time-domain. In some implementations, a real-time component receives capture instances that each indicate capturing of a corresponding sensor data frame by the sensor component. In response to a capture output instance, the real-time component or an additional real-time component assigns a real-time timestamp to the capture output instance, where the real-time timestamp is based on the real-time clock domain. Separately, a non-real-time component receives the corresponding sensor data frames captured by the sensor component, along with corresponding metadata. For each sensor data frame, it is determined whether there is a real-time timestamp that corresponds to the data frame and, if so, the real-time timestamp is assigned to the sensor data frame.

Determining real-time timestamps for robotic sensor data

An example method includes determining a plurality of frequency ranges corresponding to a plurality of types of errors, where the plurality of frequency ranges are associated with sounds occurring during operation of a robotic device. The method also includes detecting, based on sensor data from at least one audio sensor of the robotic device, a sound during a given operation of the robotic device. The method also includes determining that a frequency of the detected sound is within a particular frequency range of the plurality of frequency ranges. Based on the frequency being within the particular frequency range, the method also includes determining a type of error of the plurality of types of errors corresponding to the particular frequency range. The method also includes providing an output signal indicating an error of the determined type.

Methods and systems for determining errors based on detected sounds during operation of a robotic device

An example method includes determining an expected sound profile corresponding to a given task for a robotic device. The method further includes detecting a sound profile during execution of the given task by the robotic device. The method also includes determining one or more differences in amplitude for at least one frequency range between the detected sound profile and the expected sound profile corresponding to the given task for the robotic device. In response to determining the one or more differences in amplitude for the at least one frequency range between the detected sound profile and the expected sound profile, the method additionally includes identifying at least one component of the robotic device associated with the detected sound profile during execution of the given task. The method further includes adjusting control data for the at least one component of the robotic device.

Methods and systems for adjusting operation of a robotic device based on detected sounds

An example method includes receiving, from a sensor, an image of an environment, where the environment includes a robotic device. The method also includes determining, based on the received image of the environment, a pose of the robotic device relative to the sensor. The method further includes determining, based on the pose of the robotic device relative to the sensor, respective positions of a plurality of trajectory points for the robotic device relative to the sensor. The method additionally includes providing for display of the image of the environment and an overlaid plurality of virtual trajectory points corresponding to the plurality of trajectory points, where the plurality of virtual trajectory points are positioned in the image based on the determined respective positions of the plurality of trajectory points for the robotic device relative to the sensor.

Growing community of inventors

Palo Alto, CA, United States of America

Matthieu Guilbert