Google Inc.

Microsoft Technology Licensing, LLC

Cisco Technology, Inc.

X Development LLC

Cisco Systems Inc.

Simple Rules, Inc.

Johnny Hsienchow Lee

Fadi Ramzi Jabbour

Craig C Peeper

Tommer Leyvand

David C Lee

Eric Teller

Joel Hesch

Labhesh Patel

William Graham Patrick

Eric Peeters

Szymon Piotr Stachniak

James Fung

Ryan M Geiss

Jamie Daniel Joseph Shotton

Mark James Finocchio

Robert Matthew Craig

Momin Mahmoud Al-Ghosien

Oliver M Williams

Matt Bronder

Stefan Welker

Shao Liu

Eric H Lee

Stephen G Latta

Kevin Geisner

Gregory Nelson Snook

Shmuel Shaffer

Shantanu Sarkar

Relja Markovic

Masaaki Fukumoto

David William Baumert

Katsushi Ikeuchi

Adam Jethro Langridge

Stephen David Bader

Adam Edwin Behringer

Hsiao-Lan Wang

Jordan Lee Kravitz

Ryan Michael Hickman

Eric N Lee

Sergey Brin

John D Perreault

Daniel Cory

Robert Brett Rose

Matthew Kuhlke

Rahul Garg

James Watts

Ashish A Shah

Daniel M Aminzade

Eric Zhang

Shiqi Chen

Simon Lynen

Dan P Piponi

Bradley Richard DeGrazia

Priya Nayak

Renzo De Nardi

Jerry Carollo

Aruna Balakrishnan

David Cheng-Wei Lee

Jason J Ganetsky

Mike Sorvillo

Hubert Ming-Hwa Chao

Alan Beltran

Chun To Tse

Relia Markovic

Adam Jethro Lanygridge

Aristotle H Cruz

Utilization of user interface inputs, from remote client devices, in controlling robot(s) in an environment. Implementations relate to generating training instances based on object manipulation parameters, defined by instances of user interface input(s), and training machine learning model(s) to predict the object manipulation parameter(s). Those implementations can subsequently utilize the trained machine learning model(s) to reduce a quantity of instances that input(s) from remote client device(s) are solicited in performing a given set of robotic manipulations and/or to reduce the extent of input(s) from remote client device(s) in performing a given set of robotic operations. Implementations are additionally or alternatively related to mitigating idle time of robot(s) through the utilization of vision data that captures object(s), to be manipulated by a robot, prior to the object(s) being transported to a robot workspace within which the robot can reach and manipulate the object.

Efficient robot control based on inputs from remote client devices

An example computer-implemented method includes receiving, from one or more vision components in an environment, vision data that captures features of the environment, including object features of an object that is located in the environment, and prior to a robot manipulating the object: (i) determining based on the vision data, at least one first adjustment to a programmed trajectory of movement of the robot operating in the environment to perform a task of transporting the object, and (ii) determining based on the object features of the object, at least one second adjustment to the programmed trajectory of movement of the robot operating in the environment to perform the task, and causing the robot to perform the task, in accordance with the at least one first adjustment and the at least one second adjustment to the programmed trajectory of movement of the robot.

Optimization of motion paths of a robot using vision data

Noise reduction in a robot system includes the use of a gesture library that pairs noise profiles with gestures that can be performed by the robot. A gesture to be performed by the robot is obtained, and the robot performs the gesture. The robot's performance of the gesture creates noise, and when a user speaks to the robot while the robot performs a gesture, incoming audio includes both user audio and robot noise. A noise profile associated with the gesture is retrieved from the gesture library and is applied to remove the robot noise from the incoming audio.

Noise reduction in robot human communication

Noise reduction in a robot system includes the use of a gesture library that pairs noise profiles with gestures performed by the robot. A gesture to be performed by the robot is obtained, and the robot performs the gesture. The robot's performance of the gesture creates noise, and when a user speaks to the robot while the robot performs a gesture, incoming audio includes both user audio and robot noise. A noise profile associated with the gesture is retrieved from the gesture library and is applied to remove the robot noise from the incoming audio.

A method for controller tracking with multiple degrees of freedom includes generating depth data at an electronic device based on a local environment proximate the electronic device. A set of positional data is generated for at least one spatial feature associated with a controller based on a pose of the electronic device, as determined using the depth data, relative to the at least one spatial feature associated with the controller. A set of rotational data is received that represents three degrees-of-freedom (3DoF) orientation of the controller within the local environment, and a six degrees-of-freedom (6DoF) position of the controller within the local environment is tracked based on the set of positional data and the set of rotational data.

Controller tracking for multiple degrees of freedom

A machine intelligent communication and control system is able to dynamically predict and adapt information presented to various users. The system provides a personalized experience with its ability to identify users, become contextually aware of the user's location and environment, identify objects and actions, and present customized information tailored for the user and the current environment. The system is further able to control connected items within the environment based upon various user preference considerations.

Machine intelligent predictive communications and control system

Machine intelligent predictive communication and control system

An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A background included in the grid of voxels may then be discarded to isolate one or more voxels associated with a foreground object such as a human target and the isolated voxels associated with the foreground object may be processed.

Systems and methods for removing a background of an image

A method and apparatus for a networked desktop environment is provided. Executing a networked desktop environment may include identifying a networked desktop state of the networked desktop environment, which may include a networked desktop workspace and an information element. The networked desktop state may include information indicating a multi-dimensional position of a representation of the information element in the networked desktop workspace. Executing the networked desktop environment may include generating instructions for use in rendering the networked desktop workspace, which may include information indicative of the networked desktop state, and transmitting the instructions to an interface device. Information indicating an interaction with the networked desktop environment may be received and the networked desktop state may be updated based on the interaction information.

Networked desktop environment

An electronic device () includes a depth sensor (), a first imaging camera (), and a controller (). The depth sensor () includes a modulated light projector () to project a modulated light pattern (). The first imaging camera () is to capture at least a reflection of the modulated light pattern (). The controller () is to selectively modify () at least one of a frequency, an intensity, and a duration of projections of the modulated light pattern by the modulated light projector responsive to at least one trigger event (). The trigger event can include, for example, a change () in ambient light incident on the electronic device, detection () of motion of the electronic device, or a determination () that the electronic device has encountered a previously-unencountered environment.

Context-based depth sensor control

Methods and systems for determining features of interest for following within various frames of data received from multiple sensors of a device are disclosed. An example method may include receiving data from a plurality of sensors of a device. The method may also include determining, based on the data, motion data that is indicative of a movement of the device in an environment. The method may also include as the device moves in the environment, receiving image data from a camera of the device. The method may additionally include selecting, based at least in part on the motion data, features in the image data for feature-following. The method may further include estimating one or more of a position of the device or a velocity of the device in the environment as supported by the data from the plurality of sensors and feature-following of the selected features in the images.

Odometry feature matching

A user portable device includes a device chassis comprising at least one opening at a surface of the device chassis and a sensor assembly aligned with the at least one opening. The sensor assembly includes a mounting structure and a plurality of sensors mounted to the mounting structure. The sensors include at least two sensors utilized by the user portable device based on a specified geometric configuration between the at least two sensors. The user portable device further includes a mounting fastener that mounts the sensor assembly to the device chassis so as to isolate the sensor assembly from deformation of the surface of the device chassis along one or more axes during user handling, and thus aid in preventing alteration of a baseline geometric configuration of one or more sensors of the sensor assembly due to the chassis deformation.

User portable device having floating sensor assembly to maintain fixed geometric configuration of sensors

Systems, methods and computer readable media are disclosed for controlling perspective of a camera-controlled computer. A capture device captures user gestures and sends corresponding data to a recognizer engine. The recognizer engine analyzes the data with a plurality of filters, each filter corresponding to a gesture. Based on the output of those filters, a perspective control is determined, and a display device displays a new perspective corresponding to the perspective control.

Method to control perspective for a camera-controlled computer

Growing community of inventors

San Gabriel, CA, United States of America

Johnny Hsienchow Lee