Sanctuary Cognitive Systems Corporation

Kindred Systems Inc.

D-wave Systems Inc.

Suzanne Gildert

Geordie Rose

Olivia Norton

William G Macready

James Sterling Bergstra

George Samuel Rose

Dominic Christoph Walliman

Miles F H Steininger

Mani Ranjbar

Firas Hamze

Dmytro Korenkevych

Graham Taylor

Thomas Mahon

Rodrigo Furlan

Steven Varghese Jacob

Graham William Taylor

James Bergstra

Robot control systems, methods, control modules and computer program products that leverage one or more large language model(s) (LLMs) in order to achieve at least some degree of autonomy are described. Robot control parameters and/or instructions may advantageously be specified in natural language (NL) and communicated with the LLM via a recursive sequence of NL prompts or queries. Corresponding NL responses from the LLM may then be converted into robot control parameters and/or instructions. In this way, an LLM may be leveraged by the robot control system to enhance the autonomy of various operations and/or functions, including without limitation task planning, motion planning, human interaction, and/or reasoning about the environment.

Robot systems, methods, control modules, and computer program products that leverage large language models

Robot control systems, methods, control modules and computer program products that leverage one or more large language model(s) (LLMs) in order to achieve at least some degree of autonomy are described. Robot control parameters and/or instructions may advantageously be specified in natural language (NL) and communicated with the LLM via an NL prompt or query. The LLM module provides a task plan in NL, which can be evaluated for at least one fault or error. If at least one fault or error is identified, the LLM module can be queried to provide a resolution.

Robots, methods, and computer program products for training and operating (semi-) autonomous robots to complete multiple different work objectives are described. A robot accesses a library of reusable work primitives from a catalog of libraries of reusable work primitives, each reusable work primitive corresponding to a respective basic sub-task or sub-action that the robot is operative to autonomously perform. A work objective is analyzed to determine a sequence (i.e., a combination and/or permutation) of reusable work primitives that, when executed by the robot, will complete the work objective. The robot executes the sequence of reusable work primitives to complete the work objective. A robot can be deployed with an appropriate stored library (or access to an appropriate library) of reusable work primitives, based on what the robot is expected to do, or what service category or role the robot will operate in.

Multi-purpose robots and computer program products, and methods for operating the same

Systems, devices, and methods for training and operating (semi-)autonomous robots to complete multiple different work objectives are described. A robot control system stores a library of reusable work primitives each corresponding to a respective basic sub-task or sub-action that the robot is operative to autonomously perform. A work objective is analyzed to determine a sequence (i.e., a combination and/or permutation) of reusable work primitives that, when executed by the robot, will complete the work objective. The robot executes the sequence of reusable work primitives to complete the work objective. The reusable work primitives may include one or more reusable grasp primitives that enable(s) a robot's end effector to grasp objects. Simulated instances of real physical robots may be trained in simulated environments to develop control instructions that, once uploaded to the real physical robots, enable such real physical robots to autonomously perform reusable work primitives.

Systems, devices, and methods for grasping by multi-purpose robots

The present disclosure describes robots, tele-operation systems, methods, and computer program products where a robot is selectively operable in a plurality of control modes. Based on identification of a fault condition (when the robot fails to act in a suitable or sufficient manner), a control mode of the robot can be changed to provide a human operator with more explicit control over the robot. In this way, the fault condition can be resolved by human operator input, and the control modes, AI, or control paradigm for the robot can be trained to perform better in the future.

Robots, teleoperation systems, and methods of operating the same

The present disclosure describes systems, robots, and methods for organizing and selecting classifiers of a library of classifiers. The classifiers of the library of classifiers can be organized in a relational model, such as a hierarchy or probability model. Instead of storing, activating, or executing the entire library of classifiers at once by a robot system, computational resource demand is reduced by executing subset of classifiers to determine context, and the determined context is used as a basis for selection of another subset of classifiers. This process can be repeated, to iteratively refine context and select more specific subsets of classifiers. A selected subset of classifiers can eventually be specific to a task to be performed by the robot system, such that the robot system can take action based on output from executing such specific classifiers.

Systems, robots, and methods for selecting classifiers based on context

The present disclosure relates to utilizing idle processing resources of a robot to reduce future burden on such processing resources. In particular, idle processing resources are utilized to identify future scenarios, and generate reactions to such future scenarios. The generated reactions are stored, and quickly retrieved as needed if corresponding identified future scenarios occur.

Robots and methods for utilizing idle processing resources

Systems, methods, and computer program products for automating tasks are described. A multi-step framework enables a gradient towards task automation. An agent performs a task while sensors collect data. The data are used to generate a script that characterizes the discrete actions executed by the agent in the performance of the task. The script is used by a robot teleoperation system to control a robot to perform the task. The robot teleoperation system maps the script into an ordered set of action commands that the robot is operative to auto-complete to enable the robot to semi-autonomously perform the task. The ordered set of action commands is converted into an automation program that may be accessed by an autonomous robot and executed to cause the autonomous robot to autonomously perform the task. In training, simulated instances of the robot may perform simulated instances of the task in simulated environments.

Systems, methods, and computer program products for automating tasks

Robot control systems, methods, control modules and computer program products that leverage one or more large language model(s) (LLMs) in order to achieve at least some degree of autonomy are described. Robot control parameters, environment details, and/or instructions may advantageously be specified in natural language (NL) and communicated with the LLM via an NL prompt or query. The NL query may include a request for one or more work objectives from the LLM, such as 'What can I do here?', thereby establishing a form of agency by which the robot system may identify activities to perform without operator intervention. The LLM may also be queried to convert each work objective into a task plan providing a sequence of steps that the robot system may execute to complete the work objective. Optionally, the robot system may communicate with an operator to determine whether or not to execute a task plan.

Robot control systems, methods, control modules, and computer program products that leverage one or more large language model(s) (LLMs) in order to achieve at least some degree of autonomy are described. Robot control parameters, environment details, and/or instruction sets may advantageously be specified in natural language (NL) and communicated with the LLM via an NL prompt or query. An NL response from the LLM may then be converted into a task plan. A task plan that successfully completes a first instance of a work objective may be parameterized and re-used to complete a second instance of the work objective. Parameterization of a task plan may include replacing one or more nouns/objects in the NL task plan with variables, while optionally preserving one or more verbs/actions in the NL task plan.

Robots, robot systems, and methods for operating the same based on environment models including haptic data are described. An environment model which includes representations of objects in an environment is accessed, and a robot system is controlled based on the environment model. The environment model incudes haptic data, which provides more effective control of the robot. The environment model is populated based on visual profiles, haptic profiles, and/or other data profiles for objects or features retrieved from respective databases. Identification of objects or features can be based on cross-referencing between visual and haptic profiles, to populate the environment model with data not directly collected by a robot which is populating the model, or data not directly collected from the actual objects or features in the environment.

Growing community of inventors

Burnaby, Canada

Suzanne Gildert