Amazon Technologies, Inc.

Sony Group Corporation

Kiva Systems, Inc.

Sony Corporation of America

University of Michigan

Sony Interactive Entertainment Inc.

Peter R Wurman

Michael Cordell Mountz

Michael T Barbehenn

Andrew E Hoffman

Raffaello D'Andrea

Matthew David Verminski

Peter K Mansfield

Joseph W Durham

John Joseph Enright

Raffaello D'andrea

Jeremiah David Brazeau

Michael Harrison Decker

Eryk Brian Nice

Daniel T Brunner

Dennis Polic

James MacGlashan

Andrew Stubbs

Timothy Aaron Bragg

Piyush Khandelwal

Zhouzhou Peng

Jonathan Harris Schechter

James Raymond Allard

Ned Lecky

John W Cullen

Samuel Barrett

Pardeep Singh Farwaha

Ryan Alexander Holt

Alex Glazkov

Jared Brian Holcomb

Oleksandr Glazkov

Joseph Romano

Patrick R Dingle

Fabrizio Santini

Steven Augustine Wilson

Ishan Mukherjee

William James Watt

Andrew Brenden Tinka

Satinder Singh Baveja

Aron Klair Insinga

Eric Leventhal Arthen

Sean Hugh Breheny

Gavin Cotter

Peter Stone

Victoria Eddy

Dion Whitehead

Leon Barrett

Kaushik Subramanian

Varun Kompella

Caroline F Mallary

David C Koehler

Joshua B Chaitin-Pollak

Matthew E Kimmel

Rory Douglas

Kenta Kawamoto

Robert Raymond Ambrogi

James Christopher Curlander

Jules Cook Graybill

Andriy Palamarchuk

Justin V Beltran

Mark Ring

David Levy

Michael Wellman

Ibolya Horvath

Peter Martin Hess

Roberto Capobianco

Rabih Abdul Ahad

Bandaly Azzam

Michael Bowling

Leilani Gilpin

Thomas Walsh

Takuma Seno

Goker Erdogan

Florian Fuchs

Jill Kathleen Sestini

Houmehr Aghabozorgi

Reinforcement learning methods can use actor-critic networks where (1) additional laboratory-only state information is used to train a policy that much act without this additional laboratory-only information in a production setting; and (2) complex resource-demanding policies are distilled into a less-demanding policy that can be more easily run at production with limited computational resources. The production actor network can be optimized using a frozen version of a large critic network, previously trained with a large actor network. Aspects of these methods can leverage actor-critic methods in which the critic network models the action value function, as opposed to the state value function.

Training actor-critic algorithms in laboratory settings

A method and system for teaching an artificial intelligent agent where the agent can be placed in a state that it would like it to learn how to achieve. By giving the agent several examples, it can learn to identify what is important about these example states. Once the agent has the ability to recognize a goal configuration, it can use that information to then learn how to achieve the goal states on its own. An agent may be provided with positive and negative examples to demonstrate a goal configuration. Once the agent has learned certain goal configurations, the agent can learn policies and skills that achieve the learned goal configuration. The agent may create a collection of these policies and skills from which to select based on a particular command or state.

Method and system for an intelligent artificial agent

A task prioritized experience replay (TaPER) algorithm enables simultaneous learning of multiple RL tasks off policy. The algorithm can prioritize samples that were part of fixed length episodes that led to the achievement of tasks. This enables the agent to quickly learn task policies by bootstrapping over its early successes. Finally, TaPER can improve performance on all tasks simultaneously, which is a desirable characteristic for multi-task RL. Unlike conventional ER algorithms that are applied to single RL task learning settings or that require rewards to be binary or abundant, or are provided as a parameterized specification of goals, TaPER poses no such restrictions and supports arbitrary reward and task specifications.

Task prioritized experience replay algorithm for reinforcement learning

A real-time temporal convolution network (RT-TCN) algorithm reuses the output of prior convolution operations in all layers of the network to minimize the computational requirements and memory footprint of a TCN during real-time evaluation. Further, a TCN trained via the fixed-window view, where the TCN is trained using fixed time splices of the input time series, can be executed in real-time continually using RT-TCN.

Computing temporal convolution networks in real time

Systems and methods are used to adapt the coefficients of a proportional-integral-derivative (PID) controller through reinforcement learning. The approach for adapting PID coefficients can include an outer loop of reinforcement learning where the PID coefficients are tuned to changes in the environment and an inner loop of PID control for quickly reacting to changing errors. The outer loop can learn and adapt as the environment changes and be configured to only run at a predetermined frequency, after a given number of steps. The outer loop can use summary statistics about the error terms and any other information sensed about the environment to calculate an observation. This observation can be used to evaluate the next action, for example, by feeding it into a neural network representing the policy. The resulting action is the coefficients of the PID controller and the tunable parameters of things such as the filters.

Methods and systems to adapt PID coefficients through reinforcement learning

Dynamic driving aides, such as driving lines, turn indicators, braking indicators and acceleration indicators, for example, can be provided for players participating in a racing game. Typically, driving lines are provided for each class of cars. However, even within a class of cars, each car differs enough that the ideal driving lines and breaking points can vary. Therefore, with an agent trained via reinforcement learning, an ideal lines and other driving aides can be established for every individual car. These guides can even be varied to account for variations in the weather or other track conditions.

Artificial intelligence for driving lines in racing games

A user interface (UI), for analyzing model training runs, tracking and visualizing various aspects of machine learning experiments, can be used when training an artificial intelligent agent in, for example, a racing game environment. The UI can be web-based and can allow researchers to easily see the status of their experiments. The UI can include an experiment synchronized event viewer that can synchronizes visualizations, videos, and timeline/metrics graphs in the experiment. This viewer allows researchers to see how experiments unfold in great detail. The UI can further include experiment event annotations that can generate event annotations. These annotations can be displayed via the synchronized event viewer. The UI can be used to consider consolidated results across experiments and can further consider videos. For example, the UI can provide a reusable dashboard that can capture and compare metrics across multiple experiments.

User interface for operating artificial intelligence experiments

Systems and methods use deep, convolutional neural networks over exponentially long history windows to learn alphabets for context tree weighting (CTW) for prediction. Known issues of depth and breadth in conventional context tree weighting predictions are addressed by the systems and methods. To deal with depth, the history can be broken into time windows, permitting the ability to look exponentially far back while having less information the further one looks back. To deal with breadth, a deep neural network classifier can be used to learn to map arbitrary length histories to a small output symbol alphabet. The sequence of symbols produced by such a classifier over the history windows would then become the input sequence to CTW.

Method and system for predicting discrete sequences using deep context tree weighting

An artificial intelligent agent can act as a player in a video game, such as a racing video game. The game can be completely external to the agent and can run in real time. In this way, the training system is much more like a real world system. The consoles on which the game runs for training the agent are provided in a cloud computing environment. The agents and the trainers can run on other computing devices in the cloud, where the system can choose the trainers and agent compute based on proximity to console, for example. Users can choose the game they want to run and submit code which can be built and deployed to the cloud system. A resource management service can monitor game console resources between human users and research usage and identify experiments for suspension to ensure enough game consoles for human users.

Methods for controlling use of computing resources, such as virtual game consoles

An inventory system can include a workspace in which is defined different areas and different stations at which tasks can be performed. A receipt receptacle received from a shipping vehicle can be moved to a rack without separating items from the receipt vehicle. A mobile drive unit can move the rack to an area on the workspace depending on information known about the items. Items from the receipt receptacle can be removed from the receipt receptacle and placed in inventory holders for movement or storage or in other receptacles for performance of other tasks such as order processing.

Inventory system with efficient operator handling of inventory items

A method for transporting inventory items includes moving a mobile drive unit to a first point within a workspace. The first point is a location of an inventory holder. The method further includes docking the mobile drive unit with the inventory holder and moving the mobile drive unit and the inventory holder to a second point within the workspace. The second point is associated with conveyance equipment. The method further includes moving the inventory holder to a third point within the workspace using the conveyance equipment.

Method and system for transporting inventory items

Techniques for controlling mobile drive units (e.g., robots) in proximity to humans in a physical area are described. A management device may send an activity message to the mobile drive units instructing the mobile drive units to perform a set of activities. If a human is detected in the physical area, the management device or another mobile drive unit may send an activity command instructing a mobile drive unit to stop, reduce the speed at which it is traveling, to change its path of travel, or to continue performing its set of activities. If the mobile drive units do not receive the activity command, the mobile drive units may stop performing the set of activities. After the management device determines that the human has left the physical area, the management device may resume sending the activity message.

Safety compliance for mobile drive units

A method includes obtaining, by a management module, an inventory unpacking sequence for a remote facility. The inventory unpacking sequence indicates a sequence for unpacking a shipment at the remote facility. An order associated with the remote facility is received that includes inventory items. The management module determines an order packing arrangement for the inventory items based on the inventory unpacking sequence. Mobile drive units are instructed to transport one or more inventory holders storing the inventory items to an inventory station, where the inventory items are packed into a shipment according to the order packing arrangement. The shipment is arranged to be unpacked at the remote facility according to the inventory unpacking sequence.

System and method for arranging an order

Robots or other machines may be used for retrieving errant objects from the floor of an automated warehouse. A system can include one or more reporting methods to alert a central control to the existence and location of an object on the warehouse floor. The central control can establish a safety zone around the object to avoid contact with normal warehouse traffic (e.g., standard warehouse robots). The system can route a cleanup robot to the location to retrieve the object. The system can include a cleanup pod comprising a convertible shelving unit with a robotic arm. The cleanup pod can have a similar form factor as shelving units used for storing inventory in the warehouse, thereby enabling standard warehouse robots to lift and transport the cleanup pod to retrieve an object.

Systems and methods for removing debris from warehouse floors

Growing community of inventors

Acton, MA, United States of America

Peter R Wurman