Tata Consultancy Services Limited

Chayan Sarkar

Pradip Pramanick

Balamuralidhar Purushothaman

Arpan Pal

Brojeshwar Bhowmick

Hrishav Bakul Barua

Marichi Agarwal

Snehasis Banerjee

Abhijan Bhattacharyya

Ashis Sau

Ruddra Dev Roychoudhury

Sayan Paul

Arijit Mukherjee

Sounak Dey

Ajay Kattepur

Himadri Sekhar Paul

Ruchira Singh

Indrajit Bhattacharya

Achanna Anil Kumar

Rajesh Kumar Sinha

Swagata Biswas

Subhrojyoti Roy Chaudhuri

Puneet Patwari

Soumyadip Maity

Dipanjan Das

Aniruddha Singhal

Arindam Pal

Amar Banerjee

Kritika Anand

Tapas Nayak

Ankush Ojha

Supratim Ghosh

Ruddra Roy Chowdhury

Avik Mitra

This disclosure relates generally to system and methods for dynamic scheduling and rescheduling using heterogeneous multi-agent fleet. The embodiments of present disclosure herein address unresolved problem of task allocation using a single solution which is not sufficient for handling various scenarios of multi-agent task allocation problems. For instance, the task allocation becomes more challenging in a scenario where each task has a deadline associated with it and execution time of the tasks cannot be pre-computed as they are dependent on previous tasks. The method of present disclosure provides a scalable solution for dynamic scheduling and rescheduling that handles tasks with multiple pickup and drop locations and dynamic execution time using the agents with heterogeneous speed in a more efficient manner, reducing indirect operating costs and increasing revenue potential while minimizing additional penalty due to run time delays that an agent may encounter.

Systems and methods for dynamic task scheduling and rescheduling using heterogeneous multi-agent fleet

The disclosure generally relates to methods and systems for complex natural language task understanding for embodied robots or agents. Conventional works on relation extraction generally find relevant triplets in a natural language phrase, but neither ground the task nor ground the arguments. The present disclosure implements a Grounded Argument and Task Extraction (GATE) technique that extracts a set of tasks and relevant arguments from the complex natural language instruction. The GATE uses an encoder-decoder neural network with nested decoding technique. The extracted tasks are mapped (grounded) to the known skill set of the robot and arguments are mapped (grounded) to objects within the environment, classifies the tokens as many times as possible which existing sequence labeling cannot do. The encoder-decoder neural network of the present disclosure extracts grounded task-argument pairs from a natural language instruction in a generative mechanism, and grounds the arguments based on object detector vocabulary.

Methods and systems for complex natural language task understanding for embodied agents

Accuracy of transcript is of foremost importance in Automatic Speech Recognition (ASR). State of the art system mostly rely on spelling correction based contextual improvement in ASR, which is generally a static vocabulary based biasing approach. Embodiments of the present disclosure provide a method and system for visual context aware ASR. The method provides biasing using shallow fusion biasing approach with a modified beam search decoding technique, which introduces a non-greedy pruning strategy to allow biasing at the sub-word level. The biasing algorithm brings in the visual context of the robot to the speech recognizer based on a dynamic biasing vocabulary, improving the transcription accuracy. The dynamic biasing vocabulary, comprising objects in a current environment accompanied by their self and relational attributes, is generated using a bias prediction network that explicitly adds label to objects, which are detected and captioned via a state of the art dense image captioning network.

Method and system for visual context aware automatic speech recognition

This disclosure addresses the unresolved problems of tackling object disambiguation task for an embodied agent. The embodiments of present disclosure provide a method and system for disambiguation of referred objects for embodied agents. With a phrase-to-graph network disclosed in the system of the present disclosure, any natural language object description indicating the object disambiguation task can be converted into a semantic graph representation. This not only provides a formal representation of the referred object and object instances but also helps to find an ambiguity in disambiguating the referred object using a real-time multi-view aggregation algorithm. The real-time multi-view aggregation algorithm processes multiple observations from an environment and finds the unique instances of the referred object. The method of the present disclosure demonstrates significant improvement in qualifying ambiguity detection with accurate, context-specific information so that it is sufficient for a user to come up with a reply towards disambiguation.

Methods and systems for disambiguation of referred objects for embodied agents

Online 3-dimensional bin packing problem (O3D-BPP) is getting renewed prominence due to the industrial automation brought by Industry 4.0. However, due to limited attention in the past and its challenging nature, a good approximate technique is in scarcity as compared to 1D or 2D problems. Present disclosure provides system and method that considers real-time O3D-BPP of cuboidal boxes with partial information (look-ahead) in an automated robotic sorting center. System presents two rolling-horizon mixed-integer linear programming (MILP) cum-heuristic based algorithms wherein a framework is provided that adapts and improves performance of BP heuristics by utilizing information in an online setting with look-ahead.

Automated placement of objects in bins using look ahead information by virtual sorting and packing

The embodiments of present disclosure herein address unresolved problem of cognitive navigation strategies for a telepresence robotic system. This includes giving instruction remotely over network to go to a point in an indoor space, to go an area, to go to an object. Also, human robot interaction to give and understand interaction is not integrated in a common telepresence framework. The embodiments herein provide a telepresence robotic system empowered with a smart navigation which is based on in situ intelligent visual semantic mapping of the live scene captured by a robot. It further presents an edge-centric software architecture of a teledrive comprising a speech recognition based HRI, a navigation module and a real-time WebRTC based communication framework that holds the entire telepresence robotic system together. Additionally, the disclosure provides a robot independent API calls via device driver ROS, making the offering hardware independent and capable of running in any robot.

Telepresence robots having cognitive navigation capability

Existing cognitive robotic applications follow a practice of building specific applications for specific use cases. However, the knowledge of the world and the semantics are common for a robot for multiple tasks. In this disclosure, to enable usage of knowledge across multiple scenarios, a method and system for ontology guided indoor scene understanding for cognitive robotic tasks is described where in scenes are processed based on techniques filtered based on querying ontology with relevant objects in perceived scene to generate a semantically rich scene graph. Herein, an initially manually created ontology is updated and refined in online fashion using external knowledge-base, human robot interaction and perceived information. This knowledge helps in semantic navigation, aids in speech, and text based human robot interactions.

System and method for ontology guided indoor scene understanding for cognitive robotic tasks

Any technical error with robotic arms that are used to automatically perform object packing can affect quality and efficiency with which the packing is being carried out, and this in turn affects space utilization when a large quantity of objects are to be accommodated in tight packing spaces. This disclosure relates generally to automated object packing and more specifically to an object packing mechanism in which corrections are made when placement of object is identified as violating one or more regulations. The system packs objects by calculating ICP-BCP pairs for each empty space in a packing space. After packing each object, the system checks whether placement of the object violates one or more regulations, and if any violation is found, then the system determines and executes one or more corrective action to correct placement of the object that violates the regulation.

Method and system for automated object packing

Conventional tele-presence robots have their own limitations with respect to task execution, information processing and management. Embodiments of the present disclosure provide a tele-presence robot (TPR) that communicates with a master device associated with a user via an edge device for task execution wherein control command from the master device is parsed for determining instructions set and task type for execution. Based on this determination, the TPR queries for information across storage devices until a response is obtained enough to execute task. The task upon execution is validated with the master device and user. Knowledge acquired, during querying, task execution and validation of the executed task, is dynamically partitioned by the TPR across storage devices namely, on-board memory of the tele-present robot, an edge device, a cloud and a web interface respectively depending upon the task type, operating environment of the tele-presence robot, and other performance affecting parameters.

Knowledge partitioning for task execution by conversational tele-presence robots in a geographically separated environment

Robotic platform for tele-presence applications has gained paramount importance, such as for remote meetings, group discussions, and the like and has sought much attention. There exist some robotic platforms for such tele-presence applications, these lack efficacy in communication and interaction between remote person and avatar robot deployed in another geographic location thus adding network overhead. Embodiments of the present disclosure for edge centric communication protocol for remotely maneuvering tele-presence robot in geographically distributed environment. More specifically, edge-centric tele-presence robot architecture is provided for maneuvering tele-presence robot in distributed geographical environment, wherein the architecture provides a framework that implements both cloud and edge-centric systems together with a first communication protocol for communication between a master device and an edge device and a second communication protocol different from the first communication protocol for communication between the edge device and the tele-presence robot to improve robustness and efficacy in communication.

Edge centric communication protocol for remotely maneuvering a tele-presence robot in a geographically distributed environment

The disclosure herein relates to methods and systems for enabling human-robot interaction (HRI) to resolve task ambiguity. Conventional techniques that initiates continuous dialogue with the human to ask a suitable question based on the observed scene until resolving the ambiguity are limited. The present disclosure use the concept of Talk-to-Resolve (TTR) which initiates a continuous dialogue with the user based on visual uncertainty analysis and by asking a suitable question that convey the veracity of the problem to the user and seek guidance until all the ambiguities are resolved. The suitable question is formulated based on the scene understanding and the argument spans present in the natural language instruction. The present disclosure asks questions in a natural way that not only ensures that the user can understand the type of confusion, the robot is facing; but also ensures minimal and relevant questioning to resolve the ambiguities.

Methods and systems for enabling human-robot interaction to resolve task ambiguity

This disclosure provides systems and methods for robotic task planning when a complex task instruction is provided in natural language. Conventionally robotic task planning relies on a single task or multiple independent or serialized tasks in the task instruction. Alternatively, constraints on space of linguistic variations, ambiguity and complexity of the language may be imposed. In the present disclosure, firstly dependencies between multiple tasks are identified. The tasks are then ordered such that a dependent task is always scheduled for planning after a task it is dependent upon. Moreover, repeated tasks are masked. Thus, resolving task dependencies and ordering dependencies, a complex instruction with multiple interdependent tasks in natural language facilitates generation of a viable task execution plan. Systems and methods of the present disclosure finds application in human-robot interactions.

Robotic task planning for complex task instructions in natural language

The systems and methods provide for optimizing the scheduling of non-preemptive tasks in the multi-robot environment. The embodiment provides for scheduling, by implementing an Online Minimum Performance Loss Scheduling (OMPLS) technique, initially, tasks with a higher performance loss value and then secondly, tasks that can be scheduled within their deadline and having a low performance loss value amongst the merged tasks; and finally minimizing, a performance loss value of a remaining subset of tasks that cannot be scheduled within a pre-defined deadline. Moreover, the embodiments herein further provides executing the priority update on each of the remaining subset of tasks for optimizing the scheduling of the non-pre-emptive tasks.

Systems and methods for optimizing scheduling of non-preemptive tasks in multi-robotic environment

This disclosure relates to system and method for enabling a robot to perceive and detect socially interacting groups. Various known systems have limited accuracy due to prevalent rule-driven methods. In case of few data-driven learning methods, they lack datasets with varied conditions of light, occlusion, and backgrounds. The disclosed method and system detect the formation of a social group of people, or, f-formation in real-time in a given scene. The system also detects outliers in the process, i.e., people who are visible but not part of the interacting group. This plays a key role in correct f-formation detection in a real-life crowded environment. Additionally, when a collocated robot plans to join the group it has to detect a pose for itself along with detecting the formation. Thus, the system provides the approach angle for the robot, which can help it to determine the final pose in a socially acceptable manner.

System and method for enabling robot to perceive and detect socially interacting groups

This disclosure relates generally to human-robot interaction (HRI) to enable a robot to execute tasks that are conveyed in a natural language. The state-of-the-art is unable to capture human intent, implicit assumptions and ambiguities present in the natural language to enable effective robotic task identification. The present disclosure provides accurate task identification using classifiers trained to understand linguistic and semantic variations. A mixed-initiative dialogue is employed to resolve ambiguities and address the dynamic nature of a typical conversation. In accordance with the present disclosure, the dialogues are minimal and directed to the goal to ensure human experience is not degraded. The method of the present disclosure is also implemented in a context sensitive manner to make the task identification effective.

Growing community of inventors

Kolkata, India

Chayan Sarkar