Microsoft Technology Licensing, LLC

Deyuan Ke

Timothy C Franklin

Gerald Haslhofer

Micheal Dunn

Aarohi Arora

Valentina I Strachan

Nabeei Shahzad

Methods and systems are provided that leverage existing information displayed or presented to a user by an app without requiring the integration of new libraries or requiring a user to use a specific app. Rather, methods and systems presented herein identify a task, select an app or multiple apps capable of performing the task, obtain information that may be required by the app or apps, determine a contextual understanding of what is displayed by the selected app or apps, and finally generate and automate the interaction with the app or apps such that the interaction with the app or apps is handled by an automation agent, allowing a user to stay focused on one or more tasks that the user needs to accomplish.

Growing community of inventors

Redmond, WA, United States of America

Deyuan Ke