International Business Machines Corporation

Rensselaer Polytechnic Institute

Matthew Peveler

Jeffrey Owen Kephart

Hui Su

Maira Gatti De Bayser

Xiangyang Mou

Melina De Vasconcelos Alberio Guerra

Rahul Divekar

Lisha Chen

A computer-implemented method for multi-modal interaction with structured web page data includes automatically injecting event monitoring code into a web page. One or more event streams generated by the injected event monitoring code are received. A request to perform an operation on data displayed on the web page is received from a user. A command is generated that has one or more parameters based at least in part on one or more mappings between the received request and the received one or more event streams. The command to fulfil the received request is executed by interacting with the web page.

Using natural language to control structured web page data

Human speech signals that are uttered within an environment are transcribed; the environment includes one or more avatars representing one or more software agents; the human speech signals are directed to at least one of the avatars. At least one non-speech behavioral trace is obtained within the environment; the trace is representative of non-speech behavior directed to the at least one of the avatars. The transcribed human speech signals and the at least one non-speech behavioral trace are forwarded to the one or more software agents. A proposed act is obtained from at least one of the agents; responsive thereto, a command is issued to cause the avatar corresponding to the software agent from which the proposed act is obtained to emit synthesized speech and to act visually in accordance with the proposed act.

Embodied negotiation agent and platform

Growing community of inventors

Troy, NY, United States of America

Matthew Peveler