Salesforce, Inc.

Ka Chun Au

Caiming Xiong

Chu Hong Hoi

Wenhao Liu

Bryan McCann

Shafiq Rayhan Joty

Shashank Harinath

Govardana Sachithanandam Ramachandran

Alexis Roos

Samson Min Rong Tan

Guangsen Wang

Gang Wu

Embodiments provide a software framework for evaluating and troubleshooting real-world task-oriented bot systems. Specifically, the evaluation framework includes a generator that infers dialog acts and entities from bot definitions and generates test cases for the system via model-based paraphrasing. The framework may also include a simulator for task-oriented dialog user simulation that supports both regression testing and end-to-end evaluation. The framework may also include a remediator to analyze and visualize the simulation results, remedy some of the identified issues, and provide actionable suggestions for improving the task-oriented dialog system.

Systems and methods for an end-to-end evaluation and testing framework for task-oriented dialog systems

Embodiments described herein provides a training mechanism that transfers the knowledge from a trained BERT model into a much smaller model to approximate the behavior of BERT. Specifically, the BERT model may be treated as a teacher model, and a much smaller student model may be trained using the same inputs to the teacher model and the output from the teacher model. In this way, the student model can be trained within a much shorter time than the BERT teacher model, but with comparable performance with BERT.

Systems and methods for distilled BERT-based training model for text classification

Growing community of inventors

Milbrae, CA, United States of America

Ka Chun Au