Meta Platforms, Inc.

Meta Platforms Technologies, LLC

Seungwhan Moon

Rajen Subba

Paul Anthony Crook

Shivani Poddar

Pararth Paresh Shah

Anuj Kumar

Xiaohu Liu

Honglei Liu

Zhiguang Wang

Xiaolei Li

Krishna Mittal

Bing Liu

Satwik Kottur

Shubham Khandelwal

Eun Joon Cho

Xin Ming Fan

Hardik Shah

Aram Markosyan

Hao Zhou

Yulong Qiu

Daniel Chai

Hu Xu

In one embodiment, a method includes receiving a first user request from a first user for generating a media montage from a client system during a dialog session with the first user, generating an initial media montage during the dialog session based on media collections associated with the first user, sending instructions for presenting the initial media montage to the client system during the dialog session, receiving a second user request from the first user from the client system during the dialog session for editing the initial media montage, generating an edited media montage from the initial media montage during the dialog session based on the second user request and a memory graph associated with the first user, and sending instructions for presenting the edited media montage to the client system during the dialog session.

Creating digital stories based on memory graphs and multi-turn dialogs for assistant systems

In one embodiment, a method includes receiving, at a client system, an audio input, where the audio input comprises a coreference to a target object, accessing visual data from one or more camera associated with the client system, where the visual data comprises images portraying one or more objects, resolving the coreference to the target object from among the one or more objects, resoling the target object to a specific entity, and providing, at the client system, a response to the audio input, where the response comprises information about the specific entity.

Multimodal entity and coreference resolution for assistant systems

In one embodiment, a method includes receiving a user request from a client system associated with a user, determining that the user request corresponds to a first suspended task, retrieving a first dialog state of the first suspended task from a dialog history associated with the user, generating a summary of the first suspended task based on the first dialog state using a natural-language generating (NLG) module, and sending instructions to the client system for providing the summary of the first suspended task to the user.

Conversation summarization during user-control task execution for assistant systems

In one embodiment, a method includes receiving, from a client system associated with a user, a user request comprising a reference to a target object, accessing visual data from the client system, wherein the visual data comprises images portraying the target object and one or more additional objects, and wherein attribute information of the target object is recorded in a multimodal dialog state, resolving the reference to the target object based on the attribute information recorded in the multimodal dialog state, determining relational information between the target object and one or more of the additional objects portrayed in the visual data, and sending, to the client system, instructions for presenting a response to the user request, wherein the response comprises the attribute information and the determined relational information.

Multimodal dialog state tracking and action prediction for assistant systems

In one embodiment, a method includes receiving a user request from a client system associated with a user, generating a response to the user request which references one or more entities, generating a personalized recommendation based on the user request and the response, wherein the personalized recommendation references one or more of the entities of the response, and sending instructions for presenting the response and the personalized recommendation to the client system.

Personalized conversational recommendations by assistant systems

In one embodiment, a method includes receiving a query from a user from a client system associated with the user, determining one or more initial memory slots based on the query, accessing a memory graph associated with the user which comprises a plurality of nodes and a plurality of edges connecting the nodes, and wherein one or more of the nodes correspond to one or more episodic memories of the user, respectively, and wherein each edge corresponds to a relationship between the connected nodes, selecting one or more candidate nodes from the memory graph by one or more machine-learning models based on the initial memory slots, generating a response based on the initial memory slots and episodic memories corresponding to the selected candidate nodes, and sending instructions for presenting the response to the client system in response to the query.

Memory grounded conversational reasoning and question answering for assistant systems

In one embodiment, a method includes receiving a query from a user from a client system associated with the user, accessing a knowledge graph comprising a plurality of nodes and edges connecting the nodes, wherein each node corresponds to an entity and each edge corresponds to a relationship between the entities corresponding to the connected nodes, determining one or more initial entities associated with the query based on the query, selecting one or more candidate nodes by a conversational reasoning model from the knowledge graph corresponding to one or more candidate entities, respectively, wherein each candidate node is selected based on the nodes corresponding to the initial entities, dialog states associated with the query, and a context associated with the query, generating a response based on the initial entities and the candidate entities, and sending instructions for presenting the response to the client system in response to the query.

Conversational reasoning with knowledge graph paths for assistant systems

In one embodiment, a method includes accessing visual data from a client system associated with a user, wherein the visual data comprises images portraying one or more objects, receiving, from the client system, a user request, wherein the user request comprises a coreference to a target object, resolving the coreference to the target object from among the one or more objects, resolving the target object to a specific entity, and sending, to the client system, instructions for providing a response to the user request, wherein the response comprises attribute information about the specific entity.

Growing community of inventors

Seattle, WA, United States of America

Seungwhan Moon