Microsoft Technology Licensing, LLC

Ebay, Incorporated

Amit Srivastava

Ji Li

Ajinkya Gorakhnath Kale

Derek Martin Johnson

Stefan Schoenmackers

Furu Wei

Adit Krishnan

Xingxing Zhang

Gencheng Wu

Konstantin Seleskerov

Priyanka Vikram Sinha

Ming Zhou

Tomer Lancewicki

Sanjika Hewavitharana

Braddock Craig Gaskill

Youjun Liu

Farah Abdallah

Gaurav Vinayak Tendolkar

Elaine Lee

Robert Enyedi

Xinyu Zhang

Jayanth Vasudevan

Dominique Jean Bouchon

Thrivikrama Taula

Dishan Gupta

Dachuan Zhang

Lawrence William Colagiovanni

Fatima Zohra Daha

Tapan Bohra

Muin Barkatali Momin

Vinh Khuc

Siliang Kang

Mingxi Cheng

Huey-Ru Tsai

Ana Karen Parra

Robert Fernand Gordan

Sheng Zhao

Han Hu

Ramya Tridandapani

Yue Cao

Akshay Mallipeddi

Qi Dai

John Christian Leone

Yixuan Wei

Xiaozhi Yu

Bohan Li

Muqi Li

Emily Lauren Tohir

SivaPriya Kalyanaraman

Huakai Liao

Aman Malik

Joshua Alexander Doctors

Sindhu Vigasini Jambunathan

Xinfeng Chen

Brittany Elizabeth Mederos

Kevin Dara Khieu

Gregory Alexander DePaul

A data processing system implements obtaining textual content in a first language from a first client device and segmenting the textual content into a plurality of first tokens. The system also implements translating the first tokens from the first language to a second language using a bilingual dictionary, extracting features information from the second tokens to create a features vector, providing the feature vector to a first natural language processing model trained to analyze textual input in the second language and to output contextual information indicating one or more topics or subject matter of the first textual content, and providing the contextual information to a first machine learning model configured to analyze the contextual information and to identify one or more content items predicted to be relevant to the contextual information. The system further implements providing the information identifying the one or more content items to the first client device.

Multilingual support for natural language processing applications

A data processing system is implemented for receiving an image of a canvas, and detecting design features within the canvas by analyzing the image. The data processing system is implemented for determining an execution of a command by comparing the image to a current image of the canvas, and modifying the design features within the canvas to generate layouts in response to determining the execution of the command. Further, the data processing system is implemented for determining parameters of the modified design features, and determining a value for each layout by applying rules to the parameters of the modified design features. In addition, the data processing system is implemented for displaying the layouts based on the determined value for each layout to a user.

Content layout systems and processes

A method and system for generating one or more questions relating to a presentation session includes receiving audio data from the presentation session, retrieving a transcript for the audio data, receiving other data relating to the presentation session, providing at least one of the transcript and the other data to a machine-learning (ML) model as input for automatically generating the one or more questions relating to the presentation session, receiving from the ML model the one or more questions, and providing the one or more questions for display on a user interface associated with the presentation session.

Method and system of automated question generation for speech assistance

Aspect pre-selection techniques using machine learning are described. In one example, an artificial assistant system is configured to implement a chat bot. A user then engages in a first natural-language conversation. As part of this first natural-language conversation, a communication is generated by the chat bot to prompt the user to specify an aspect of a category that is a subject of a first natural-language conversation and user data is received in response. Data that describes this first natural-language conversation is used to train a model using machine learning. Data is then received by the chat bot as part of a second natural-language conversation. This data, from the second natural-language conversation, is processed using the model as part of machine learning to generate the second search query to include the aspect of the category automatically and without user intervention.

Aspect pre-selection using machine learning

A system and method and for retrieving one or more one or more multimodal assets includes receiving a search query for searching for one or more multimodal assets from among a plurality of candidate multimodal assets, encoding the search query into one or more query embedding representations via a trained query representation machine-learning (ML) model, comparing, via a matching unit, the one or more query embedding representations to a plurality of multimodal tensor representations, each of the plurality of multimodal tensor representations being a representation of one of the plurality of candidate multimodal assets, and identifying, based on the comparison, at least one of the plurality of the candidate multimodal assets as a search result for the search query, and providing the at least one of the plurality of the candidate multimodal assets for display as the search result.

Method and system of retrieving multimodal assets

A system and method and for retrieving assets from a personalized asset library includes receiving a search query for searching for assets in one or more asset libraries, the one or more asset libraries including a personalized asset library; encoding the search query into embedding representations via a trained query representation machine-learning (ML) model; comparing, via a matching unit, the query embedding representations to a plurality of asset representations, each of the plurality of asset representations being a representation of one of the plurality of candidate assets; identifying, based on the comparison, at least one of the plurality of the candidate assets as a search result for the search query; and providing the identified plurality of candidate assets for display as the search result. The plurality of asset representations for the one or more assets in the personalized content library are generated automatically without human labeling.

Method and system of retrieving assets from personalized asset libraries

A data processing system for generating training data for a multilingual NLP model implements obtaining a corpus including first and second content items. The first content items are English-language textual content, and the second content items are translations of the first content items in one or more non-English target languages. The system further implements selecting a first content item from the first content items, generating a plurality of candidate labels for the first content item by analyzing the first content item with a plurality of first English-language NLP models, selecting a first label from the plurality of candidate labels, generating first training data by associating the first label with the first content item, generating second training data by associating the first label with a second content item of the second content items, and training a pretrained multilingual NLP model with the first training data and the second training data.

Multilingual model training using parallel corpora, crowdsourcing, and accurate monolingual models

A computing apparatus comprises one or more computer readable storage media, one or more processors operatively coupled with the one or more computer readable storage media, and program instructions stored on the one or more computer readable storage media. The program instructions, when executed by the one or more processors, direct the computing apparatus to at least generate an audio recording of speech, extract features from the audio recording indicative of vocal patterns in the speech, determine a register classification of the speech based at least on the features, and display an indication of the register classification in a user interface.

Application software and services with register classification capabilities

A data processing system implements obtaining first textual content in a first language from a first client device; determining that the first language is supported by a first machine learning model; obtaining a guard list of prohibited terms associated with the first language; determining that the textual content does not include one or more prohibited terms associated based on the guard list; providing the first textual content as an input to the first machine learning model responsive to the textual content not including the one or more prohibited terms; analyzing the first textual content with the first machine learning model to obtain a first content recommendation; obtaining a first content recommendation policy that identifies content associated with the first language that may not be provided as a content recommendation; determining that the first content recommendation is not prohibited; and providing the first content recommendation to the first client device.

Multilingual content recommendation pipeline

The present disclosure relates to application of artificial intelligence (AI) processing that adapts one or more video feeds relative to presentation content. Trained AI processing automatically generates a combined representation comprising one or more video feeds and presentation content. An exemplary combined representation is the result of contextual analysis by one or more trained AI models that are adapted to consider how to adapt presentation of a video feed relative to displayable presentation content (or visa-versa). A combined representation of one or more video feeds and presentation content is automatically generated (and subsequently rendered) based on a result of contextual evaluation of data associated with a video feed and data attributes of presentation content. A combined representation may comprise a modification of the one or more video feeds, objects of presentation content or a combination thereof. Further examples extend to management of combined representations through an adapted graphical user interface.

Automated adaptation of video feed relative to presentation content

A system and method and for retrieving one or more visual assets includes receiving a search query for the one or more visual assets, the search query including textual data, encoding the textual data into one or more text embedding representations via a trained text representation machine-learning (ML) model, transmitting the one or more text embedding representations to a matching and selection unit, providing visual embedding representations of one or more visual assets to the matching and selection unit, comparing, by the matching and selection unit, the one or more text embedding representations to the visual embedding representations to identify one or more visual asset search results, and providing the one or more visual asset search results for display.

Method and system of content retrieval for visual data

Automatic generation of intelligent content is created using a system of computers including a user device and a cloud-based component that processes the user information. The system performs a process that includes receiving an input document and parsing the input document to generate inputs for a natural language generation model using a text analysis model. The natural language generation model generates one or more candidate presentation scripts based on the inputs. A presentation script is selected from the candidate presentation scripts and displayed. A text-to-speech model may be used to generate a synthesized audio presentation of the presentation script. A final presentation may be generated that includes a visual display of the input document and the corresponding audio presentation in sync with the visual display.

Automated script generation and audio-visual presentations

Techniques performed by a data processing system include establishing an online presentation session for conducting an online presentation, receiving first media streams comprising presentation content from the first computing device, receiving second media streams from the second computing devices of a subset of the plurality of participants, the second media streams including audio content, video content, or both of the subset of the plurality of participants, analyzing the first media streams using first machine learning models to generate feedback results, analyzing the set of second media streams to identify first reactions by the participants to obtain reaction information, automatically analyzing the feedback results and the reactions to identify discrepancies between the feedback results and the reactions, and automatically updating one or more parameters of the machine learning models based on the discrepancies to improve the suggestions for improving the online presentation.

Techniques for presentation analysis based on audience feedback, reactions, and gestures

Methods, systems, and computer programs are presented for detecting a mission changes in a conversation. A user utterance from a user device is received. The user utterance is part of a conversation with an intelligent assistant. The conversation includes preceding user utterances in pursuit of a first mission. It is determined that the user utterance indicates a mission change from the first mission to a second mission based on an application of a machine-learned model to the user utterance and the preceding user utterances. The machine-learned model has been trained repeatedly with past utterances of other users over a time period, the determining based on a certainty of the indication satisfying a certainty threshold. Responsive to the determining that the user utterance indicates the mission change from the first mission to a second mission, a reply to the user utterance is generated to further the second mission rather than the first mission.

Detection of mission change in conversation

A real-time name mispronunciation detection feature can enable a user to receive instant feedback anytime they have mispronounced another person's name in an online meeting. The feature can receive audio input of a speaker and obtain a transcript of the audio input; identify a name from text of the transcript based on names of meeting participants; and extract a portion of the audio input corresponding to the name identified from the text of the transcript. The feature can obtain a reference pronunciation for the name using a user identifier associated with the name; and can obtain a pronunciation score for the name based on a comparison between the reference pronunciation for the name and the portion of the audio input corresponding to the name. The feature can then determine whether the pronunciation score is below a threshold; and in response, notify the speaker of a pronunciation error.

Real-time name mispronunciation detection

Growing community of inventors

San Jose, CA, United States of America

Amit Srivastava