Dropbox, Inc.

Oracle International Corporation

Jiarui Ding

Jongmin Baek

Gautam Singaraju

Ermo Wei

Scott McCrae

Vishal Vishnoi

Sangameswaran Viswanathan

Edward E Wong

Mark Joseph Sugg

Neeraj Kumar

Morgan Zerby

Maor Bar Asher

Prasang Upadhyaya

Aleksander Dash

Simon Shubbar

Wesley Liao

The present disclosure relates to systems, methods, and non-transitory computer-readable media that utilize machine learning models to generate identifier embeddings from digital content identifiers and then leverage these identifier embeddings to determine digital connections between digital content items. In particular, the disclosed systems can utilize an embedding machine-learning model that comprises a character-level embedding machine-learning model and a word-level embedding machine-learning model. For example, the disclosed systems can combine a character embedding from the character-level embedding machine-learning model and a token embedding from the word-level embedding machine-learning model. The disclosed systems can determine digital connections between the plurality of digital content items by processing these identifier embeddings for a plurality of digital content items utilizing a content management model. Based on the digital connections, the disclosed systems can surface one or more digital content suggestions to a user interface of a client device.

Utilizing machine-learning models to generate identifier embeddings and determine digital connections between digital content items

The present disclosure is directed toward systems, methods, and non-transitory computer-readable media for generating a dynamic facet by using a large language model. For example, the disclosed systems extract raw facet data from a plurality of content items stored in a content management system. In addition, the disclosed systems determine one or more facet content groups by grouping the plurality of content items according to the raw facet data. Further, the disclosed systems generate a facet prompt from the one or more facet content groups. Moreover, the disclosed systems generate a dynamic facet by providing the facet prompt to a large language model.

Dynamic facet generation using large language models

Techniques for learning and using content type embeddings. The content type embeddings have the useful property that a distance in an embedding space between two content type embeddings corresponds to a semantic similarity between the two content types represented by the two content type embeddings. The closer the distance in the space, the more the two content types are semantically similar. The farther the distance in the space, the less the two content types are semantically similar. The learned content type embeddings can be used in a content suggestion system as machine learning features to improve content suggestions to end-users.

Content type embeddings

The present disclosure is directed toward systems, methods, and non-transitory computer readable media for identifying and recommending team members for target users from a content management system utilizing a machine learning approach. In particular, the disclosed systems can generate a set of candidate team members from among users of the content management system based on various factors such as access to a common digital content item. In some embodiments, the disclosed systems further determine recommended team members from among the set of candidate team members. For example, the disclosed systems can utilize a machine learning approach to generate or predict recommended team members based on particular features extracted or determined for, or with respect to, the various candidate team members. In certain implementations, the disclosed systems further provide a recommended-team-member notification to notify a target user of a recommended team member.

Generating and providing team member recommendations for content collaboration

The present disclosure is directed toward systems, methods, and non-transitory computer readable media for generating and suggesting content collections for user accounts of a content management system using combinations of content-based features such as textual signals and visual signals. In some embodiments, the disclosed systems select a seed content item from among a plurality of content items associated with a user account within a content management system. From the seed content item, the disclosed systems can determine one or more germane topics and can cluster additional content items in relation to the germane topic(s). In addition, the disclosed systems can select one or more content items from a content cluster to provide as a suggested content collection.

Seeding and generating suggested content collections

Techniques disclosed herein relate to generating a hierarchical classification model that includes a plurality of classification models. The hierarchical classification model is configured to classify an input into a class in a plurality of classes and includes a tree structure. The tree structure includes leaf nodes and non-leaf nodes. Each non-leaf node has two child nodes associated with two respective sets of classes in the plurality of classes, where a difference between numbers of classes in the two sets of classes is zero or one. Each leaf node is associated with at least two but fewer than a first threshold number of classes. Each of the leaf nodes and non-leaf nodes is associated with a classification model in the plurality of classification models of the hierarchical classification model. The classification model associated with each respective node in the tree structure can be trained independently.

Techniques for generating a hierarchical model to identify a class among a plurality of classes

Techniques herein relate to improving quality of classification models for differentiating different user intents by improving the quality of training samples used to train the classification models. Pairs of user intents that are difficult to differentiate by classification models trained using the given training samples are identified based upon distinguishability scores (e.g., F-scores). For each of the identified pairs of intents, pairs of training samples each including a training sample associated with a first intent and a training sample associated with a second intent in the pair of intents are ranked based upon a similarity score between the two training samples in each pair of training samples. A particular pair of training samples with a highest similarity score is selected and provided as output with a suggestion for modifying the particular pair of training samples.

Utterance quality estimation

Techniques for improving quality of classification models for differentiating different user intents by improving the quality of training samples used to train the classification models are described. Pairs of user intents that are difficult to differentiate by classification models trained using the given training samples are identified based upon distinguishability scores (e.g., F-scores). For each of the identified pairs of intents, pairs of training samples each including a training sample associated with a first intent and a training sample associated with a second intent in the pair of intents are ranked based upon a similarity score between the two training samples in each pair of training samples. The identified pairs of intents and the pairs of training samples having the highest similarity scores may be presented to users through a user interface, along with user-selectable options or suggestions for improving the training samples.

Techniques disclosed herein relate to querying a hierarchical classification model that includes a plurality of classification models. The hierarchical classification model is configured to classify an input into a class in a plurality of classes and includes a tree structure. The tree structure includes leaf nodes and non-leaf nodes. Each non-leaf node has two child nodes associated with two respective sets of classes in the plurality of classes, where a difference between numbers of classes in the two sets of classes is zero or one. Each leaf node is associated with at least two but fewer than a first threshold number of classes. Each of the leaf nodes and non-leaf nodes is associated with a classification model in the plurality of classification models of the hierarchical classification model. The classification model associated with each respective node in the tree structure can be trained independently.

Growing community of inventors

Foster City, CA, United States of America

Jiarui Ding