Yummly, Inc.

Whirlpool Corporation

Gregory Allen Druck, Jr

Vadim Geshel

David B Feller

Iurii Volodimirovich Korolov

Brian Benjamin Witlin

Kane Cunningham

Ethan L Smith

Yuri Yuryev

Ryan N Smith

A system trains a computer model to classify images and to draw bounding boxes around classified objects in the images. The system uses a combination of partially labeled training images and fully labeled training images to train a model, such as a neural network model. The fully labeled training images include a classification label indicating a class of object depicted in the image, and bounding box or coordinate labels indicating a number of objects of the class in the image as well as the location of the objects of the class in the image. The partially labeled training images include a classification label but no indication of where in the image any objects of the class are located. Training the model using both types of training data makes it possible for the model to recognize and locate objects of classes that lack available fully labeled training data.

Detecting objects in images

User activity data describing how a user interacts with recipes posted on a web page or provided by an application is received. A first set of recommended recipes for the user is generated based on the user activity data. A content model that aligns recipe features extracted from the content of the recipes is built based on content of the recipes. A second set of recommended recipes is generated based on the content model. The first set of recommended recipes and the second set of recommended recipes are merged and transmitted for display to the user.

Recipe recommendation

Embodiments infer a total preparation time of a recipe. A recipe including preparation steps describing how to prepare a food item is obtained. A recipe server identifies preparation features in the preparation steps, where the preparation features represent portions of the preparations steps that are correlated with time to prepare the recipe. The recipe server obtains preparation times associated with the preparation features and combines these preparation times to estimate to total preparation time of the recipe. The estimated total preparation time is stored or transmitted to a client device in response to a request for the recipe. The estimated total preparation time is used to filter recipes sent in response to a request for recipes, where the request specifies a criterion based on total preparation time. The estimated total preparation time is used to select recipes for recommendation to a user based on that user's preferences.

Prediction of recipe preparation time

Embodiments infer a temporal attribute of a recipe. A recipe is obtained that includes recipe content such as preparation steps and ingredients. A recipe server identifies attribute features in the recipe content, where the attribute features are representative of portions of the recipe content that are correlated with temporal attributes. The recipe server determines whether the recipe is associated with a temporal attribute based on the attribute features and obtained attribute parameters corresponding to the attribute features. A temporal attribute determined to be associated with a recipe is transmitted to a client device in response to a request for the recipe. The estimated temporal attribute is used to filter recipes sent in response to a request for recipes, where the request specifies a criterion based on the temporal attribute. The estimated temporal attribute is used to select recipes for recommendation to a user based on a current time.

Inferring temporal attributes of a recipe

Embodiments infer a difficulty attribute of a recipe. A recipe is obtained that includes recipe content such as preparation steps and ingredients. A recipe server identifies attribute features in the recipe content, where the attribute features are representative of portions of the recipe content that are correlated with difficulty attributes. The recipe server determines whether the recipe is associated with a difficulty attribute based on the attribute features and obtained attribute parameters corresponding to the attribute features. A difficulty attribute determined to be associated with a recipe is transmitted to a client device in response to a request for the recipe. The estimated difficulty attribute is used to filter recipes sent in response to a request for recipes, where the request specifies a criterion based on the difficulty attribute. The estimated difficulty attribute is used to select recipes for recommendation to a user based on that user's preferences.

Inferring recipe difficulty

Recipes are hierarchically clustered into groups based on features of the recipes. Candidate clusters with a threshold number of clustered recipes having at least one feature in common are found by traversing the hierarchy. A plurality of clusters is selected for display to a user from among the candidates based on an objective function that considers the relevancy of the cluster as well as diversity of the clusters. A plurality of recipes within each selected cluster is selected for display to a user from among the recipes within the cluster based on an objective function that considers the quality of the recipe as well as the diversity of the recipes within the cluster. At least one feature that all of the recipes in a respective cluster have in common is used to generate a name for the cluster.

Clustering and display of recipes

Embodiments process a recipe from structured data to extract recipe text and select an image representative of the recipe. Recipes in structured data are retrieved and sequenced into segments to facilitate further processing. A recipe parser generates features corresponding to the segments. These generated features are inputs to a recipe model to classify the segments into components. This recipe model is trained according to classified training recipes. The trained model may then determine classifications for segments of the recipe. The classified recipe text is used to select the representative image for the recipe. To select this image, candidate images for a recipe are retrieved and filtered to remove unacceptable images. Features corresponding to these candidate images are generated and used as inputs to an image model to select the representative image. This image model is trained using representative training images of training recipes.

Growing community of inventors

San Francisco, CA, United States of America

Gregory Allen Druck, Jr