The Toronto-dominion Bank

Maksims Volkovs

Guangwei Yu

Tomi Johan Poutanen

Chundi Liu

Cheng Chang

Satya Krishna Gorti

Kin Kwan Leung

Juan Felipe Perez Vallejo

Paige Elyse Dickie

Jesse Cole Cresswell

Junwei Ma

Barum Rho

Xiao Shi Huang

Yaqiao Luo

Atiyeh Ashari Ghomi

Hamed Sadeghi

Ilan Gofman

Zhaoyue Cheng

Baiju Devani

Yichao Lu

Himanshu Rai

Kai Wang

Caitlin Messick

Lu Shu

Saba Zuberi

Jianjin Dong

Mathieu Jean Remi Ravaut

Jiapeng Wu

Jianing Sun

Keyvan Golestan Irani

Raunaq Suri

Alexander Clarence

Noël Vouitsis

Ding Tao Liu

Juan Felipe Vallejo

An object detection model and relationship prediction model are jointly trained with parameters that may be updated through a joint backbone. The offset detection model predicts object locations based on keypoint detection, such as a heatmap local peak, enabling disambiguation of objects. The relationship prediction model may predict a relationship between detected objects and be trained with a joint loss with the object detection model. The loss may include terms for object connectedness and model confidence, enabling training to focus first on highly-connected objects and later on lower-confidence items.

Co-learning object and relationship detection with density aware loss

The disclosed embodiments include computer-implemented processes and systems that establish configurable pipelines for training and deploying machine-learning processes in distributed computing environments. For example, an apparatus may obtain configuration data associated with a plurality of application engines, and pipelining data characterizing a sequential execution of at least a subset of the application engines. Based on the pipelining data, the apparatus may execute sequentially a subset of the application engines in accordance with the configuration data, which may cause the apparatus to perform operations that at least one of (i) train a machine-learning process or (ii) apply the trained machine-learning process to an input dataset. The apparatus may obtain artifact data generated by the executed subset of the application engines, store the artifact data within a data repository, and transmit a portion of the artifact data to a computing system that generated at least one element of the configuration data.

Configurable pipelines for training and deploying machine learning processes in distributed computing environments

A text-video recommendation model determines relevance of a text to a video in a text-video pair (e.g., as a relevance score) with a text embedding and a text-conditioned video embedding. The text-conditioned video embedding is a representation of the video used for evaluating the relevance of the video to the text, where the representation itself is a function of the text it is evaluated for. As such, the input text may be used to weigh or attend to different frames of the video in determining the text-conditioned video embedding. The representation of the video may thus differ for different input texts for comparison. The text-conditioned video embedding may be determined in various ways, such as with a set of the most-similar frames to the input text (the top-k frames) or may be based on an attention function based on query, key, and value projections.

Text-conditioned video representation

A computing system uses a large language model (LLM) to generate one or more synthetic queries for each document of a set of documents. For a user query, the computing system: selects one or more of the synthetic queries related to the user query; generates an adaptive few-shot prompt to instruct the LLM to generate a response to the query, wherein the adaptive few-shot prompt comprises an example query-response pair for each of the selected one more synthetic queries; provides the adaptive few-shot prompt to the LLM as an input; and generates an amended query based on the output of the LLM in response to the adaptive few-shot prompt.

Computing systems and methods for query expansion for use in information retrieval

The disclosed embodiments include computer-implemented apparatuses and processes that dynamically predict future occurrences of events using adaptively trained artificial-intelligence processes and contextual data. For example, an apparatus may generate an input dataset based on first interaction data and contextual data associated with a prior temporal interval, and may apply an adaptively trained, gradient-boosted, decision-tree process to the input dataset. Based on the application of the adaptively trained, gradient-boosted, decision-tree process to the input dataset, the apparatus may generate output data representative of a predicted likelihood of an occurrence of an event during a future temporal interval, which may be separated from the prior temporal interval by a corresponding buffer interval. The apparatus may also transmit a portion of the generated output data to a computing system, and the computing system may be configured to generate or modify second interaction data based on the portion of the output data.

Prediction of future occurrences of events using adaptively trained artificial-intelligence processes and contextual data

An image retrieval system receives an image for which to identify relevant images from an image repository. Relevant images may be of the same environment or object and features and other characteristics. Images in the repository are represented in an image retrieval graph by a set of image nodes connected by edges to other related image nodes with edge weights representing the similarity of the nodes to each other. Based on the received image, the image traversal system identifies an image in the image retrieval graph and alternatively explores and traverses (also termed 'exploits') the image nodes with the edge weights. In the exploration step, image nodes in an exploration set are evaluated to identify connected nodes that are added to a traversal set of image nodes. In the traversal step, the relevant nodes in the traversal set are added to the exploration set and a query result set.

Automated image retrieval with image graph

An online system trains a transformer architecture by an initialization method which allows the transformer architecture to be trained without normalization layers of learning rate warmup, resulting in significant improvements in computational efficiency for transformer architectures. Specifically, an attention block included in an encoder or a decoder of the transformer architecture generates the set of attention representations by applying a key matrix to the input key, a query matrix to the input query, a value matrix to the input value to generate an output, and applying an output matrix to the output to generate the set of attention representations. The initialization method may be performed by scaling the parameters of the value matrix and the output matrix with a factor that is inverse to a number of the set of encoders or a number of the set of decoders.

Initialization of parameters for machine-learned transformer neural network architectures

A recommendation system generates recommendations for user-item pairs based on embeddings in hyperbolic space. Each user and item may be associated with a local hyperbolic embedding representing the user or item in hyperbolic space. The hyperbolic embedding may be modified by neighborhood information. Because the hyperbolic space may have no closed form for combining neighbor information, the local embedding may be converted to a tangent space for neighborhood aggregation information and converted back to hyperbolic space for a neighborhood-aware embedding to be used in the recommendation score.

Recommendation with neighbor-aware hyperbolic embedding

The disclosed embodiments include computer-implemented processes that generate adaptive textual explanations of output using trained artificial intelligence processes. For example, an apparatus may generate an input dataset based on elements of first interaction data associated with a first temporal interval, and based on an application of a trained artificial intelligence process to the input dataset, generate output data representative of a predicted likelihood of an occurrence of an event during a second temporal interval. Further, and based on an application of a trained explainability process to the input dataset, the apparatus may generate an element of textual content that characterizes an outcome associated with the predicted likelihood of the occurrence of the event, where the element of textual content is associated with a feature value of the input dataset. The apparatus may also transmit a portion of the output data and the element of textual content to a computing system.

Generating adaptive textual explanations of output predicted by trained artificial-intelligence processes

A video localization system localizes actions in videos based on a classification model and an actionness model. The classification model is trained to make predictions of which segments of a video depict an action and to classify the actions in the segments. The actionness model predicts whether any action is occurring in each segment, rather than predicting a particular type of action. This reduces the likelihood that the video localization system over-relies on contextual information in localizing actions in video. Furthermore, the classification model and the actionness model are trained based on weakly-labeled data, thereby reducing the cost and time required to generate training data for the video localization system.

Weakly supervised action selection learning in video

The disclosed embodiments include computer-implemented processes that flexibly and dynamically analyze a machine learning process, and that generate analytical output characterizing an operation of the machine learning process across multiple analytical periods. For example, an apparatus may receive an identifier of a dataset associated with the machine learning process and feature data that specifies an input feature of the machine learning process. The apparatus may access at least a portion of the dataset based on the received identifier, and obtain, from the accessed portion of the dataset, a feature vector associated with the machine learning process. The apparatus may generate a plurality of modified feature vectors based on the obtained feature vector, and based on an application of the machine learning process to the obtained and modified feature vectors, generate and transmit, to a device, first explainability data associated with the specified input feature for presentation within a digital interface.

Dynamic analysis and monitoring of machine learning processes

A modeling system trains a recurrent machine-learned model by determining a latent distribution and a prior distribution for a latent state. The parameters of the model are trained based on a divergence loss that penalizes significant deviations between the latent distribution the prior distribution. The latent distribution for a current observation is a distribution for the latent state given a value of the current observation and the latent state for the previous observation. The prior distribution for a current observation is a distribution for the latent state given the latent state for the previous observation independent of the value of the current observation, and represents a belief about the latent state before input evidence is taken into account.

Regularization of recurrent machine-learned architectures with encoder, decoder, and prior distribution

A content retrieval system uses a graph neural network architecture to determine images relevant to an image designated in a query. The graph neural network learns a new descriptor space that can be used to map images in the repository to image descriptors and the query image to a query descriptor. The image descriptors characterize the images in the repository as vectors in the descriptor space, and the query descriptor characterizes the query image as a vector in the descriptor space. The content retrieval system obtains the query result by identifying a set of relevant images associated with image descriptors having above a similarity threshold with the query descriptor.

Growing community of inventors

Toronto, Canada

Maksims Volkovs