Advanced New Technologies Co., Ltd.

Alibaba Group Holding Limited

Alipay (Hangzhou) Information Technology Co., Ltd.

Advantageous New Technologies Co., Ltd.

Le Song

Hui Li

Xiaofu Chang

Kailiang Hu

Lin Wang

Chen Chen

Guanyin Zhu

Xin Huang

Chunyang Wen

Yiguang Wang

Jianfeng Wen

Zhibang Ge

Yuan Qi

Xuqin Liu

This disclosure relates to processing sequential interaction data through machine learning. In one aspect, a method includes obtaining a dynamic interaction graph constructed based on a dynamic interaction sequence. The dynamic interaction sequence includes interaction feature groups corresponding to interaction events. Each interaction feature group includes a first object, a second object, and an interaction time of an interaction event that involved the first object and the second object. The dynamic interaction graph includes multiple nodes including, for each interaction feature group, a first node that represents the first object of the interaction feature group and a second node that represents the second object of the interaction feature group. A current sequence corresponding to a current node to be analyzed is determined. The current sequence is input into a Transformer-based neural network model. The neural network model determines a feature vector corresponding to the current node.

Processing sequential interaction data

A graphical structure model trained by using labeled samples is obtained. The graphical structure model is defined based on an enterprise relationship network that includes nodes and edges. Each labeled sample includes a label indicating whether a corresponding node is a risky credit node. The graphical structure model is configured to iteratively calculate an embedding vector of at least one node in a hidden feature space based on an original feature of the at least one node and/or a feature of an edge associated with the at least one node. An embedding vector corresponding to a test-sample is calculated by using the graphical structure model. Credit risk analysis is performed on the test-sample. The credit risk analysis is performed based on a feature of the test-sample represented in the embedding vector. A node corresponding to the test-sample is labeled as a credit risk node.

Graphical structure model-based credit risk control

One or more implementations of the present specification provide risk control of transactions based on a graphical structure model. A graphical structure model trained by using labeled samples is obtained. The graphical structure model is defined based on a transaction data network that includes nodes representing entities in a transaction and edges representing relationships between the entities. Each labeled sample includes a label indicating whether a node corresponding to the labeled sample is a risky transaction node. The graphical structure model is configured to iteratively calculate an embedding vector of the node in a latent feature space based on an original feature of the node or a feature of an edge associated with the node. An embedding vector of an input sample is calculated by using the graphical structure model. Transaction risk control is performed on the input sample based on the embedding vector.

Graphical structure model-based transaction risk control

Computer-implemented methods, computer-implemented systems, and non-transitory, computer-readable media for processing interaction sequence data are disclosed. One computer-implemented method includes: obtaining a dynamic interaction graph is obtained, where the dynamic interaction graph is constructed based on a dynamic interaction sequence, including a plurality of interactions arranged in a chronological order, where each interaction includes two objects involved in the interaction and a time of the interaction. In the dynamic interaction graph, a sub-graph corresponding to a target node is determined, where nodes in the sub-graph comprise the target node and connection nodes connected to the target node through a predetermined amount of edges originating from the target node. A feature vector corresponding to the target node is determined based on a node feature of each of the nodes of the sub-graph and directions of edges of the sub-graph.

Method and apparatus for processing user interaction sequence data

A graphical structure model trained with labeled samples is obtained. The graphical structure model is defined based on an account relationship network that comprises a plurality of nodes and edges. The edges correspond to relationships between adjacent nodes. Each labeled sample comprises a label indicating whether a corresponding node is an abnormal node. The graphical structure model is configured to iteratively calculate, for at least one node of the plurality of nodes, an embedding vector in a hidden feature space based on an original feature of the least one node and/or a feature of an edge associated with the at least one node. A first embedding vector that corresponds to a to-be-tested sample is calculated using the graphical structure model. Abnormal account prevention and control is performed on the to-be-tested sample based on the first embedding vector.

Graphical structure model-based prevention and control of abnormal accounts

Computer-implemented methods, systems, and apparatus, including computer-readable medium, for generating an action selection policy for causing an execution device to complete a task are described. Data representing a task that is divided into a sequence of subtasks are obtained. Data specifying a strategy neural network (SNN) for a subtask in the sequence of subtasks are obtained. The SNN receives inputs include a sequence of actions that reach an initial state of the subtask, and predicts an action selection policy of the execution device for the subtask. The SNN is trained based on a value neural network (VNN) for a next subtask that follows the subtask in the sequence of subtasks. An input to the SNN is determined. The input includes a sequence of actions that reach a subtask initial state of the subtask. An action selection policy for completing the subtask is determined based on an output of the SNN.

Determining action selection policies of an execution device

Computer-implemented methods, systems, and apparatus, including computer-readable medium, for generating an action selection policy for causing an execution device to complete a task are described. Data representing a task that is divided into a sequence of subtasks are obtained. For a specified subtask except for a first subtask in the sequence of subtasks, a value neural network (VNN) is trained. The VNN receives inputs include reach probabilities of reaching a subtask initial state of the specified subtask, and predicts a reward of the execution device in the subtask initial state of the specified subtask. A strategy neural network (SNN) for a prior subtask that precedes the specified subtask is trained based on the VNN. The SNN receives inputs include a sequence of actions that reach a subtask state of the prior subtask, and predicts an action selection policy of the execution device in the subtask state of the prior subtask.

Disclosed herein are methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an action selection policy of an execution device for completing a task in an environment. The method includes computing a hybrid sampling policy at a state of the execution device based on a sampling policy and an exploration policy, wherein the exploration policy specifies a respective exploration probability corresponding to each of multiple possible actions in the state, wherein the exploration probability is negatively correlated with a number of times that the each of the multiple possible actions in the state has been sampled; sampling an action among the multiple possible actions in the state according to a sampling probability of the action specified in the hybrid sampling policy; and updating an action selection policy in the state by performing Monte Carlo counterfactual regret minimization based on the action.

Disclosed herein are methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an action selection policy for completing a task in an environment. The method includes identifying multiple possible actions in a state, wherein the state corresponds to a vector of information sets; identifying a vector of current action selection policies in the state, wherein each current action selection policy in the vector of current action selection policies corresponds to an information set in the vector of information sets; computing a sampling policy based on the vector of current action selection policies in the state; sampling an action among the multiple possible actions in the state according to a sampling probability of the action specified in the sampling policy; and updating each current action selection policy of the execution device in the state based on the action.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, are described, for generating an action selection policy of an execution device for completing a task in an environment. The method includes, in a current iteration, computing a counterfactual value (CFV) of the execution device in a terminal state based on a payoff of the execution device and a reach probability of other devices reaching the terminal state; computing a baseline-corrected CFV of the execution device in the terminal state; for each non-terminal state having child states, computing a CFV of the execution device in the non-terminal state based on a weighted sum of the baseline-corrected CFVs of the execution device in the child states; computing a baseline-corrected CFV and a CFV baseline of the execution device in the non-terminal state; and determining an action selection policy in the non-terminal state for the next iteration.

Disclosed herein are methods, systems, and apparatus for generating an action selection policy (ASP) of an execution device. One method includes, in a current iteration, computing a first reward for a current state based on respective first rewards for actions in the current state and an ASP of the current state in the current iteration; computing an accumulative respective regret value of each action in the current state based on a difference between the respective first reward for the action and the first reward for the current state; computing an ASP of the current state in the next iteration; computing a second reward for the current state based on the respective first rewards for the actions and the ASP of the current state in the next iteration; and determining an ASP of the previous state in the next iteration based on the second reward for the current state.

Disclosed herein are methods, systems, and apparatus for generating an action selection policy (ASP) for an execution device. One method includes obtaining an ASP in a current iteration; obtaining a respective first reward for each action in a current state; computing a first reward of the current state based on the respective first rewards for the actions and the ASP; computing a respective regret value of each action based on a difference between the respective first reward for the action and the first reward for the current state; computing an incremental ASP based on the respective regret value of the each action in the current iteration; computing a second reward for the current state based on the incremental ASP; determining an ASP in a next iteration based on the second reward of the current state; and controlling actions of the execution device according to the ASP.

Growing community of inventors

Hangzhou, China

Le Song