Adobe, Inc.

Fair Isaac Corporation

Zhenyu Yan

Abhishek Pani

Wuyang Dai

Jun He

Chen Dong

Lei Zhang

Xiang Wu

Yi-Hong Kuo

Jin Xu

Yuan Yuan

Tianyu Wang

Wenqing Yang

Xinyue Liu

Polina Bartik

Marc-david Cohen

Fnu Arava Venkata Kesava Sai Kumar

Ning Li

Xiaojing Dong

Eugene Youjun Chen

Ashutosh Joshi

Shafi Ur Rahman

Chang Liu

Gerald Fahner

Shailesh Kumar

Amit Kiran Sowani

Yiwen Sun

Yang Wang

Jalaj Bhandari

Tingting Xu

Lijun Yu

Hsiang-Yu Yang

Suofei Wu

Shiyuan Gu

Arava Sai Kumar

Maoqi Xu

Shengyun Peng

Pinak Panigrahi

Deenadayalan Narayanaswamy

Bo Peng

Julia Viladomat

Jie Zhang

An improved analytics system generates actionable KPI-based customer segments. The analytics system determines predicted outcomes for a key performance indicator (KPI) of interest and a contribution value for each variable indicating an extent to which each variable contributes to predicted outcomes. Topics are generated by applying a topic model to the contribution values for the variables. Each topic comprises a group of variables with a contribution level for each variable that indicates the importance of each variable to the topic. User segments are generated by assigning each user to a topic based on attribution levels output by the topic model.

Actionable KPI-driven segmentation

Methods and systems are provided for improved electronic communication campaign technologies, which can automatically balance objectives or goals of an electronic communication campaign against an overall opt-out rate for the electronic communication campaign. An electronic communications frequency optimizer can generate individual contact frequencies for individual email recipients. Embodiments can avoid unnecessary or counterproductive communications while achieving overall campaign goals, and can use processes to improve the efficiency of systems. In some cases, embodiments cluster communication recipients into different groups based on their past actions, then optimizes the communication contact frequency on different groups, to avoid performing optimization directly on millions of recipients. Some embodiments automatically self-update, for example with recipients' recent responses, to generate and/or implement campaign communication schedules on an individual level.

Multi-objective electronic communication frequency optimization

Methods and systems are provided for facilitating time zone prediction using electronic communication data. Electronic message data associated with a message recipient of electronic communications is obtained. The electronic message data includes message delivery data associated with an electronic message and message response data associated with a response, by the message recipient, to a received electronic message. Using a machine learning model and based on the message delivery data and the message response data, a time-zone score is determined for a time zone. Such a time-zone score can indicate a probability the time zone corresponds with the message recipient. Based on the time-zone score, the time zone is identified as corresponding with the message recipient.

Facilitating time zone prediction based on electronic communication data

Systems and methods for customer journey optimization in email marketing are described. The systems and methods may identify a plurality of messages for a first time period, wherein the plurality of messages are categorized according to a plurality of messages types, identify user information for a customer, wherein the user information includes user interaction data, determine a message type from the plurality of message types for the first time period based on the user information, wherein the message type is determined using a decision making model comprising a deep Q-learning neural network, select a message from the plurality of messages based on the determined message type, and transmit the message to the customer during the first time period based on the selection.

Multi-objective customer journey optimization

The present disclosure relates to systems, non-transitory computer-readable media, and methods for generating and utilizing a touchpoint attribution attention neural network to identify and measure performance of touchpoints in digital content campaigns. For example, a deep learning attribution system trains a touchpoint attribution attention neural network using touchpoint sequences, which include user interactions with content via one or more digital media channels. In one or more embodiments, the deep learning attribution system utilizes the trained touchpoint attribution attention neural network to determine touchpoint attributions of touchpoints in a target touchpoint sequence. In addition, the deep learning attribution system can utilize the trained touchpoint attribution attention neural network to generate conversion predictions for target touchpoint sequences and to provide targeted digital content over specific digital media channels to client devices of individual users.

Identifying touchpoint contribution utilizing a touchpoint attribution attention neural network

The present disclosure relates to systems, methods, and non-transitory computer readable media for determining send times to provide electronic communications based on predicted response rates by utilizing a Bayesian approach and multi-armed bandit algorithms. For example, the disclosed systems can generate predicted response rates by training and utilizing one or more response rate prediction models to generate a weighted combination of user-specific response information and population-specific response information. The disclosed systems can further utilize a Bayes upper-confidence-bound send time model to determine send times that are more likely to elicit user responses based on the predicted response rates and further based on exploration and exploitation considerations. In addition, the disclosed systems can update the response rate prediction models and/or the Bayes upper-confidence-bound send time model based on providing additional electronic communications and receiving additional responses to modify model weights.

Utilizing a bayesian approach and multi-armed bandit algorithms to improve distribution timing of electronic communications

The present disclosure relates to systems, methods, and non-transitory computer readable media for generating a target distribution schedule for providing electronic communications based on predicted behavior rates by utilizing a genetic algorithm and one or more objective functions. For example, the disclosed systems can generate predicted behavior rates by training and utilizing one or more behavior prediction models. Based on the predicted behavior rates, the disclosed systems can further utilize a genetic algorithm to apply objective functions to generate one or more candidate distribution schedules. In accordance with the genetic algorithm, the disclosed systems can select a target distribution schedule for a particular user/client device. The disclosed systems can thus provide one or more electronic communications to individual users based on respective target distribution schedules.

Utilizing a genetic algorithm in applying objective functions to determine distribution times for electronic communications

Quantitative rating systems and techniques are described that prioritize customers by propensity to buy and buy size to generate customer ratings. In one example, a propensity model is used to determine a likelihood of a potential customer to purchase a product, and a projected timeframe buy size for the potential customer is determined. An expected value for the potential customer is generated by combining the likelihood of the potential customer to purchase the product and the projected timeframe buy size. In another example, a ratio model of annualized recurring revenue (ARR) is used to determine a timeframe buy size for an existing customer in consecutive time frames. An upsell opportunity for the existing customer is determined based on the timeframe buy size less an ARR for a current time frame for the existing customer. A rating of the potential or existing customer is output in a user interface.

Quantitative rating system for prioritizing customers by propensity and buy size

Methods, systems, and non-transitory computer readable storage media are disclosed for using reject inference to generate synthetic data for modifying lead scoring models. For example, the disclosed system identifies an original dataset corresponding to an output of a lead scoring model that generates scores for a plurality of prospects to indicate a likelihood of success of prospects of the plurality of prospects. In one or more embodiments, the disclosed system selects a reject inference model by performing simulations on historical prospect data associated with the original dataset. Additionally, the disclosed system uses the selected reject inference model to generate an imputed dataset by generating synthetic outcome data representing simulated outcomes of rejected prospects in the original dataset. The disclosed system then uses the imputed dataset to modify the lead scoring model by modifying at least one parameter of the lead scoring model using the synthetic outcome data.

Generating synthetic data using reject inference processes for modifying lead scoring models

Introduced here are approaches for identifying the optimal send time for messages by accounting for hidden confounders, such as the content of those messages, delivery channel, etc. These approaches use a causal inference framework to discover and then remove the impact of hidden confounders. These approaches may be employed by a marketing and analytics platform (or simply 'marketing platform') that may be used to design, implement, or review digital marketing campaigns. The marketing platform can consider the send time as a treatment and then employ machine learning (ML) models that consider the send time, features of the recipient, and hidden confounders to produce a ranked series of send times with the effect of the hidden confounders marginalized. Approaches to performing offline evaluations that mimic A/B tests using data related to existing field experiments are also introduced here.

Optimization of send time of messages

The present disclosure includes systems, methods, and non-transitory computer readable media that train and utilize multi-phase learning models to predict performance during digital content campaigns and provide digital content to client devices in a real-time bidding environment. In particular, one or more embodiments leverage organizational structure of digital content campaigns to train two learning models, utilizing different data sources, to predict performance, generate bid responses, and provide digital content to client devices. For example, the disclosed systems can train a first performance learning model in an offline mode utilizing parent-level historical data. Then, in an online mode, the disclosed systems can train a second performance learning model utilizing child-level historical data and utilize the first performance learning model and the second performance learning model to generate bid responses and bid amounts in a real-time bidding environment.

Training and utilizing multi-phase learning models to provide digital content to client devices in a real-time digital bidding environment

Utilizing a touchpoint attribution attention neural network to identify significant touchpoints and measure touchpoint contribution in multichannel, multi-touch digital content campaigns

Methods, systems, and computer readable storage media are disclosed for generating joint-probabilistic ensemble forecasts for future events based on a plurality of different prediction models for the future events. For example, in one or more embodiments the disclosed system determines error values for various predictions from a plurality of different prediction models (i.e., 'forecasters') for previous events. Moreover, in one or more embodiments the system generates an error probability density function by mapping the error values to an error space and applying a kernel density estimation. Furthermore, the system can apply the error probability density function(s) to a plurality of predictions from the forecasters for a future event to generate a likelihood function and a new prediction for the future event.

Utilizing joint-probabilistic ensemble forecasting to generate improved digital predictions

Growing community of inventors

Cupertino, CA, United States of America

Zhenyu Yan