Beijing Didi Infinity Technology and Development Co., Ltd.

Walmart Apollo, LLC

Didi Research America, LLC

Zhiwei Qin

Xiaocheng Tang

Tao Huang

Jieping Ye

Xuewen Chen

Jagtej Bewli

Yan Jiao

Shuaiji Li

Ishan Jindal

Chenxi Wang

Yinhong Chang

Arash Asadi-Shahmirzadi

Qingyang Li

Fangfang Zhang

Fan Zhang

Rishabh Kothari

Yang Yu

Somnath Banerjee

Mohan Akella

Fei Feng

Vvs Varaprasad Nagalla

Arash Asadi

Jiao Yan

Yintai Ma

Zhaodong Wang

Wenjie Shang

Yiping Meng

Hongtu Zhu

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for constructing a virtual environment for a ride-hailing platform are disclosed. An exemplary method comprises: obtaining a plurality of historical interaction trajectories each comprising one or more interaction records between a driver and a ride-hailing platform, each interaction record comprising a program recommendation of the ride-hailing platform to the driver and a reaction of the driver in response to the program recommendation; training a simulator based on the plurality of historical interaction trajectories; and integrating a reward function with the simulator to construct the virtual environment, wherein the plurality of first program recommendations and the plurality of reactions form a plurality of simulated interactions, and a data distribution of the plurality of simulated interactions approximates a data distribution of a plurality of interaction records in the plurality of historical interaction trajectories.

Method and system for constructing virtual environment for ride-hailing platforms

Deep reinforcement learning may be used for vehicle repositioning on mobility-on-demand platforms. Information may be obtained. The information may include a current location of a vehicle on a ride-sharing platform. A set of paths originated from the current location of the vehicle may be obtained. Each of the set of paths may have a length less than a preset maximum path length. A set of expected cumulative rewards along the set of paths may be obtained based on a trained deep value-network. A best path from the set of paths may be selected based on a heuristic tree search of the set of expected cumulative rewards. A next step along the best path may be recommended as a reposition action for the vehicle.

Vehicle repositioning on mobility-on-demand platforms

A unified learning approach for large-scale ride-hailing is described, the approach includes obtaining an offline state value network for predicting a value of a vehicle state, the offline state value network being trained based on a plurality of historical vehicle trajectories; initializing an online state value network and dispatching a plurality of vehicles according to the online state value network for a period of time; training the online state value network based on vehicle states of the plurality of vehicles before and after the dispatching and rewards associated with the dispatching; ensembling the trained online state value network and the offline state value network to obtain an ensembled online state value network; and dispatching the plurality of vehicles according to the ensembled online state value network.

Systems and methods for order dispatching and vehicle repositioning

A system and method of determining a policy to prevent fading drivers is described. The system and method creates virtual trajectories of incentives such as coupons offered to drivers in a transportation hailing system and corresponding states of drivers in response to the incentives. A joint policy simulator is created from an incentive policy, a confounding incentive policy, and an incentive object policy to generate the simulated actions of drivers in response to different incentives. The rewards of the simulated actions of the drivers is determined by a discriminator. The incentive policy for preventing fading drivers is optimized by reinforcement learning based on the virtual trajectories generated by the joint policy simulator and discriminator.

Reinforcement learning method for driver incentives: generative adversarial network for driver-system interactions

Multi-agent reinforcement learning may be used for rider order-dispatching via matching the distribution of orders and vehicles. Information may be obtained. The information may include a plurality of vehicle locations of a plurality of vehicles, a plurality of ride orders, and a current time. The obtained information may be input into a trained model. The trained model may be based on Kullback-Leibler divergence optimization and independent agents under a guidance of a joint policy. A plurality of order-dispatching tasks may be generated for the plurality of vehicles to fulfill.

Multi-agent reinforcement learning for order-dispatching via order-vehicle distribution matching

A method includes: obtaining a plurality of first signals corresponding to a vehicle and a plurality of second signals corresponding to a plurality of candidate carpool combinations each comprising one or more unassigned transportation orders, wherein: the plurality of first signals comprise a current time, a location of the vehicle at the current time, and one or more static features corresponding to the vehicle, the plurality of second signals comprise timestamps, origins, and destinations of the unassigned transportation orders, and the vehicle has an on-going transportation order at the current time; inputting the plurality of first and second signals to a trained machine learning model; and obtaining, from an output of the trained machine learning model, a utility score of each of the plurality of candidate carpool combinations.

Systems and methods for dispatching shared rides through ride-hailing platform

Systems and methods are provided for estimating travel time and distance. Such method may comprise obtaining a vehicle trip dataset comprising an origin, a destination, a time-of-day, a trip time, and a trip distance associated with each of a plurality of trips, and training a neural network model with the vehicle trip dataset to obtain a trained model. The neural network model may comprise a first module and a second module, the first module may comprise a first number of neuron layers, the first module may be configured to obtain the origin and the destination as first inputs to estimate a travel distance, the second module may comprise a second number of neuron layers, and the second module may be configured to obtain the information of a last layer of the first module and the time-of-day as second inputs to estimate a travel time.

System and method for estimating travel time and distance

Systems and methods are provided for ride order dispatching and vehicle repositioning. A method for ride order dispatching and vehicle repositioning, comprises: obtaining information comprising a location of a vehicle, current orders, and a current time; inputting the obtained information to a trained model; and determining action information for the vehicle based on an output of the trained model, the action information comprising: re-positioning the vehicle or accepting a ride order. The model is configured with: receiving information of drivers and information of orders as inputs; obtaining a global state based on the information of drivers, the information of orders, and a global time; and querying a plurality of driver-order pairs and driver-reposition pairs based at least on the obtained global state to determine the action information as the output.

System and method for ride order dispatching and vehicle repositioning

Systems and methods are provided for ride order dispatching. Such method may comprise obtaining information on a location of a vehicle and a time to input into a trained neural network algorithm; and based on a policy generated from the trained neural network algorithm, obtaining action information for the vehicle, the action information comprising: staying at a current position of the vehicle, re-positioning the vehicle, or accepting a ride order.

System and method for ride order dispatching

A ride order dispatching system comprises a processor, and a non-transitory computer-readable storage medium storing instructions that, when executed by the processor, cause the processor to perform a method. The method comprises: obtaining, from a computing device, a current location of a vehicle; inputting the current location of the vehicle and a time to a trained neural network model to obtain action information for the vehicle, the action information comprising: staying at the current location of the vehicle, re-positioning the vehicle, or accepting a ride order; and transmitting the action information to the computing device to cause the vehicle to stay at the current location, re-position to another location, or accept the ride order by proceeding to a pick-up location of the ride order.

Hierarchical multi-agent reinforcement learning may be used for joint order dispatching and fleet management for ride-sharing platforms. Information may be obtained. The information may include a status of a ride-sharing platform and a set of messages. The obtained information may be input into a trained hierarchical reinforcement learning (HRL) model. The trained HRL model may include at least one manager module corresponding to a region, and the at least one manager module may include a set of worker modules each corresponding to a division the region. At least one goal of the division in the region may be obtained based on the status of the ride-sharing platform and the set of messages. A vehicle action may be generated for each vehicle in the division in the region based on the status of the ride-sharing platform, the set of messages, and the at least one goal.

Joint order dispatching and fleet management for online ride-sharing platforms

A computer-implemented method for domain analysis comprises: obtaining, by a computing device, a domain; and inputting, by the computing device, the obtained domain to a trained detection model to determine if the obtained domain was generated by one or more domain generation algorithms. The detection model comprises a neural network model, a n-gram-based machine learning model, and an ensemble layer. Inputting the obtained domain to the detection model comprises inputting the obtained domain to each of the neural network model and the n-gram-based machine learning model. The neural network model and the n-gram-based machine learning model both output to the ensemble layer. The ensemble layer outputs a probability that the obtained domain was generated by the domain generation algorithms.

System and method for detecting generated domain

Systems and methods including one or more processing modules and one or more non-transitory storage modules storing computing instructions configured to run on the one or more processing modules and perform acts of receiving a plurality of advertisement creatives for an advertisement campaign, generating first predefined frequency weights for the plurality of advertisement creatives, and coordinating a display of the plurality of advertisement creatives within the impression slot of the webpages displayed to the online users based on the first predefined frequency weights. The first predefined frequency weights include a weighted frequency that each advertisement creative of the plurality of advertisement creatives should be displayed to the online users. A first advertisement creative of the can be displayed more frequently than a second advertisement creative because the first advertisement comprises a first frequency weight that is higher than a second frequency weight of the second advertisement creative.

Systems and methods of advertisement creatives optimization

A method for ride order dispatching comprises: obtaining a current location of a current vehicle from a computing device associated with the current vehicle; obtaining a current list of available orders nearby based on the current location; feeding the current location, the current list of available orders nearby, and a current time to a trained Markov Decision Process (MDP) model to obtain action information, the action information being repositioning the current vehicle to another current location or completing a current ride order by the current vehicle; and transmitting the generated action information to the computing device to cause the current vehicle to reposition to the another current location, stay at the current location, or accept the current ride order by proceeding to a pick-up location of the current ride order.

Growing community of inventors

San Jose, CA, United States of America

Zhiwei Qin