Shanghai Icekredit, Inc.

Lingyun Gu

Zhipan Guo

Wei Wang

Minqi Xie

Wan Duan

Yang Zhang

Haiquan Li

Yizeng Huang

Shuhui Chen

Kai Zhang

Chang Liu

Kai Wang

Xiaofeng Zhang

Xuan Wang

Zhenyu Wang

Pengfei Xie

Kaiping He

Junhong Zheng

Jianye Liu

Jie Xie

Anwei Jiang

A data processing method includes: obtaining first sample data, and determining a target model and a feature set corresponding to the target model; obtaining second sample data, and dividing the second sample data into a development data set and a validation data set based on a predetermined proportion or a predetermined chronological order; respectively determining final sample data of the first sample data and retained sample data of the development data set based on the target model, the feature set corresponding to the target model, the first sample data, the development data set and the validation data set; and merging the final sample data and the retained sample data to obtain a modeling data set corresponding to a first business project. A data processing apparatus, and a computer device for implementing the data processing method are further provided.

Data processing method and apparatus, and computer device

A distributed transaction processing method and system based on a message queue and a database is provided. In the method, a component encapsulation server generates a target compressed package according to the obtained first configuration information of a business request server and the obtained second configuration information of a business execution server, and sends the target compressed package to the business request server and the business execution server, respectively, so that the business request server and the business execution server can decompress and configure the target compressed package to deploy a transaction processing component and a message transmission path. The system includes the component encapsulation server, the business request server and the business execution server. The business request server and the business execution server communicate with each other through the message queue. The component encapsulation server communicates with the business request server and the business execution server, respectively.

Distributed transaction processing method and system based on message queue and database

A big data regression verification method includes: adding first data source information, second data source information, and data feature information to a preset configuration file; when a script running instruction is detected, running a python automation script to establish a first data access channel to a database of a business system and a second data access channel to a database of a big data system based on the first data source information and the second data source information; processing and calculating data in the database of the business system and the database of the big data system according to the data feature information; and determining whether a calculated first result file corresponding to the database of the business system is consistent with a calculated second result file corresponding to the database of the big data system, to verify the data. A big data regression verification apparatus is further provided.

Big data regression verification method and big data regression verification apparatus

A business data processing method includes: step S1: extracting domestic business sample data; step S2: selecting a sample data set S by training a model; step S3: extracting overseas business sample data; step S4: merging the data set S and a data set T to obtain a data set A; step S5: setting an initial sample weight; step S6: setting a variable for a quantity of iterations; step S7: training the model based on a current weight to obtain a model At; step S8: calculating a loss et of the model At in the data set T; step S9: caching the model At and the loss et; step S10: updating a sample weight; step S11: updating the quantity of iterations t=t+1; step S12: determining whether a termination condition is met; and step S13: when the termination condition is met, determining a final model.

Business data processing method and computer device

A business prediction method includes: obtaining a first business sample set and a second business sample set; performing training based on the first business sample set and the second business sample set to obtain a business prediction model, and predicting received to-be-predicted business information based on the business prediction model to obtain a business prediction result corresponding to the received to-be-predicted business information. A business prediction apparatus is further provided. The business prediction method and the business prediction apparatus take into account data features of some business samples of being rejected in a business validation, while considering business samples of passing the business validation. This restores a business scenario, reduces the waste of costs of the rejected samples, and balances demands for a modeling sample and a rejected sample reasonably when there are insufficient samples of passing the business validation.

Business prediction method and apparatus

A data query method and a data query system are provided. A data query server loads, based on a preset configuration interface, configuration metadata sent by a central cluster server for a target application programming interface (API), to a target storage region being located in a database server and associated with the target API, and loads queryable data associated with the target API to the target storage region. Then, the central cluster server sends a query instruction to the data query server based on query metadata in a data query request sent by a user terminal for the target API. After that, the data query server queries corresponding target query data in the target storage region and sends the target query data to the user terminal through the central cluster server. In this way, a data query service can be provided in a form of an API interface.

Data query method and system

Disclosed are a method and a device for graph data quality verification, which can perform quality verification of the graph data to be processed before importing the graph data to be processed to the target graph database, thereby avoiding generating a target list based on the graph data to be processed with errors. By determining whether there is an outlier in the target list, the abnormal graph data in the graph data to be processed can be detected to ensure the correctness of the graph data to be processed imported into the target graph database. By generating a graph data quality report, it is possible to verify whether the graph data to be processed has errors during the import process.

Method and device for graph data quality verification

Disclosed are a method and a device for processing distributed data. The method includes: integrating and configuring data analysis services of multiple users with different data analysis requirements into a distributed computing engine program to obtain an analysis service data package; configuring a distributed scheduler in the cluster server according to the analysis service data package, and calling the distributed scheduler to monitor a message content transmitted by a message middleware including multiple data analysis services to be executed; and generating a distributed data execution plan according to the message content, and performing distributed scheduling calculation on the distributed data execution plan to obtain a distributed calculation result.

Method and device for processing distributed data solving problem of manual intervention by data analysts

Disclosed are a method and a device for processing multi-source heterogeneous data. The data source to be processed of multi-source heterogeneous data and the field data of the field to be converted under each data source to be processed are determined, then the target standard attribute field of the field to be converted under each data source to be processed in the target data dimension is determined from a pre-configured conversion field library. Then, the fields to be converted under each data source to be processed are converted into corresponding target standard attribute fields, to obtain the field data of the target standard attribute field under each data source to be processed, thereby synthesizing the multi-source heterogeneous standard data of the target data dimension.

Method and device for processing multi-source heterogeneous data

Disclosed are a method of indicator information determination and an apparatus of indicator information determination. Firstly, configuration information in json format is obtained and a query path list is generated based on the configuration information. Secondly, a script file is generated according to a constraint condition of the information nodes corresponding to the query path and stored in a first database server. Then, according to a third identification information of the query request input by a terminal device, a target script file corresponding to a fourth identification information is searched in the first database server, then the target script file is executed in the second database server to obtain a query result.

Growing community of inventors

Shanghai, China

Lingyun Gu