Lemon Inc.

Beijing Volcano Engine Technology Co., Ltd.

Douyin Vision Co., Ltd.

Jianjun Chen

Li Feng Zhang

Rui Shi

Wei Ding

Yonghua Ding

Fangshi Li

Ye Liu

Mingyi Zhang

Lei Zhang

Yang Liu

Jianfeng Qian

Lixun Cao

Xiangrui Meng

Yuanjin Lin

Shicai Zeng

Junda Zhao

Jason Yang Sun

Kui Wei

Kai Wu

This disclosure provides a query task execution method, apparatus, computer device, storage medium. The method includes: receiving a query task for a target data table; determining at least one candidate degree of parallelism for executing the query task based on a first number of child tables obtained by pre-partitioning the target data table; selecting a target degree of parallelism within a preset range of degrees of parallelism from the at least one candidate degree of parallelism; and determining required computing units according to the target degree of parallelism, and evenly allocating the child tables to the computing units which concurrently execute the query task based on the allocated child tables.

Query task execution method, apparatus, computer device and storage medium

A hybrid transactional/analytical processing (HTAP) database includes an online transaction processing (OLTP) engine, an online analytical processing (OLAP) engine provided separately from the OLTP engine, and a shared storage holding data of multiple tenants. The shared storage is configured to receive a request from the OLAP engine for evaluation of a predicate against the data stored on the shared storage. The shared storage is further configured to determine that a CPU utilization of the shared storage exceeds a predicate refusal threshold. Upon determining that the CPU utilization of the shared storage exceeds the predicate refusal threshold, the shared storage is further configured to refrain from evaluating at least a portion of the predicate.

Predicate evaluation based on CPU utilization of shared storage

The present disclosure describes hybrid transactional and analytical processing (HTAP) techniques. A HTAP system comprises a first processing engine configured to perform online transactional processing, a second processing engine configured to perform online analytical processing, and a storage in communication with the first processing engine and the second processing engine. The first processing engine, the second processing engine, and the storage may be modularized and configured to be decoupled from each other. The system may be configured to capture data by the first processing engine in real time, organize the data in a first format in a first part of the storage for use by the first processing engine, propagate the data to a second part of the storage subsystem, and organize the data in a second format in the second part of the storage for use by the second processing engine.

Hybrid data processing system and method

The present disclosure describes techniques of providing data consistency for hybrid transactional and analytical processing. Logical logs and log serial numbers (LSNs) associated with the logical logs may be generated based on data captured by a first processing engine. The logical logs and the LSNs may be propagated to a storage subsystem configured to be in communication with the first processing engine and a second processing engine. The LSNs and information indicative of LSN schema versions may be stored and distributed by a metadata service. The first processing engine, the second processing engine, the storage subsystem and the metadata service are modularized, and support a LSN mechanism for maintaining data consistency.

Data consistency mechanism for hybrid data processing

The present disclosure describes storage techniques for hybrid transactional and analytical processing. Data captured by a first processing engine may be received. The first processing engine may be configured to perform online transactional processing). Multiple replicas of logical logs generated based on the data may be distributed to a Delta Store by applying a quorum protocol on the multiple replicas. Data in the Delta Store are stored in a row format and are visible to a query for online analytical processing performed by a second processing engine. Data may be flushed from the Delta Store to a Base Store based on one or more predetermined rules. Data in the Base Store are stored in a columnar format and may be accessible by the second processing engine.

Storage engine for hybrid data processing

Growing community of inventors

Los Angeles, CA, United States of America

Jianjun Chen