Future Wei Technologies, Inc.

Huawei Technologies Co., Limited

Yu Dong

Qingqing Zhou

Guogen Zhang

Mingyi Zhang

Huaizhi Li

Lei Liu

Yantao Qiao

Methods and apparatus are provided for using machine learning to estimate query resource consumption in a massively parallel processing database (MPPDB). In various embodiments, the machine learning may jointly perform query resource consumption estimation for a query and resource extreme events detection together, utilize an adaptive kernel that is configured to learn most optimal similarity relation metric for data from each system settings, and utilize multi-level stacking technology configured to leverage outputs of diverse base classifier models. Advantages and benefits of the disclosed embodiments include providing faster and more reliable system performance and avoiding resource issues such as out of memory (OOM) occurrences.

Using machine learning to estimate query resource consumption in MPPDB

A computer-implemented method and system at a network switch provides using one or more processors to perform a pre-defined database function on query data contained in data messages received at the network switch, with the performing producing result data, and wherein the pre-defined database function is performed on the query data in a first mode of operation to a state of full completion, generating complete result data and no skipped query data, and in a second mode of operation to a state of partial completion, generating partially complete result data and skipped query data. Further, the method and system performing one or more network switch functions to route the complete result data, and/or route the partially complete result data and skipped query data, to one or more destination nodes. In addition, an application programming interface (API) is used to define the database function.

Best-efforts database functions

A computer-implemented method and system are provided, including executing an application programming interface (API) in a network switch to define at least one of one or more database functions, performing, using one or more processors, the one or more database functions on at least a portion of data contained in a data message received at the switch, to generate result data, and routing the result data to one or more destination nodes. A database function-defined network switch includes a network switch and one or more processors to perform a pre-defined database function on query data contained in data messages received at the switch, to produce result data, wherein the pre-defined database function is performed on the query data in a first mode of operation to a state of full completion, generating complete result data and no skipped query data, or to a state of partial completion, generating partially completed result data and skipped query data.

Database functions-defined network switch and database system

Embodiments of the present technology relate managing database query concurrency. A method of the present technology can include receiving a query, generating a first query plan that can be used to execute the query in system memory without any system memory constraints, and estimating a system memory cost for executing the query in the system memory using the first query plan. The method can also include placing the query in a queue if available system memory does not satisfy the estimated system memory cost. The method can further include conditionally selecting the query from the queue, conditionally generating a second query plan for the query that can be used to execute the query in the system memory in compliance with a system memory constraint, and conditionally executing the query in the system memory.

Memory-aware plan negotiation in query concurrency control

Data messages having different priorities may be stored in different communication buffers of a network node. The data messages may then be forwarded from the communication buffers to working buffers as space becomes available in the working buffers. After being forwarded to the working buffers, the data messages may be available to be processed by upper-layer operations of the network node. Priorities may be assigned to the data messages based on a priority level of a query associated with the data messages, a priority level of an upper-layer operation assigned to process the data messages, or combinations thereof.

Growing community of inventors

San Jose, CA, United States of America

Yu Dong