Microsoft Technology Licensing, LLC

Arnd Christian König

Vivek Ravindranath Narasayya

Ajay Kalhan

John Morgan Oslake

Pankaj Arora

Shize Xu

Junfeng Dong

Jiaqi Liu

Changsong Li

Manoj A Syamala

Danyel Aharon Fisher

Surajit J Chaudhuri

Steven Mark Drucker

Mark Steven Manasse

Michael Gamon

Sudipto Das

Feng Li

Qiang Wu

Eric David Brill

Cesar Alejandro Galindo-Legaria

Paul Adrian Oltean

Sumit Basu

Kenneth Ward Church

Venkatesh Ganti

Shubha Umesh Nabar

Matthew Francis Hurst

Sudip Roy

Edgars Sedols

Parag Nandan Paul

Sumeet Priyadarshee Dash

Yi Shan

Manish Kumar

Igor Dvorkin

Martin Miroslavov Markov

Dang Hai Pham

Ye Eun Park

Chi Yang

Mahadevan Sankara Subramanian

Tobias Ziegler

Aarati Kakaraparthy

Manoj Syamala

Roger P Menezes

Monwhea Jeng

Dmitriy A Belenko

Kabita Mahapatra

Intelligent memory brokering for multiple process instances, such as relational databases (e.g., SQL servers), reclaims memory based on value, thereby minimizing cost across instances. An exemplary solution includes: based at least on a trigger event, determining a memory profile for each of a plurality of process instances at a computing node; determining an aggregate memory profile, the aggregate memory profile indicating a memory unit cost for each of a plurality of memory units; determining a count of memory units to be reclaimed; identifying, based at least on the aggregate memory profile and the count of memory units to be reclaimed, a count of memory units to be reclaimed within each process instance so that a total cost is minimized to reclaim the determined count; and communicating, to each process instance having identified memory units to be reclaimed, a count of memory units to be reclaimed within the process instance.

Memory management for multiple process instances

Placement of a tenant database in an oversubscribed, database-as-a-service cluster comprised of a plurality of nodes is described. The placement may be based on per-node estimates of a probability of resource demand violation if the tenant database is placed on the node. Past resource usage of similar tenant databases subscribed to the cluster that are collected and stored as compressed traces may be used to obtain the estimates. In some examples, based on the estimates, a per-node expected number of resource violations is determined and compared across nodes, where the determined placement minimizes the number of resource violations. In other examples, when the tenant database is being placed in parallel with other tenant databases, a score assigned to each valid configuration for the placement may be modified based on the estimates, where the determined placement is the configuration having a lowest score.

Tenant database placement in oversubscribed database-as-a-service cluster

Methods for distributed histogram computation in a framework utilizing data stream sketches and samples are performed by systems and devices. Distributions of large data sets are scanned once and processed by a computing pool, without sorting, to generate local sketches and value samples of each distribution. The local sketches and samples are utilized to construct local histograms on which cardinality estimates are obtained for query plan generation of distributed queries against distributions. Local statistics of distributions are also merged and consolidated to construct a global histogram representative of the entire data set. The global histogram is utilized to determine a cardinality estimation for query plan generation of incoming queries against the entire data set. The addition of new data to a data set or distribution involves a scan of the new data from which new statistics are generated and then merged with existing statistics for a new global histogram.

Distributed histogram computation framework using data stream sketches and samples

Bias correcting system for small number estimators. A computer system includes a distinct value estimator configured to estimate a number of distinct values in a data set. The computer system includes a bias table for the estimator. The bias table includes entries with values corresponding to biases caused by the distinct value estimator correlated to values corresponding to numbers estimated. The entries in the table are optimized by having a set of entries with an optimized number of biases in the entries. The biases in the entries are associated with predetermined confidence intervals. The system includes a bias corrector configured to correct the number of distinct values in the multiset data estimated by the distinct value estimator set using values from the bias table to produce a corrected value. The system includes a user interface coupled to the bias corrector configured to output the corrected value to a user.

Low cardinality bias correction system

Methods for determining cache activity and for optimizing cache reclamation are performed by systems and devices. A cache entry access is determined at an access time, and a data object of the cache entry for a current time window is identified that includes a time stamp for a previous access and a counter index. A conditional counter operation is then performed on the counter associated with the index to increment the counter when the time stamp is outside the time window or to maintain the counter when the time stamp is within the time window. A counter index that identifies another counter for a previous time window where the other counter value was incremented for the previous cache entry access causes the other counter to be decremented. A cache configuration command to reclaim, or additionally allocate space to, the cache is generated based on the values of the counters.

System and method for determining cache activity and optimizing cache reclamation

A solution is disclosed for memory management of serverless databases that includes: based at least on detecting a trigger event, determining whether memory is to be reclaimed; based at least on determining that memory is to be reclaimed, determining an amount of memory to be reclaimed; identifying memory to be reclaimed; and reclaiming the identified memory. Disclosed solutions are flexible, enabling customization of the aggressiveness and manner of memory reclamation. This permits users to specify a tailored balance point between performance and cost, for arrangements that bill users based on resource usage (e.g., memory consumed by a serverless database). In some examples, users specify a ramp-down parameter that is used to determine how much memory can be evicted in a particular reclamation event, time intervals (or another criteria) for triggering a reclamation event, and a definition for whether a cache is active.

Memory management for serverless databases

Architecture that enables a Database-as-a-Service (DaaS) to auto-scale container sizes on behalf of tenants. An abstraction is provided that enables tenants to reason about monetary budget and query latency, rather than resource provisioning. An auto-scaling module automatically determines a container size for a subsequent billing interval based on telemetry that comprises latencies (e.g., waits), resource utilizations, and available budget, for example. A set of robust signals are derived from database engine telemetry and combined to significantly improve accuracy of resource demand estimation for database workloads. In a more specific implementation, resource demands can be estimated for arbitrary SQL (structured query language) workloads in a relational database management system (RDBMS).

Automatic demand-driven resource scaling for relational database-as-a-service

The described implementations relate to tunable predicate discovery. One implementation is manifest as a method for obtaining a data set and determining anomaly scores for anomalies of an attribute of interest in the data set. The method can also generate a ranked list of predicates based on the anomaly scores and cause at least one of the predicates of the ranked list to be presented.

Performance anomaly diagnosis

Incremental query results and confidence interval values associated with respective incremental query results may be obtained. Visualization shape objects indicating uncertainty values may be determined, based on mapping values of respective incremental query results and confidence interval values to points in the associated visualization shape objects, the uncertainty values visualized based on proportional shapes of the visualization shape objects. At least one visualization comparison object representing a comparison of a plurality of distributions associated with the obtained incremental query results and confidence interval values may be determined. Display of the plurality of visualization shape objects and the at least one visualization comparison object may be initiated.

Visualization of changing confidence intervals

Hash values corresponding to a file are processed in windows to determine a minimum hash value for each window. Each window may begin at a minimum hash value determined for a previous window and end after a fixed number of hash values. If a hash value is less than a threshold hash value, it is added to a buffer that is used to store the hash values in sorted order for a current window. If a hash value is greater than the threshold, it is added to another buffer whose hash values are not stored in sorted order. At the end of the current window, the minimum hash value in the first buffer is selected as the landmark for the window. If the first buffer is empty, then the hash values in the other buffer are sorted and the minimum hash value is selected as the landmark for the window.

Determination of landmarks

The subject disclosure is directed towards simulating query execution to provide incremental visualization for a global data set. A data store may be configured for searching at least a portion of a global data set being stored at an enterprise-level data store. In response to a user-issued query, partial query results are provided to a front-end interface for display to the user. The front-end interface also provides statistical information corresponding to the partial query results in relation to the global data set, which may be used to determine when a current set of query results becomes acceptable as a true/accurate estimate.

Incremental visualization for structured data in an enterprise-level data store

Described is estimating whether an online search query is a news-related query, and if so, outputting news-related results in association with other search results returned in response to the query. The query is processed into features, including by accessing corpora that corresponds to relatively current events, e.g., recently crawled from news and blog articles. A corpus of static reference data, such as an online encyclopedia, may be used to help determine whether the query is less likely to be about current events. Features include frequency-related data and context-related data corresponding to frequency and context information maintained in the corpora. Additional features may be obtained by processing text of the query itself, e.g., 'query-only' features.

Click-through prediction for news queries

Growing community of inventors

Kirkland, WA, United States of America

Arnd Christian König