Microsoft Technology Licensing, LLC

Network Appliance, Inc.

Netapp, Inc.

Sriram S Rao

Christopher William Douglas

Srikanth Kandula

Carlo Aldo Curino

Subramaniam Venkatraman Krishnan

Konstantinos Karanasos

Virajith Jalaparti

John K Edwards

Ishai Menache

Avrilia Floratou

Ashvin Agrawal

Joseph Naor

Mostafa Elhemali

Kishore Chaliparambil

Mainak Ghosh

Kannababu Karanam

Chuan Liu

Peng T Li

Alan Rowe

Cheryl Marie Thompson

Douglas P Doucette

Sandeep Yadav

Ashish Prakash

Joydeep Sen Sarma

Alekh Jindal

Hiren Shantilal Patel

Baskar Sridharan

Karthick Krishnamoorthy

Milan Vojnovic

Raghunath Ramakrishnan

John Zhuge

Sri Sai Kameswara Pavan Kumar Kasturi

Jeffrey Thomas Rasley

Krishna Pradeep Reddy Tamma

Chakrapani Bhat Talapady

Rodrigo Lopes Cancado Fonseca

Andrew F Chung

Methods, systems, apparatuses, and computer program products are provided for prefetching data. A workload analyzer may identify job characteristics for a plurality of previously executed jobs in a workload executing on a cluster of one or more compute resources. For each job, identified job characteristics may include identification of an input dataset and an input bandwidth characteristic for the input dataset. A future workload predictor may identify future jobs expected to execute on the cluster based at least on the identified job characteristics. A cache assignment determiner may determine a cache assignment that identifies a prefetch dataset for at least one of the future jobs. A network bandwidth allocator may determine a network bandwidth assignment for the prefetch dataset. A plan instructor may instruct a compute resource of the cluster to load data to a cache local to the cluster according to the cache assignment and the network bandwidth assignment.

Cache and I/O management for analytics over disaggregated stores

A system that maps files into multiple parts and initiates storage of each part within a respective storage provider. The mapping facilitates this by correlating each of the multiple parts to a corresponding storage provider which may be different than the storage provider that stores a different part of the same file. The system then initiates storage of the multiple parts of the file in each of their corresponding storage provider in accordance with the mapping, and using a storage-provider-facing API. Similarly, retrieval and/or editing of a partial file may be accomplished by referencing the mapping, and using the same API issued to the appropriate storage provider. Each storage provider is capable of interfacing with the system using this same API.

Mapping storage across storage providers

Embodiments for efficient queue management for cluster scheduling and managing task queues for tasks which are to be executed in a distributed computing environment. Both centralized and distributed scheduling is provided. Task queues may be bound by length-based bounding or delay-based bounding. Tasks may be prioritized and task queues may be dynamically reordered based on task priorities. Job completion times and cluster resource utilization may both be improved.

Efficient queue management for cluster scheduling

Described herein is a system and method for ranking and/or taking an action regarding execution of jobs of a shared computing cluster based upon predicted user impact. Information regarding previous executions of a plurality of jobs is obtained, for example, from job execution log(s). Data dependencies of the plurality of jobs are determined. Job impact of each of the plurality of jobs as a function of the determined data dependencies is calculated. User impact of each of the plurality of jobs as a function of the determined data dependencies, the calculated job impact, and time is calculated. The plurality of jobs are ranked in accordance with the calculated user impact. An action is taken in accordance with the ranking of the plurality of jobs. The action can include automatic scheduling of the jobs and/or providing information regarding the rankings to a user.

Provenance driven job relevance assessment

Described herein is a system and method for selecting subexpressions to be materialized. For a predefined storage budget, subexpressions of a set of candidate subexpressions to be materialized to minimize query evaluation cost are selected based upon a calculated utility of the set of candidate subexpressions, interactions of the candidate subexpressions, and, a cost of evaluating the candidate subexpressions. Based upon the subexpressions selected to be materialized, subexpression(s) of the set of candidate subexpressions to use when evaluating particular queries of the set of queries to minimize query evaluation cost are determined.

Selection of subexpressions to materialize for datacenter scale

Embodiments are directed to progressively migrating source computer nodes where the source computer nodes perform a computer-implemented service. In one embodiment, a computer system determines that execution of the performed service is to be migrated from the source computer nodes to target computer nodes. The computer system groups the source computer nodes into multiple source subgroups, where each source subgroup includes at least one source computer node. The computer system then schedules creation of target subgroups of target nodes. These target subgroups include at least one source computer node and, themselves, correspond to a source subgroup. The computer system activates a first target subgroup corresponding to a first source subgroup, and deactivates the first source subgroup. In this manner, the first target subgroup replaces the first source subgroup. Still further, the target subgroups are scheduled to be created only after the first source subgroup has been deactivated.

Seamless cluster servicing

A system is provided to improve storage server performance. The system comprises a receiving component of a storage server to receive a partner server's file system metadata while the partner server operates in a normal mode. The partner server's file system metadata is stored in the storage servers metadata cache and is used to mount the partner server's file system when the first server transitions into a takeover mode of operation.

Method and apparatus to improve server performance associated with takeover and giveback procedures

A storage server maintains an active file system and at least one read-only, persistent point-in-time image (RPPI) of the active file system. The booting time of the storage server is reduced by delaying the mounting of the RPPI(s) until the storage server receives a request to access a corresponding RPPI after the storage server has already booted.

Method and system for reducing boot time of a storage server

A multi-stage technique invalidates and replaces loadable physical volume block numbers (pvbns) stored in indirect blocks of a dual vbn ('flexible') virtual volume (vvol) of a storage system to enable efficient image transfers and/or fragmentation handling of the flexible vvol. Each loadable pvbn of a pvbn/virtual vbn (vvbn) block pointer pair is converted into a special block pointer having a predefined reserved value that provides a temporary 'pvbn_unknown' placeholder until replaced by a real (actual) pvbn. The technique further allows the storage system to serve data from the flexible vvol using the placeholders while the actual pvbns are computed, thereby eliminating latencies associated with completion of actual pvbn replacement for the pvbn_unknown placeholders.

System and method for using pvbn placeholders in a flexible volume of a storage system

A file system migrates a traditional volume to a virtual volume without data copying. In an embodiment, a traditional volume index node is selected for migration. The traditional volume index node is converted to a virtual volume index node. In one embodiment, the virtual volume index node provides both physical address information and virtual address information.

Migrating a traditional volume to a virtual volume in a storage system

Growing community of inventors

Saratoga, CA, United States of America

Sriram S Rao