Advanced Micro Devices Corporation

Core Scientific, Inc.

Core Scientific Operating Company

Max Alt

Jesse Barnes

Paulo Roberto Pereira De Souza Filho

Ian Ferreira

Gabriel Martin

Albert Hse-Lin Chen

Systems and methods for allocating computing resources within a distributed computing system are disclosed. Computing resources such as CPUs, GPUs, network cards, and memory are allocated to jobs submitted to the system by a scheduler. System configuration and interconnectivity information is gathered by a mapper and used to create a graph. Resource allocation is optimized based on one or more quality of service (QoS) levels determined for the job. Job performance characterization, affinity models, computer resource power consumption, and policies may also be used to optimize the allocation of computing resources.

Intelligent allocation of resources in a computing system

A system and method for ranking computing resources in a distributed computing marketplace is disclosed. Ranking may be based on the performance factors that the system predicts will have the greatest impact on the particular application the user plans to run. A performance database stores historical performance data for applications that have been executed on multiple different computer systems. The database is checked to see if the application, or one similar, has already been run on any of the computing systems participating in the distributed computing marketplace. If so, the existing performance data is used to predict which performance factors will have the greatest impact on the application. Those factors are then used to rank the available computing systems options for the user.

Ranking computing resources

A method for configuring replicas in a distributed computing system is disclosed. In one example embodiment, a plurality of replicas with associated bootstrap modules may be created. The same bootstrap module code may be used for each replica, thereby simplifying configuration. Using the bootstrap module, the replicas may automatically configure themselves and self-assign a role for a set of predetermined roles such as master and worker. The bootstrap module may check a predetermined location such as a shared network folder for earlier registration entries and then self-select based on the remaining available roles. The bootstrap module may also store its own registration entry to inform subsequent replicas of the role and network address for the current replica so that they may self-configure correctly.

Autonomous organization and role selection of homogenous workers

Systems, methods, and apparatuses are disclosed for implementation and management of a network of computing clusters and interfaces. In various embodiment, a dynamic supercomputing resource marketplace system can include a cluster network having one or more interconnected computing clusters. The dynamic supercomputing resource marketplace system also can include a user interface system or an application program interface system for enabling a user to access the computing clusters. Advantageously, the dynamic supercomputing resource marketplace system can be used to facilitate increased utilization of computing clusters.

Dynamic network of supercomputing resources with unified management interface

Systems and methods for building applications by automatically incorporating application performance data into the application build process are disclosed. By capturing build settings and performance data from prior applications being executed on different computing systems such as bare metal and virtualized cloud instances, a performance database may be maintained and used to predict build settings that improve application performance (e.g., on a specific computing system or computing system configuration).

Build process for application performance

Systems, methods and apparatuses are disclosed for implementation and management of a dynamic compute and application marketplace. The dynamic computer marketplace system can coordinate access to one or more other computing resources, including on-premises computing resources, external (or off-premises) computing resources or a combination thereof. In various embodiments, the dynamic computer marketplace system advantageously can be used to facilitate inter-provider migration, transparent pricing, and/or competitive pricing, among other things.

Dynamic computer marketplace system and method

A system and method for processing application performance using application phase differentiation and detection is disclosed. Phase detection may be accomplished in a number of different ways, including by using a deterministic algorithm that looks for changes in the computing resource utilization patterns (as detected in the performance data collected). Machine learning (ML) and neural networks (e.g. sparse auto encoder SAE) may also be used. Performance data is aggregated according to phase and stored in a database along with additional application and computing system information. This database may then be used to find similar applications for performance prediction.

Job performance breakdown

Systems and apparatuses for provisioning computer services or resources and methods for making and using the same. In one embodiment, an exemplary method for performing an iterative search can include selecting a service from a group of available services for adjustment. An application associated with the selected service can be run, and an amount of resources consumed while the application is run can be captured. A provision level for the selected service, a provision type for the selected service or both can be adjusted based upon the captured amount of resources consumed. The method then can determine whether provision levels of the available services, provision types of the available services or both require further adjustment to be most performant. The approaches described herein advantageously can be applied, for example, to 'right-size' or 'scale' multiple resources.

System and method for scaling provisioned resources

A system and method for efficiently creating and managing application instances in distributed computing systems is disclosed. Controls are presented for specifying an application for instantiation, a data file for use with the application, and a destination for results from the application. Application resources and topology may be recommended to the user based on prior application execution, and CPU, GPU, and interconnect parameters such as bandwidth and latency. The controls may enable to user to customize the recommendations prior to automated instantiation based on the user's needs, such as whether the application is to be run in batch mode or interactive mode.

Instance creation in a computing system

A system and method for efficiently creating and managing application instances in distributed computing systems is disclosed. Controls are presented for specifying an application for instantiation, a data file for use with the application, and a destination for results from the application. Based on user input, graphical representations of available system resources in the distributed computing system are presented along with a logical topology for application instance creation to permit efficient user mapping of application primitives to available system resources. To further improve the user's ability to map, a hierarchical presentation of nodes illustrating resource and interconnection attributes may also be presented, and an error checking process may be performed to detect problems and offer suggested corrections based on application requirements and performance data collected from prior application executions.

Growing community of inventors

San Francisco, CA, United States of America

Max Alt