Databricks Inc.

Aaron Daniel Davidson

Clemens Mewald

Tomas Nykodym

Yandong Mao

Ka-Hing Cheung

Xinmei Gui

Sai Teja Pratap Reddy Suram

Jason Yongjoon Kim

Lixiang Ao

Anders Leif Christian Rex

Xinyang Ge

Haonan Jing

The present application discloses a method, system, and computer system for providing an integrated workspace file system. The method includes receive a receiving a request to access a first file, determining a first user associated with the request to access the first file, determining whether the first user is authorized to access the first file, and in response to determining that the first user is authorized to access the first file: generating a uniform resource identifier (URI) associated with the first file, wherein the URI comprises a credential for accessing the first file, wherein the credential is based on a user credential and a first file access authorization, and providing the URI to a user system.

Integrated workspace file system

The present application discloses a method, system, and computer system for starting up and maintaining a cluster in a warmed up state, and/or allocating clusters from a warmed up state. The method includes instantiating a set of virtual machines, wherein instantiating the set of virtual machines includes setting a temporary security credential for each virtual machine of the set of virtual machines, receiving a virtual machine allocation request associated with a workspace, a customer, or a tenant, in response to the virtual machine allocation request: allocating a virtual machine, wherein allocating the virtual machine comprises replacing the temporary security credential with a security credential associated with the workspace, the customer, or the tenant.

Reducing cluster start up time

A system performs efficient startup of executors of a distributed computing engine used for processing queries, for example, database queries. The system starts an executor node and processes a set of queries using the executor node to warm up the executor node. The system performs a checkpoint of the warmed-up executor node to create an image. The image is restored in the target executor nodes. The system may store a checkpoint image for each configuration of an executor node. The configuration is determined based on various factors including the hardware of the executor node, memory allocation of the processes, and so on. The user or restore based on checkpoint images improves efficiency of execution of the startup of executor nodes.

Checkpoint and restore based startup of executor nodes of a distributed computing engine for processing queries

A system includes an interface, a processor, and a memory. The interface is configured to receive a version of a model from a model registry. The processor is configured to store the version of the model, start a process running the version of the model, and update a proxy with version information associated with the version of the model, wherein the updated proxy indicates to redirect an indication to invoke the version of the model to the process. The memory is coupled to the processor and configured to provide the processor with instructions.

Model ML registry and model serving

Growing community of inventors

Berkeley, CA, United States of America

Aaron Daniel Davidson