Red Hat, Inc.

Giuseppe Scrivano

Daniel J Walsh

Michael Tsirkin

Colin G Walters

Andrea Arcangeli

Alexander Larsson

Pierre-Yves Chibon

Sergio Lopez Pascual

Mrunal Patel

Embodiments of the present disclosure provide a hybrid approach to performing a lazy pull of a container image. A file system in user space (FUSE) is utilized to lazy pull the container image, and manage file requests from the container while the container image is being fetched locally. During the retrieving, the FUSE may receive from the container, one or more file requests, and may temporarily block each of the one or more file requests until it can process them. Once the container image is fully fetched locally, the overlay structure of the container image expected by a file system in the kernel (e.g., Overlay FS) is created and control is passed to the file system in the kernel. The FUSE may then unmount itself, to expose the container to the underlying mount point.

Hybrid approach to performing a lazy pull of container images

A computing device receives a container image from a container registry, the container image comprising a plurality of layers. The computing device extracts, from the container image, a plurality of files into a filesystem of the computing device, each file of the plurality of files corresponding to one of the layers of the plurality of layers. The computing device generates a metadata file that describes a directory structure of the plurality of files in the filesystem. The computing device mounts the metadata file to a location in the filesystem, wherein mounting the metadata file causes a digest of the metadata file to be generated and the metadata file and the plurality of files in the filesystem to be read-only.

Validation of container images

Systems and methods for duplication avoidance are disclosed. In one implementation, a VM can receive a request to perform a file access operation with respect to a file and determine a hash value corresponding to a content of the file. The VM can search the file identified by the hash value in in a host file system. Responsive to failing to find the hash value in the host file system, the VM can search the hash value in a guest file system of the VM and responsive to finding the file identified by the hash value in the guest file system, can perform the file access operation with respect to the file.

Virtual machine storage system for duplication avoidance

Embodiments of the present disclosure provide a substitute audit log for use by applications in the user-space of a host operating system to write audit information. When a container makes a system call attempting to write audit information to an audit log of the kernel, the kernel may utilize a predefined set of instructions indicating how to detect such system calls and how such system calls are to be modified so as to reroute the system call to an unprivileged socket. The kernel write the audit information of the system call to an unprivileged socket that is connected to a substitute audit log. A container management program monitoring the unprivileged socket may write the audit information to the substitute log which is defined in container specific directories of the container.

Containers system auditing through system call emulation

A computing device executing a kernel-based operating system creates a content-addressed object store containing files of a filesystem. The computing device generates a metadata file that describes characteristics of the files of the filesystem that are in the content-addressed object store. The computing device, in a kernel of the operating system, generates a Merkle tree of the metadata file and a Merkle tree of the content-addressed object store. The Merkle trees contain hashes of blocks of the metadata file and hashes of blocks of the files in the content-addressed object store. The computing device, in the kernel, verifies that a digest corresponding to the metadata file matches an expected digest for the metadata file and that digests corresponding to the files in the content-addressed object store match expected digests for the files in the content-addressed object store. The computing device mounts the metadata file to a location in the filesystem.

Using a custom filesystem to secure the contents of a filesystem

A sparse files aware rolling checksum is provided by passing, in sequence, each byte of an archival file to a hash function; and in response to: detecting that a sequence of bytes from the archival file produce outputs from the hash function of zero, wherein a number of bytes in the sequence of bytes satisfies a chunk-end threshold, and determining that the sequence of bytes is located in a hole in the archival file of a greater number of bytes than the chunk-end threshold: designating a hole-chunk of the archival file that includes metadata for a location and a length of the hole in the archival file.

Sparse files aware rolling checksum

A method includes receiving, at a container registry, a container image including a set of files, determining that at least one file of the set of files is a duplicate of a previously stored container image file at the container registry. The method further includes storing the container image at the container registry with the at least one file of the plurality of files that is a duplicate of the previously stored container image file removed from the container image.

Deduplication of container image files

Methods, systems, and techniques are disclosed for managing replication in a deployable object, such as a pod (e.g., a group of one or more containers). For example, when a pod is started in a cluster, the deployable object may start one or more virtual computer systems (e.g., containers), which may pull (e.g., initiate and run) container images from a registry server. The processing device may thus identify, such as on the file level, which container image should be pulled first. A scheduler of the one or more virtual computer systems may prioritize, based on the computed scores, a subsequent replication of archived data of the one or more virtual computer systems to be performed. The processing device may then execute, based on the prioritization by the scheduler, the subsequent replication of the archived data of the one or more virtual computer systems in the deployable object.

Chunk aware image locality scores for container images in multi-node clusters

Systems and methods for deploying and running webassembly workloads on compute nodes using a webassembly unikernel. A computing device having a processor generates, using a container engine, a container runtime. The computing device generates, using the container runtime, a unikernel configured to run a plurality of webassembly applications, thereby resulting in a webassembly unikernel. The computing device may receive a request to run an application. A container image corresponding to the application may be retrieved, via the container engine, from a container registry. A webassembly payload may be stored for the application from the container image. Furthermore, the computing device may cause the webassembly unikernel to run the webassembly payload for the application.

Unikernel for webassembly containers

Systems and methods providing efficient dirty memory page expiration. In one implementation, a processing device may identify a storage device. The processing device may determine a value of an indicator associated with the storage device. The indicator may indicate a level of consistency between a volatile memory device and a non-volatile memory device of the storage device. In view of the value of the indicator, the processing device may modify a synchronization timeout value associated with the volatile memory device.

Efficient dirty page expiration

A method includes receiving a system call from an application within a container executing on an operating system, the system call comprising a synchronization operation to synchronize memory of the application to storage. The method further includes determining, by the kernel, whether a system call filtering policy associated with the container indicates that the system call is to be prevented. preventing, by the kernel, performance of the synchronization operation in view of the system call filtering policy.

Speedup build container data access via system call filtering

A method includes receiving, an operation from a container to synchronize container data from memory to a file system mounted by the container and determining whether the file system indicates that the operation is to be ignored. The method further includes, in response to determining that the file system indicates that the operation is to be ignored, preventing, by the operating system kernel executing on the processing device, performance of the operation.

Speedup containers in production by ignoring sync to file system

An example system includes a memory, a processor in communication with the memory, and a supervisor. The supervisor is configured to allocate a memory space in the memory to a workload executing on the processor. The supervisor is configured to store data written by the workload as dirty memory in the memory space at least until the data is written back to a data storage. Based on a type of the workload being a first type, the supervisor is configured to trigger write back of at least a portion of the dirty memory into the data storage in response to the dirty memory exceeding a threshold level. Based on the type of the workload being a second type, the supervisor is configured to delay write back of the dirty memory into the data storage in response to the dirty memory exceeding the threshold level.

Writeback overhead reduction for workloads

Systems, methods, and machine-readable instructions stored on machine-readable media are disclosed for copying a first permission of a file to a second permission of the file, wherein the file is stored on a host file system. The first permission is changed to a third permission. A request is received to access the file from a container file system. In response to the request and before providing the container file system with access to the file, changing the third permission to the second permission. The file is provided to the container file system based on the second permission.

Differentiated file permissions for container users

A method includes, with a computing system, storing a first container image. The first container image may be defined by a first set of layers. The method further includes, with the computing system, storing a second container image, the second container image defined by a second set of layers. The second set of layers has at least one file in common with a file in the first set of layers. The method further includes, with the computing system, performing an analysis of the first set of layers and the second set of layers. The method further includes, with the computing system, using the analysis to create a new set of layers such that both the first container image and the second container image can be derived from the new set of layers.

Growing community of inventors

Spezzano della Sila, Italy

Giuseppe Scrivano