Amazon Technologies, Inc.

Nabanita Maji

Dumanshu Goyal

Sayantan Chakravorty

Vijaya Rama Reddy Kistampalli

Sahil Dorwat

Sankalp Bhatia

Nagarjuna Koduru

Gaurav Gupta

Gaurav Saxena

Mustafa Ozan Ozen

Sen Yue

Various embodiments of systems and methods for providing virtualized (e.g., serverless) broker clusters for a data streaming service are disclosed. A data streaming service uses a front-end proxy layer and a back-end broker layer to provide virtualized broker clusters, for example in a Kafka-based streaming service. Resources included in a virtualized broker cluster are monitored and automatically scaled-up, scaled-down, or re-balanced in a way that is transparent to data producing and/or data consuming clients of the data streaming service.

Data streaming service with virtualized broker clusters

Methods, systems, and computer-readable media for auto-scaling using temporal splits in a time-series database are disclosed. One or more stream processors of a time-series database write time-series data of a plurality of time series into a plurality of two-dimensional tiles, including an individual tile representing spatial boundaries and temporal boundaries. The heat of time-series data written to the individual tile exceeds a threshold. The current time is compared to the temporal boundaries, and the comparison indicates that the current time is beyond a threshold point within the temporal boundaries. Based at least in part on the comparison, a split is performed of the individual tile into a first new tile and a second new tile. The first new tile represents the spatial boundaries and a first portion of the temporal boundaries, and the second new tile represents the spatial boundaries and a second portion of the temporal boundaries.

Auto-scaling using temporal splits in a time-series database

Methods, systems, and computer-readable media for ingestion partition auto-scaling in a time-series database are disclosed. A first set of one or more hosts divides elements of time-series data into a plurality of partitions. A second set of one or more hosts stores the elements of time-series data from the plurality of partitions into one or more storage tiers of a time-series database. An analyzer receives first data indicative of the resource usage of the time-series data at the first set of one or more hosts. The analyzer receives second data indicative of the resource usage of the time-series data at the second set of one or more hosts. Based at least in part on analysis of the first data and the second data, the analyzer initiates a split of an individual one of the partitions into two or more partitions.

Ingestion partition auto-scaling in a time-series database

Growing community of inventors

Kenmore, WA, United States of America

Nabanita Maji