Amazon Technologies, Inc.

Jakub Wojciak

Petr Shveykin

Kelvin Sjek Yiu

Marc John Brooker

Cecilia Deng

Rishi Baldawa

Arijit Choudhury

Amin Suzani

Shunjia Ding

Shawn Patrick Jones

Rogerio Sathio Yamaguti

Somar Musa

James Lance Eather

Bernardo Vecchia Stein

Systems and processes are described for a message service with distributed key caching for server-side encryption. Message requests are received by message handlers of the message service that cache data encryption keys used to encrypt and decrypt messages that are stored to message containers in back end storage. A metadata service obtains the data encryption keys from a key management service, caches the keys locally, and sends the keys to the message handlers upon request, where the keys are cached, again. The key management service may generate the data encryption keys based on a master key (e.g., a client's master key). The message handlers may send both message data encrypted using the data encryption key and an encrypted copy of the data encryption key to be stored together in the data store.

Message service with distributed key caching for server-side encryption

A probabilistic data structure such as a bloom filter may be used to provide a space efficient representation of rules that define events of interest to sources and may be used to reduce the number of events ingested by an event bus system. An event bus system may generate a probabilistic data structure based on the rules that may be used at the event source as a filter. The probabilistic data structure predetermines whether the events to be sent to the event bus system will be of interest to any event targets and if an event is of no interest to any of the event targets, the event will not be sent.

Preemptive filtering of events of an event bus with a deterministic filter

A shard polling system fairly distributes stored items from producers to consumer processes and includes polling threads that poll for items from respective portions of a storage source, place the items in respective queues, and increment a global permit counter, restricted to a configurable maximum, that tracks the quantity of messages across the respective queues. The polling threads are restricted by respective shard permit counters that limit a quantity of items that may be moved from a storage source to a respective queue. A global thread applies a fairness technique, such as round robin, to fairly obtain the items from each of the respective queues and places the items in a global queue, decrementing the global permit counter, and incrementing the respective shard permit counter for each item obtained. Consumer threads poll the global queue (e.g., a blocking output queue) for the messages.

Sharded polling system

A message processing technology is provided that includes subscription of stateless compute functions to messaging queues of the message queueing service. The message queueing service may be configured to provide highly scalable hosted messaging queues for reliable delivery of messages. When messages are received in a messaging queue, then a stateless compute function, of a serverless compute service, that is subscribed to the messaging queue may be invoked to process the message. The serverless compute service manages compute resource for execution of the stateless compute function in response to dynamic message traffic.

Message processing using messaging services

Technologies are disclosed for providing a workload management service for first-in, first-out ('FIFO') queues in a network-accessible message queuing service within a service provider network. When a host is overloaded, or about to become overloaded, the affected host sends an alarm to the workload management service. The workload management service determines a set of other hosts that have lower workloads based upon performance metrics. The workload management service selects a new host from the set of hosts for moving a FIFO queue from the overloaded cluster. The workload management service seals the FIFO queue fragment at the overloaded host for writing of new messages and new messages having the same message group identifier are written to a new FIFO queue fragment on the new host. Messages are not read from the new FIFO queue fragment until all messages are read from the sealed FIFO queue fragment on the overloaded cluster.

Workload management service for first-in first-out queues for network-accessible queuing and messaging services

Growing community of inventors

Vancouver, Canada

Jakub Wojciak