International Business Machines Corporation

Nissan Motor Company Limited

Zexel Gmbh

Mikio Takeuchi

Kiyokuni Kawachiya

Kinichiro Nakano

Michihiro Horie

Hiroshi Horii

Motoki Hirano

Gaku Yamamoto

Tomotaka Kurozu

Takahisa Tomoda

Hisashi Miyashita

Hideaki Komatsu

Akira Koseki

Kazunori Ogata

Hiroyuki Nomura

Teruo Kawasaki

Yuki Makino

Ryoh Neyama

Yohsuke Ozawa

Haruo Mochida

Tamiya Onodera

Kazuaki Ishizaki

Moriyoshi Ohara

Yasushi Negishi

Takeshi Ogasawara

Hideki Tai

Takeo Kushida

Tatsuhiro Chiba

Yoshimitsu Takeda

Takeshi Yoshimura

Tohru Futami

Hiroshi Yano

Kenichi Mitamura

Yasuo Hamano

Issei Ohmura

Tamiya Nodera

Salikh S Zakirov

Takesji Ogasawara

Rina Inoue

Hiroshi Kato

Embodiments are disclosed for a method. The method includes determining that a thief thread attempted a work steal from a garbage collection (GC) owner queue. Additionally, the method includes determining that a number of tasks in the GC owner queue meets a predetermined threshold. Further, the method includes determining that the GC owner queue comprises a heavy-weight task. The method also includes moving the heavy-weight task to a top position of the GC owner queue.

Reducing load balancing work stealing

Methods and systems for traffic jam management include generating a representation of a road network that comprises a road graph, with vertices of the road graph representing road segments and edges of the road graph representing pairs of adjacent road segments at each intersection. The road graph is partitioned into sub-graphs to balance a number of messages received from vehicles within each sub-graph and to minimize vehicle transitions between sub-graphs. It is determined that a traffic jam is present on one or more road segments. Navigational information is provided to one or more vehicles responsive to the traffic jam.

Vehicle traffic information analysis and traffic jam management

A computer-implemented method is presented for reducing data loading overhead of middleware to facilitate direct data loading to a database management system (DBMS). The method includes, while data loading, sending external-writes to a memory-based DBMS if corresponding internal-writes are for vertices, exporting all the external-writes to a disk-based DBMS as an export file, and sending an external-read for vertices to an in-memory DBMS if the middleware requests data. The method further includes, at the end of data loading, generating files for raw data of the disk-based DBMS from the export file and sending the generated raw files to the disk-based DBMS.

Direct data loading of middleware-generated records

Systems and methods for processing hierarchical tasks in a garbage collection mechanism are provided. The method includes determining chunks in a task queue. Each chunk is a group of child tasks created after processing one task. The method includes popping, by an owner thread, tasks from a top side of the task queue pointed at by a chunk in a first in first out (FIFO) pop. The method also includes stealing, by a thief thread, tasks from a chunk in an opposite side of the task queue.

Garbage collection work stealing mechanism

A computer-implemented method for increasing record availability on a distributed database system implementing multiple partitioning strategies includes identifying first processing devices of a distributed database system for storing one or more first records in accordance with multiple partitioning strategies, determining that at least two of the first processing devices are identified for storing a given one of the one or more first records, and selecting one or more new processing devices of the distributed database system for storing the given first record to achieve sufficiently high availability of the given record within the distributed database system.

Record replication for multi-column partitioning on distributed database systems

A method is provided for consistent data processing by first and second distributed processing systems having different data partitioning and routing mechanisms such that the first system is without states and the second system is with states. The method includes dividing data in each system into a same number of partitions based on a same key and a same hash function. The method includes mapping partitions between the systems in a one-to-one mapping. The mapping step includes calculating a partition ID based on the hash function and a total number of partitions, and dynamically mapping a partition in the first system to a partition in the second system, responsive to the partition in the first system being unmapped to the partition in the second system.

Integrating multiple distributed data processing servers with different data partitioning and routing mechanisms, resource sharing policies and lifecycles into a single process

Methods and systems for performing garbage collection include issuing a memory fence that indicates that a number of tasks in a garbage collection deque, belonging to a first garbage collection thread, has decreased by more than one. A length of the garbage collection deque, after the memory fence is issued, is determined to be greater than zero. Multiple tasks are popped from the garbage collection deque responsive to the determination that the length of the garbage collection deque is greater than zero. Garbage collection is performed on the popped tasks.

Garbage collection work stealing with multiple-task popping

A computer-implemented method is provided for deleting a given object from among a plurality of objects in an object-oriented programming language computing system which uses a Reference Count (RC) of each of the plurality of objects to check a liveness of the plurality of objects. The method includes decrementing, in a Reference Counts (RCs) decrement operation, RCs of objects referenced from the given object using one or more non-atomic operations in a transaction that utilizes a hardware transactional memory mechanism to accelerate the reference counts decrement operation.

Accelerating reference count processing in object deletion

A method is provided for consistent data processing by first and second distributed processing systems having different data partitioning and routing mechanisms such that the first system is without states and the second system is with states. The method includes dividing data in each system into a same number of partitions based on a same key and a same hash function. The method includes mapping partitions between the systems in a one-to-one mapping. The mapping step includes (i) checking if a partition in the first system is mapped to a partition in the second system; and (ii) calculating a partition ID based on the hash function and a total number of partitions, and dynamically mapping the partition in the first system to the partition in the second system, responsive to the partition in the first system being unmapped to the partition in the second system.

Described herein are methods of transferring arrays of data information by remote data memory access (RDMA). The method may include identifying data arrays in a local place that are to be copied to a remote place; and determining whether the data arrays are to be overwritten by analyzing asynchronous blocks from the data arrays in the local place at a start compilation time using a static compiler. The method may further include executing transfer of the data arrays from the local place to the remote place with a pull type RDMA.

Remote direct memory access-based on static analysis of asynchronous blocks

Methods are provided for copying, from a local place to a remote place, only array variables from a set at the local place that are used at the remote place. A method includes identifying the array based on the array meeting a condition of being defined outside of a code block for a copy command and used inside of the code block. The copy command mandates a copying of all the array variables in the set from the local place to the remote place. The method includes generating a function having code that selectively serializes or deserializes only the array variables from the set that are used at the remote place. The method includes serializing, at the local place, and deserializing at the remote place, only the array variables from the set that are used at the remote place, by invoking the function.

Reducing cost of deserialization by skipping unused array regions

A method for copying and transferring data by remote data memory access resulting in faster data transfer speeds is provided. The method includes determining an amount of garbage data of data information, and comparing the amount of garbage data to a threshold amount. In response to the amount of garbage data being below the threshold amount, a first portion of data that includes intended data of the data information and the garbage data is copied to a remote place. In response to the amount of garbage data exceeding threshold amount, a second portion of data that includes only the intended data is copied to the remote place by a scatter/gather input/output method of a remote data memory access protocol. The copied data is returned to a free list of the remote place.

Remote direct memory access-based method of transferring arrays of objects including garbage data

Methods and systems for optimizing an application include optimizing, with a processor on a first device, an application for a second device in accordance with an application execution profile received from the second device to generate a binary for the application that is optimized for use indicated by the application execution profile. The optimized binary is set to be a default application binary, to be sent to devices requesting the application for a first time, if a percentage of matching application profiles exceeds a threshold. The optimized binary for the application is transmitted to the second device.

Profile-based per-device code optimization

Described herein are methods of transferring arrays of data information by remote data memory access (RDMA). According to the method, data information in data arrays in a local place identified. The intended data and garbage data are identified. Contiguous intended data and garbage data are copied to a remote place when the amount of garbage data is below a threshold garbage data criteria, and only intended data is copied to the remote place when the amount of garbage data is above the threshold garbage data criteria amount. Copying relies on scatter/gather input/output of RDMA. The garbage data transferred to the remote place is forwarded to a free list of the remote place.

Growing community of inventors

Kanagawa, Japan

Mikio Takeuchi