Sap Se

Frank Koehler

Wolfgang Lehner

David Kernert

Embodiments relate to storing sparse matrices in an in-memory column-oriented database system. Specifically, recent hardware shifts of primary storage from disc into memory, allow execution of linear algebra queries directly in the database engine. Dynamic matrix manipulation operations (like online insertion or deletion of elements) are not covered by most linear algebra frameworks. Therefore a hybrid architecture comprises a read-optimized main structure, and a write-optimized delta structure. The resulting system layout derived from the Compressed Sparse Row (CSR) representation, integrates well with a columnar database design. Moreover, the resulting architecture is amenable to a wide range of non-numerical use cases when dictionary encoding is used. Performance in specific examples is evaluated for dynamic sparse matrix workloads, by applying work flows of nuclear science and network graphs. Embodiments allow performing linear algebra operations on large, sparse matrices commonly associated with scientific computations and analytical business applications.

Sparse linear algebra in column-oriented in-memory database

According to some embodiments, matrix A data may be loaded into a temporary, unordered starting representation that contains coordinates and values for each element of matrix A. Z-curve ordering of matrix A may be performed to create a two-dimensional density map of matrix A by counting matrix elements that are contained in logical two-dimensional block cells of a given size. A quad-tree recursion may be executed on the two-dimensional density map structure in reduced Z-space to identify areas of different densities in the two dimensional matrix space. An adaptive tile matrix representation of input matrix A may then be created. According to some embodiments, an adaptive tile matrix multiplication operation may perform dynamic tile-granular optimization based on density estimates and a cost model.

Growing community of inventors

Heidelberg, Germany

Frank Koehler