Methods and apparatus for storing and manipulating variable length and fixed length data elements as a sequence of fixed length integers

Abstract

Apparatus for storing and processing a plurality of data items each comprising supplied data values organized in one or more fields each of which stores typed data. Character strings and natural language text are converted to numerical token values in an array of fixed length integers and other forms of typed data (real numbers, dates, times, boolean values, etc.) are also converted to integer form and stored in the array. Stored metadata specifies the data type of all data in the integer array to enable each integer to be rapidly accessed and interpreted. When fixed length data types are present, the metadata specifies location, size and type of each fixed length element. When variable length data is stored in the integer array, size and location data stored in the integer array is accessed to rapidly and directly access the variable size data. The presence of implicit or explicit size information for each data structure, including variable size structures, speeds processing by eliminating the need to scan the data for delimiters, and by reducing the processing needed to perform memory allocation, data movement, lookup operations and data addressing functions. Data stored in the integer array is subdivided into items, and items are subdivided into fields. Items may be organized into more complex data structures, such as relational tables, hierarchical object structures, linked lists and trees, and the like, using special fields called links which identify other referenced items.