Document architecture with efficient storage

Abstract

A document architecture system includes an intelligent document processing engine that can receive raw data streamed from external data source(s), analyze the raw data to identify triggering events, apply rules to the triggering events identified, break down the raw data accordingly, and generating messages in an internal format to represent the pieces of data without any formatting or layout information. The messages can be augmented with custom metadata and stored as fragments that reference, but that do not physically incorporate, the messages. The stored fragments can be used by the document process engine to intelligently render documents and versions thereof. Some embodiments store only unique fragments, resulting in a significant reduction in storage requirements and an extremely efficient way of processing huge amounts of data (e.g., millions of documents) for document production. The messages and fragments are in internal formats under control of the document architecture system.