Methods and apparatus for functional model-based data provenance in stream processing environments

Abstract

Techniques for deriving a provenance of one or more of a plurality of output data elements generated from a given output port of a PC are provided. At least one dependency function is created that relates the one or more output data elements to a set of one or more input ports of the PC and a corresponding plurality of input data elements. The dependency function comprises an encoding of at least one of one or more temporal filters and one or more sequence filters relating to the plurality of input data elements. The at least one dependency function is stored. A history of stream-level bindings of one or more input streams to one or more input ports of the processing component and one or more output streams from one or more output ports of the processing component is stored. The plurality of input data elements belonging to the one or more input streams and the plurality of output data elements belonging to the one or more output streams are stored. The set of one or more input data elements from the plurality of input data elements are determined that relate to the one or more output data elements in accordance with at least one dependency function and the history of stream-level bindings.