Hitachi, Ltd.

Itaru Nishizawa

Tsuneyuki Imaki

Toshihiko Kashiyama

Shinji Fujiwara

Kazutomo Ushijima

Nobutoshi Sagawa

Shigekazu Inohara

Takahiko Shintani

Naoki Watanabe

Akira Shimizu

Mamoru Sugie

Frederico Buchholz Maciel

Aki Tomita

Hiroaki Odawara

Shinichi Tanaka

Norihiko Moriwaki

Nobuo Sato

Yoshimasa Masuoka

Akinori Asahara

Masashi Egi

Daisuke Ito

Kazuo Masai

Michiko Tanaka

Yori Takahashi

Atsuro Handa

Masanori Kawashima

Satoshi Torikai

A network interface receives stream data and a processor processes the stream data. The processor: constructs an operator graph comprising information regarding a plurality of operators for processing the stream data based on a query; assigns an operator execution order to the plurality of operators based on an inter-operator input and output relation, with respect to all of the operators on the operator graph; extracts a set of the operators that maintains the stream data assigned with a time stamp, a time of which is earlier than or equal to a reference time, in an input queue; and executes processing of the stream data assigned with the time stamp, having the earliest time, in the operator assigned a minimum operator graph execution order among the operators and maintaining the stream data assigned with the time stamp having the earliest time, among the extracted set of the operators.

Method for processing stream data and system thereof

Provided is a stream data processing method cooperable with external data in real time. By providing a stream predict cache corresponding to a hierarchical cache of relation data predicted as being a data matching target with stream data, the number of accesses to a database is reduced to improve performance. Data to be stored in the stream predict cache is determined based on a characteristic of the stream data input by a user or monitoring information obtained by monitoring the stream data.

Stream data processing method cooperable with reference external data

In executing query distributed processing in a computer system having a plurality of computers, a query and a sliding window are inherited between different computers. If a tuple having a new principal key value is detected when a query is to be executed, a relevant individual query and a relevant sliding window are inherited between computers, and further a sliding window of a relevant common query is inherited between computers.

Method and computing system for distributed stream data processing using plural of computers

The present invention provides a method for processing stream data and a system thereof capable of implementing general data processing including recursive processing with low latency. In the system for processing stream data, a single operator graph is prepared from operator trees of a plurality of queries, an execution order of the operators is determined so that execution of a streaming operator is progressed in one way from an input to an output, the ignition time of an external ignition operator that inputs data from the outside of the system and an internal ignition operator that time-limitedly generates data is monitored, and an operator execution control unit repeats processing that completes the processing in the operator graph at the time according to the determined operator execution order, assuming the operator of the earliest ignition time as a start point.

A mechanism for managing ranking information using a sign of a stream tuple generated when stream data is inserted into, or deleted from, a window is provided. A mechanism for generating only the differential information of ranking calculation results, a mechanism for adding ranking information according to a request, an interface for generating and outputting all ranking information from the differential information, a mechanism for generating all ranking calculation results, and an interface for using these mechanisms are provided.

Ranking query processing method for stream data and stream data processing system having ranking query processing mechanism

Voice of plural participants during a meeting is obtained and dialogue situations of the participants that change every second are displayed in real time, so that it is possible to provide a meeting visualization system for triggering more positive discussions. Voice data collected from plural voice collecting units associated with plural participants is processed by a voice processing server to extract speech information. The speech information is sequentially input to an aggregation server. A query process is performed for the speech information by a stream data processing unit of the aggregation server, so that activity data such as the accumulation value of speeches of the participants in the meeting is generated. A display processing unit visualizes and displays dialogue situations of the participants by using the sizes of circles and the thicknesses of lines on the basis of the activity data.

Meeting visualization system

In a stream data processing method in which part of stream data is defined as a processing target and the time control information which indicates the advance of the time is inserted into the stream data, time information of the received stream data is stored in a next ignition time holding area as an ignition time. A processing module for generating the time control information at a time different from the time of reception of the stream data is extracted out of a query that indicates how the stream data is to be processed. Based on the extracted processing module and the time information indicating the time of the reception of the stream data, the ignition time is calculated and stored in the next ignition time holding area. The time generation module inserts the time control information at the ignition time held in the next ignition time holding area.

Machine-readable medium for storing a stream data processing program and computer system

Stream data is provided for realizing precise calculation processing to keep a constant memory usage of a query including a time-based window, and to take all of input data into consideration. A stream data processing server sections the time-based window into sub-windows each with a smaller width by a query time resolution change unit, and a query processing engine executes the aggregation processing based on the sub-window when receiving the stream data to generate an aggregation tuple, and calculates the calculation result of the query including the time-based window by aggregation processing on the aggregation tuple.

Stream data processing method and system

Provided is a stream data processing control method of controlling a stream data processing apparatus for receiving data added with time information in time series, and processing the data with a registered query therein, the stream data processing apparatus including the query for processing the data, a window in which the received data is stored, and control management information including a control code for identifying the content of the controlling, the method including the steps of: creating data including the control code based on the control management information; determining whether or not the data stored in the window includes the control code in a case where the received data is stored in the window; and controlling the stream data processing apparatus based on the control code included in the data if the data stored in the window includes the control code.

Stream data processing control method, stream data processing apparatus, and stream data processing control program

A stream data processing system capable of highly reliable and highly usable real time data processing, is provided with: (1) a mechanism of copying some or all of stream data, and archiving the stream data in a non-volatile memory to allow real time data and archive data to be used seamlessly; and (2) a mechanism of improving the performance of query processings by using a plurality of stream data processing systems cooperatively.

Query processing method for stream data processing systems

In order to make real time and the avoidance of risk compatible according to the contents of processing by an application, an index ID applying module applies an index computed utilizing streams before and after stream data to the stream data, computes the index again according to a stream that varies every moment, applies an index computed based upon the index of the stream data to the result of the processing of the stream data according to a predetermined procedure and outputs it. An application that receives the result of the processing and the index of the result of the processing switches processing depending upon the index.

Stream data processing method and stream data processing system

Provided is an effective index processing method for a key corresponding to characteristics of a key series. The index processing method includes: holding a key tendency for representing the characteristics of the key series and a node split ratio, corresponding to the key tendency, for representing a key split ratio at the time of node split; and switching the node split ratio of an index () based on the key tendency. The key tendency/distribution is determined based on characteristic information of data input by a user or monitoring information acquired through monitoring of the data.

Index processing method and computer systems

Provided is a stream data processing method that can effectively handle delay data. In the stream data processing method of processing data whose lifetime is defined by a window, an operation result excluding a delay tuple is immediately output along with an unconfirmed flag according to delay processing HBT while a midway processing result necessary for reproduction is retained along with the lifetime, and when the delay tuple arrives, a correct processing result is calculated from the delay tuple and the processing result restore tuple.

Stream data processing method and computer systems

A stream data processing system restrains the generation of communication traffic in the case of distributing loads by using a plurality of information processing devices. The global query is divided into a plurality of processes. At every divided process, a local query which includes the corresponding process is generated and each local query is set to a query processing device different from each other. In addition, the stream data is transmitted among a plurality of query processing devices in order so as to execute the local query in each query processing device. At this time, the entire stream data does not transmitted among the query processing devices in order. At least a part of the corresponding stream data is stored in a proxy device. When data received from an information processing device located in immediately before does not include the target data of the local query, the query processing device reads the corresponding data from the proxy device and executes the local query.

Growing community of inventors

Koganei, Japan

Itaru Nishizawa