International Business Machines Corporation

Other

Michael J Branson

John Matthew Santosuosso

Eric L Barsness

James E Carey

Bin Cao

Mathew Richard Odden

Lance Bragstad

Bradford Lee Cobb

Ryan Kirk Cradick

Daniel Beuch

Brandon William Schulz

Gregory Richard Hintermeister

Jay Symmes Bryant

Frederick Douglis

Zachary Adam Garbow

Alexander Cook

Steven Gene Halverson

Jason Allan Nikolai

Fan Ye

Andrew James Streit

Zhen Liu

Susette M Townsend

Bradley William Fawcett

William Robert Taylor

George Francis DeStefano

Gregory Scott Hurlebaus

David Henry Sandifer

Brian Robert Muras

Parijat Dube

Jim C Chen

David M Koster

Adam Thomas Stallman

James Franklin Macon, Jr

Candace Trielle Pederson

Todd Mark Kelsey

Erik Duane Lindberg

Scott Anthony Sylvester

Michael P Koranda

Paul William Wertzler

David Roy Limpert

Ronald Joseph Haugen

William Waller

Royan Herbert Bartley

David Alan Jaeger

Devaughn L Rackham

David Arlen Elko

Jeffrey Alan Frey

Michael D Rahn

John Elliott Arwe

Eric Howland Jenney

Georg Ochs

Patrick S Botz

Charles S Gauthier

Patrick Michael Wildt

Anthony Wayne Erwin

David Joseph Misheski

Bin Rong

Brian Richard Lucier

Jason E Tedor

Stephen Matthew Stupca

Brandon W Schultz

Raymond Severt Barsness

Neela Patel

Jaylene Branson

John M Santosuosso

A stream of tuples is processed by a stream application. The stream application includes a plurality of processing elements that operate on one or more compute nodes, each processing element includes one or more stream operators. One or more databases that are capable of communicating with the stream application are monitored during the processing of the stream of tuples. A potential performance condition of a first database of the one or more databases is detected based on the monitoring. An output adjustment is performed, in response to the potential performance condition.

Datastore aware streams computing

Information indicating a source of electric current can be transmitted alongside or within the electric current itself, enabling downstream recipients to identify a source of their electricity. The information may be embedded directly into the electric current, such as by adding a modulated carrier signal to an alternating current before transmitting the current to a downstream recipient.

Identification and selection of the source of electric power

A stream computing application may be configured to manage the flow of tuples through a section of an operator graph. A window may be generated over one or more stream operators. The window may include breakpoint thresholds that set the maximum flow of tuples within the window. The stream operators within the window may be monitored to determine the flow of tuples occurring within the window using tuple flow counts. The tuple flow counts may be compared to the breakpoint thresholds to determine whether a breakpoint condition has occurred. If a breakpoint condition has occurred, a tuple flow change may be implemented to reduce the flow of tuples within the window.

Windowing across operators in a streaming environment

Information indicating a source of electric current can be received alongside or detected within the electric current itself, enabling recipients to identify a source of their electricity. The information may be determined based on analysis of the received electric current, such as by detecting a modulated carrier signal embedded within a received alternating current.

Feedback tuples are implemented in a stream computing environment to provide data flow status information for use in detecting, identifying and/or handling data flow anomalies for a stream computing application.

Feedback tuples for detecting data flow anomalies in stream computing environment

An embodiment includes determining, in response to receiving data at a source node for transmission as streaming data on a network, whether a characteristic of the data indicates that an attribute of the data includes sensitive information. The embodiment also includes calculating, responsive to determining that the attribute includes sensitive information, an entropy value for the attribute. The embodiment also includes determining, responsive to the calculating of the entropy value, a likelihood that the attribute is encrypted by comparing the entropy value to a threshold value. The embodiment also includes generating a parameter value for a cryptographic parameter for the attribute.

Selective encryption of streaming data

Provided is a method for disabling encryption of data in motion in response to an event. The method includes a service processing data. The service may process the data while in a public mode, in which the service is configured to encrypt data in motion. The method further comprises detecting an event that triggers the service to go into a protected mode. The method further comprises isolating the service from one or more public systems in response to detecting the event. The method further comprises deactivating encryption of data in motion, and processing the data without encrypting the data while in motion.

Selective encryption of data in motion

An embodiment includes monitoring a distributed computing application at runtime for occurrence of a condition, where the condition includes occurrence of a degradation condition on a processing element (PE). The embodiment also includes starting a provisional PE on a second node at runtime while maintaining runtime operation of the PE. The embodiment compares operation of the PE to operation of the provisional PE based on a performance metric associated with the computing resource. The embodiment connects the provisional PE to replace the stream of tuples from the PE to a downstream PE with a stream of tuples from the provisional PE to the downstream PE.

Dynamic replacement of degrading processing elements in streaming applications

Certain aspects of the present disclosure provide techniques for identifying temporal data in data streams to create a temporal database for a stream(s) application to query for temporal data. An example technique includes receiving streams of data at a streams engine and processing the streams of data according to a priority order. The streams engine identifies whether the database is a temporal database and identifies temporal data in each stream of data based on frame analysis, natural language processing techniques, metadata, and optical character recognition. Further, the streams engine generates context data corresponding to the temporal data. The streams engine generates a temporal data record based on the temporal data and context data, and the streams engine generates a reliability factor. The temporal data record and reliability factor are stored in the temporal database for a stream application to query regarding temporal information at a later point in time.

Integrating relational database temporal tables with a distributed programming environment

In a streaming application receiving streaming data to be processed by a plurality of processing elements (PEs), one of a plurality of upstream PEs that reside on separate compute nodes and transmit data to a multi-input port PE is selected based on a data flow rate measured between at least two components in the plurality of PEs (e.g., an amount of incoming data per unit time or an amount of network bandwidth). At least one operator from the selected upstream PE is redeployed into the multi-input port PE. In some embodiments, the operator is selected from among a plurality of operators fused together in the selected upstream PE, un-fused from the selected upstream PE, and fused into the multi-input port PE. In some embodiments, a dynamic fusing capability is realized by selecting different ones of the plurality of upstream PEs in response to changes in the measured data flow rate.

Autonomic fusion changes based off data rates

Profiling data characterizing a data streaming application is used to predict data which will need to be retrieved by a processing element during execution of the data streaming application. Data is retrieved responsive to the prediction, in advance of actual demand by the processing element which requires it. Prediction may be based at least in part on upstream tuple contents, and could include other historical data retrieval patterns. In some embodiments, retrieval of predicted data may be delayed so that data is retrieved just in time.

Reducing flow delays in a data streaming application caused by lookup operations

The method can include receiving the first stream of tuples to be processed by the stream operator hosted by one or more computer processors, the stream operator having at least one input port and a condition input port. The method can also include receiving, at runtime, a customized condition at the condition input port of the stream operator. The method can also include monitoring a first group of one or more tuples from the first stream of tuples at a first window of the stream operator. The method can also include determining whether the first group of one or more tuples at the first window fulfills the customized condition. The method can also include deleting, in response to the first group of one or more tuples fulfilling the customized condition, the first group of one or more tuples.

Eviction stream for data joins

First and second simulated processing of a stream-based computing application using respective first and second simulation conditions may be performed. The first and second simulation conditions may specify first and second operator graph configurations. Each simulated processing may include inputting a stream of test tuples to the stream-based computing application, which may operate on one or more compute nodes. Each compute node may have one or more computer processors and a memory to store one or more processing elements. Each simulated processing may be monitored to determine one or more performance metrics. The first and second simulated processings may be sorted based on a first performance metric to identify a simulated processing having a first rank. An operator graph configuration associated with the simulated processing having the first rank may be selected if the first performance metric for the simulated processing having the first rank is within a processing constraint.

Selecting an operator graph configuration for a stream-based computing application

A method, system and computer program product for staggering a stream application's deployment, by: providing one or more configuration settings that define one or more delays for instantiation or initialization of at least one target processing element of the stream application based on the stream application's run-time conditions or events; and instantiating or initializing the target processing element of the stream application when the delays defined by the configuration settings have been satisfied. The configuration settings may define the delays to wait until a specified period of time has lapsed after one or more other processing elements have been instantiated or initialized, or to wait until a specified amount of data has been processed by one or more other processing elements, or to wait until one or more signals is received from one or more other processing elements, before instantiating or initializing the target processing element.

Staggering a stream application's deployment

Growing community of inventors

Rochester, MN, United States of America

Michael J Branson