International Business Machines Corporation

Adobe, Inc.

Sonali Surange

Stephen Andrew Brodsky

Rebecca Braun Nin

Azadeh Ahadian

Zeus Orion Courtois

Brian Gerrit Payton

Tom William Jacopi

Anshul Dawra

Michael Leon Pauser

Tony Leung

Michael George Burke

Tony Kai-chi Leung

Igor Peshansky

Ardeshir Jamshidi

Michael Yiupun Kwong

Todd Robert Bender

Clifford D Chu

Hongdi Zhang

Ryan A Rossi

Eunyee Koh

Roger C Raphael

Rajesh M Desai

Fan Du

Maryela Evelin Weihrauch

Atanu Ranjan Sinha

Mukund Raghavachari

Shunan Guo

Vasanthi Holtcamp

Jane Elizabeth Hoffswell

Saurabh Mahapatra

Daya Vivek

John Anderson

Michael Schenker

Arpit Ajay Narechania

Hani Talal Jamjoon

Vasantha Jayakumar

Embodiments provide systems, methods, and computer storage media for management, assessment, navigation, and/or discovery of data based on data quality, consumption, and/or utility metrics. Data may be assessed using attribute-level and/or record-level metrics that quantify data: 'quality'—the condition of data (e.g., presence of incorrect or incomplete values), its “consumption”—the tracked usage of data in downstream applications (e.g., utilization of attributes in dashboard widgets or customer segmentation rules), and/or its “utility”—a quantifiable impact resulting from the consumption of data (e.g., revenue or number of visits resulting from marketing campaigns that use particular datasets, storage costs of data). This data assessment may be performed at different stages of a data intake, preparation, and/or modeling lifecycle. For example, an interactive tree view may visually represent a nested attribute schema and attribute quality or consumption metrics to facilitate discovery of bad data before ingesting into a data lake.

Interactive tree representing attribute quality or consumption metrics for data ingestion and other applications

A method adapts a dataflow instance. A set of source data nodes, a set of terminal data nodes, and a set of computation nodes in the dataflow are identified from a directed graph. The set of computation nodes performs operations on data flowing from the set of source data nodes through computation nodes and onwards to terminal data nodes in the dataflow. The data nodes are evaluated with policies and a user context. A number of transformation compute nodes is computed from the policy decisions and added downstream of the set of source data nodes and optionally upstream of the set of terminal data nodes when the data, the dataflow or system does not meet the declared policies without the necessary computed number of transformation compute nodes. The number of transformation compute nodes are an additional portion of the overall set of computation nodes to enforce the declared policy.

Enforcing policy in dataflows

Provided are techniques for invoking with a processor executing on a computer a source code parser to obtain source information that includes a first location of an Application Programming Interface (API) call and parameters of the API call in source code of a client application, where the parameters the API call do not include query text for a query that is to be used to access a database; examining a stack trace to determine a second location of the API call in the stack trace; and deriving the query of the API call and a third location of the query in the source code by identifying the query in the stack trace at the location of the API call in the stack trace.

Correlating queries issued by applications with their source lines and analyzing applications for problem determination and where used analysis

Embodiments of the invention provide database integration within an integrated development environment (IDE) tool for displaying database structure and other database information, query editing and execution, and error detection for database statements embedded in program source code. Embodiments of the invention integrate database connectivity and error detection into the IDE tool, thereby potentially reducing the time required for database-aware application development.

Processing database queries embedded in application source code from within integrated development environment tool

Database connectivity and database model integration within integrated development environment tool

Provided are techniques for collecting client information for client applications running on a client computer, wherein the client information includes a client application identifier for each client application that issues queries to a database along with text of each of the queries that the client application issues; collecting database monitoring information that includes text of each query issued against the database and performance information for each query; combining the client information and the database monitoring information based on the text of the queries to obtain combined information that provides the client application identifier and the performance information for each of the queries; and identifying a problem query and at least one client application that has issued the problem query using the combined information.

Client and database problem determination and monitoring

Development tools are disclosed that connect various artifacts in a database aware fashion. For example, the development environment may integrate SQL query builders across multiple application development interfaces (e.g., a source code editor, XML editor, database scripting tools, etc.), significantly improving developer productivity, reducing development cost and time. Thus, the process of consuming heterogeneous artifacts in an application (or creating them for consumption) becomes a very productive experience, eliminating the need to manually cut/paste information in multiple project files in many cases.

Development environment configured to generate application source code from database objects

Embodiments of the invention introduce development tools that connect various artifacts in a database aware fashion. For example, embodiments of the invention integrate SQL query builders across multiple application development interfaces (e.g., a source code editor, XML editor, database scripting tools, etc.), significantly improving developer productivity, reducing development cost and time. Thus, the process of consuming heterogeneous artifacts in an application (or creating them for consumption) becomes a very productive experience, eliminating the need to manually cut/paste information in multiple project files in many cases.

Development environment for managing database aware software projects

Detecting and displaying errors in database statements within integrated development environment tool

A technique is disclosed for identifying impact of database changes on an application. Given a proposed database change, affected source code is identified that is affected by the proposed database change. References to the affected source code are created and organized into a hierarchy. A user can ascertain the extent of the impact by viewing the hierarchy and by using the hierarchy to access and view the affected source code in a visually distinct manner.

Identifying impact of database changes on an application

Embodiments of the invention provide application development tools that allow developers to rapidly build database-aware applications and database unit tests. Embodiments of the invention support multiple scenarios for database-aware application development, including beginning from a database table and automatically creating application code to access the table, beginning from an existing database query, beginning from existing application code that accesses a database, and hybrids or variations of these approaches.

Rapid application development for database-aware applications

Embodiments of the invention provide an IDE tool that provides rapid application development features for a variety of disparate data sources. Further, developers may customize and extend the rapid application development features to suit their development needs in a particular case. In a particular embodiment, the rapid application development template may be configured to generate a set of configuration files localized used to deploy a 'lightweight' application, such as a web-service, to a particular application server.

Extensible rapid application development for disparate data sources

Growing community of inventors

San Rafael, CA, United States of America

Sonali Surange