Palantir Technologies Inc.

Joseph Rafidi

Parvathy Menon

Adam Borochoff

Ryan Rowe

Matthew MacLean

Matthew Jenny

Salar Al Khafaji

Tyler Uhlenkamp

Patrick Szmucer

James Thompson

Hao Zuo

Robert Fink

Jared Newman

Lynn Cuthriell

Adam Anderson

Catherine Lu

Karanveer Mohan

Michelle Guo

Andrew Elder

Daniel Patrick Cervelli

Adam Storr

Robert Kruszewski

Walker Burgin

Aditya Dahiya

Arthur Wu

Roger Chen

Matthew Bango

Mark Elliot

Timothy James Slatcher

Kamran Khan

Joseph Hashim

Vilmos Ioo

Rahij Ramsharan

Timothy Slatcher

Martin Copes

Stawomir Mucha

Felix-Gabriel Mance

Mohamed Zaki Trache

Morten Telling

Slawomir Mucha

Matthew Maclean

Computing systems methods, and non-transitory storage media are provided for retrieving information regarding an operation to be performed by a platform, performing a preliminary validation of the operation, generating details regarding the preliminary validation, transmitting at least a subset of the details of the preliminary validation to the platform, and populating the generated details on an interface. If the preliminary validation fails, the platform refrains from performing the operation. Furthermore, the logic describing the operation can be executed on different platforms and is not bound or limited to one platform.

Approaches of augmenting and streamlining task execution

A method of determining and displaying metadata that represents provenance of columns in a data store comprises receiving query data expressed in a human-readable language and describing one or more transformations of data tables of a distributed database system or columns of the data tables; parsing the query data to create a data structure for unresolved transforms in which the data tables are not matched with the columns; based in part upon table schema metadata describing the columns of the data tables, creating an additional data structure for resolved transforms, the additional data structure matching the data tables with the columns and specifying inputs and outputs of each transformation of the one or more transformations; based on information in the additional data structure, creating and causing display of a visual graph of the columns of the data tables and relationships between the columns, relating to the one or more transformations.

Generation and graphical display of data transform provenance metadata

Systems and methods are provided for improved graphical user interfaces. The system enables multiple separate applications, each of which may typically be in their own separate window or tab, to be interacted within a single window, such as a tab of a web browser application. The main window includes smaller sub-windows that can correspond to a distinct application with its own graphical user interface. A large sub-window within the main window is opened for the primary application where the user is currently interacting with a graphical user interface of the primary application. The user then is able to switch between applications (all within the same main window) and applications that are no longer being used can be minimized in smaller sub-windows off to the side of the primary sub-window. The system enables a user to drag and drop an item from one sub-window to another sub-window. Some of the interactions between the windows and data transformations are stored and can be visually presented in a graph.

Graphical user interface system

A method comprises storing first transaction data for a first transaction producing a first version of a first dataset, second transaction data for a second transaction transforming a first version of the first dataset to a second version of the first dataset, and third transaction data for a third transaction transforming the second version of the first dataset to a third version of the first dataset; storing dependency information indicating a first dependency of the third transaction on the second transaction and a second dependency of the second transaction on the first transaction; receiving a first instruction to revoke a first permission of a first user to access the second version of the first dataset; automatically revoking a certain permission of the first user to access the third version of the first dataset based on the dependency information.

Data revision control in large-scale data analytic systems

A resource dependency system may track data dependencies and data transformations for individual columns of the data sets over the span of the data pipeline (referred to as a provenance or lineage of a column). Column provenance/lineage can be logged using metadata or graph-like data structures, which the resource dependency system can generate, store, manage, and access. Column provenance/lineage can be used to generate user interfaces displaying visual node graphs with columns as nodes and the data dependencies and data transformations associated with the columns as edges between the nodes.

Column lineage for resource dependency system and graphical user interface

A resource dependency system and its associated user interfaces, used for tracking data dependencies and data transformations between resources, may display visual node graphs with resources as nodes and the data dependencies and data transformations associated with the columns as edges between the nodes. The nodes representing the resources may be displayed differently based on relevant differences in the resources they represent, which can be set through various selectable criteria and schemes. The user interfaces may include selectable options for visually arranging the nodes or grouping the nodes into superseding nodes according to how the nodes are displayed or the relevant differences in the resources they represent. Updated properties and data dependencies associated with superseding nodes can be presented to the user.

Resource grouping for resource dependency system and graphical user interface

Display screen or portion thereof with graphical user interface

A resource dependency system displays two dynamically interactive interfaces in a resource dependency user interface, a hierarchical resource repository and a dependency graph user interface. User interactions on each interface can dynamically update either interface. For example, a selection of a particular resource in the dependency graph user interface causes the system to update the dependency graph user interface to indicate the selection and also updates the hierarchical resource repository to navigate to the appropriate folder corresponding to the stored location of the selected resource. In another example, a selection of a particular resource in the hierarchical resource repository causes the system to update the hierarchical resource repository to indicate the selection and also updates the dependency graph user interface to display an updated graph, indicate the selection and, in some embodiments, focus on the selected resource by zooming into a portion of the graph.

Resource dependency system and graphical user interface

System and method for generating and displaying data pipelines according to certain embodiments. For example, a method includes: receiving a natural language (NL) query; receiving a model result generated based on the NL query, the model result including a query in a standard query language, the model result being generated using one or more computing models; and generating the data pipeline based at least in part on the query in the standard query language, the data pipeline comprising one or more data pipeline elements, at least one data pipeline element of the one or more pipeline elements being corresponding to a query component of the query in the standard query language.

Systems and methods for generating and displaying a data pipeline using a natural language query, and describing a data pipeline using natural language

Systems and methods are provided for creating and managing a data integration workspace. The workspace may comprise one or more views of data (or datasets) stored in or accessible by the system. Models may be generated and updated based on the plurality of datasets and presented via a graphical user interface. Feedback received via a graphical user interface presenting a model may be used to annotate an underlying dataset associated with the model. Responsive to a modification of the underlying dataset or the rules for using the underlying dataset to generate the model, other related datasets and/or models may be automatically updated accordingly. Templates associated with one or more types of users may be defined. Each template may comprise one or more specific models related to a specific type of user.

Systems and methods for creating and managing a data integration workspace

A method comprises creating metadata identifying columns of tables and column operations of one or more data transforms of the columns in a data pipeline and including links to code segments in human-readable form corresponding to the one or more data transforms; executing a build job that effects the one or more data transforms on one or more datasets to generate one or more derived datasets; causing, after the executing, a presentation of a graphical user interface (GUI) including a graphical representation of the one or more data transforms based on the metadata, wherein the method is performed by one or more processors.

A computer-implemented system and method for data revision control in a large-scale data analytic systems. In one embodiment, for example, a computer-implemented method comprises the operations of storing a first version of a dataset that is derived by executing a first version of driver program associated with the dataset; and storing a first build catalog entry comprising an identifier of the first version of the dataset and comprising an identifier of the first version of the driver program.

Techniques for propagation of deletion operations among a plurality of related datasets are described herein. In an embodiment, a data processing method comprises, using a distributed database system that is programmed to manage a plurality of different raw datasets and a plurality of derived datasets that have been derived from the raw datasets based on a plurality of derivation relationships that link the raw datasets to the derived datasets: from a first dataset that is stored in the distributed database system, determining a subset of records that are candidates for propagated deletion of specified data values; determining one or more particular raw datasets that contain the subset of records; deleting the specified data values from the particular raw datasets; based on the plurality of derivation relationships and the particular raw datasets, identifying one or more particular derived datasets that have been derived from the particular raw datasets; generating and executing a build of the one or more particular derived datasets to result in creating and storing the one or more particular derived datasets without the specified data values that were deleted from the particular raw datasets; repeating the generating and executing for all derived datasets that have derivation relationships to the particular raw datasets; wherein the method is performed using one or more processors.

Growing community of inventors

Mountain View, CA, United States of America

Joseph Rafidi