Palantir Technologies Inc.

Mingyu Kim

Mark Elliot

Robert Fink

Jared Newman

Michael Nazario

Kevin Chen

Lynn Cuthriell

Jakob Juelich

Divyanshu Arora

Matthew Cheah

Punyashloka Biswal

Andrew Colombi

Juan Tamayo

David Tolnay

Akshay Pundle

Justin Uang

Paul Nepywoda

Yupeng Fu

Ashar Fuadi

A computer-implemented method comprises obtaining a first build task for building first source code in a first programming language of a plurality of programming languages; retrieving, by the processor, the first source code based on the first build task; building the first source code into one or more artifacts and one or more job specifications; storing the one or more artifacts in a cache shared across a cluster; and initializing an application module on the cluster based on the first programming language, the application module configured to receive a job specification of the one or more job specifications and execute a data transformation job using a reference to a location in the cache.

Extensible data transformation authoring and validation system

Data transformation in a distributed system of applications and data repositories is described. The subsystems for the overall framework are distributed, thereby allowing for customization to require only isolated changes to one or more subsystems. In one embodiment, a source code repository is used to receive and store source code. A build subsystem can retrieve source code from the source code repository and build it, using one or more criteria. By building the source code, the build subsystem can generate an artifact, which is executable code, such as a JAR or SQL file. Likewise, by building the source code, the build subsystem can generate one or more job specifications for executing the executable code. In one embodiment, the artifact and job specification may be used to launch an application server in a cluster. The application server can then receive data transformation instructions and execute the data transformation instructions.

Techniques related to a domain-specific language for transformations are disclosed. A server computer may process a table definition composed in a domain-specific language. The table definition may include a sequence of one or more transformations to be performed on one or more source tables to generate a target table. The sequence may include a customized transformation. A source dataset may be provided as input to an implementation of the customized transformation. An output dataset may be generated as a result of executing the implementation. An intermediate table may be generated based on performing at least one transformation on a particular source table. A supplemental portion for the intermediate table may be generated based on performing the at least one transformation on an appended portion of the particular source table. The target table may be generated based on combining the supplemental portion with the intermediate table.

Domain-specific language for dataset transformations

Techniques related to a domain-specific language for dataset transformations are disclosed. A server computer may process a table definition composed in a domain-specific language. The table definition may include a sequence of one or more dataset transformations to be performed on one or more source tables to generate a target table. The sequence may include a customized transformation. A source dataset may be provided as input to an implementation of the customized transformation. An output dataset may be generated as a result of executing the implementation. An intermediate table may be generated based on performing at least one dataset transformation on a particular source table. A supplemental portion for the intermediate table may be generated based on performing the at least one dataset transformation on an appended portion of the particular source table. The target table may be generated based on combining the supplemental portion with the intermediate table.

Growing community of inventors

Palo Alto, CA, United States of America

Mingyu Kim