Palantir Technologies Inc.

Palanter Technologies Inc.

Robert Fink

Mark Elliot

Jared Newman

Lynn Cuthriell

James Thompson

Allen Cai

Mark Schafer

Marvin Sum

John Chakerian

Joseph Ellis

Sandor Van Wassenhove

Parvathy Menon

Adam Borochoff

Joseph Rafidi

Michael Nazario

Ryan Rowe

Matthew MacLean

Kevin Chen

Adam Anderson

Matthew Jenny

Amr Al Mallah

Jakob Juelich

Divyanshu Arora

Mingyu Kim

Haithem Turki

Karanveer Mohan

Catherine Lu

Michelle Guo

Matthew Cheah

Benjamin Duffield

Thomas Powell

Lawrence Manning

Harkirat Singh

Thomas Pearson

Samuel Rogerson

Jasjit Grewal

Viktor Nordling

Justin Uang

Felix De Souza

Nitish Kulkarni

Graham Dennis

Huw Pryce

Audrey Kuan

Paul Nepywoda

Daniel Fox

James Neale

Andrew Kaier

Corey Garvey

Patrick Koenig

Artur Jonkisz

Greg DeArment

Punyashloka Biswal

Eric A Lee

Ashwin Ramaswamy

Bradley Moylan

Ryan McNamara

James Ross

James Baker

Andrew Colombi

William J Wappler

Grace Wang

Alexander Ince-Cushman

Charles Edwin Dressler

Eric C Lee

Tushar Narayan

Michael James Dressler

David Lee Smith, Jr

Jimin Song

Ryan Mcnamara

Eric Lee

A method comprises receiving, from one or more search nodes of a distributed system, one or more requests for log data, the one or more search nodes being associated with one or more first storage systems; identifying an indexed portion of the log data stored in a second storage system of one or more second storage systems based on a particular request of the one or more requests, the log data being indexed by one or more indexing nodes independently from the one or more requests being received and processed by the one or more search nodes; changing a first number of the one or more search nodes and updating a second number of the one or more indexing nodes independently.

Indexing and relaying data to hot storage

A method comprises storing first transaction data for a first transaction producing a first version of a first dataset, second transaction data for a second transaction transforming a first version of the first dataset to a second version of the first dataset, and third transaction data for a third transaction transforming the second version of the first dataset to a third version of the first dataset; storing dependency information indicating a first dependency of the third transaction on the second transaction and a second dependency of the second transaction on the first transaction; receiving a first instruction to revoke a first permission of a first user to access the second version of the first dataset; automatically revoking a certain permission of the first user to access the third version of the first dataset based on the dependency information.

Data revision control in large-scale data analytic systems

A method comprises receiving, at a build service of a build server, an external dataset and an adaptor application module, the external dataset being in a specific format, the adaptor application module providing information relevant to a build pipeline maintained by the build service for building an output dataset based on the external dataset, the information including changes to the external dataset since a previous build of the output dataset is performed and a data schema used in the previous build, the build pipeline involving data only in one or more formats other than the specific format; incorporating the external dataset into the build pipeline without the external dataset being reformatted in accordance with requirements of the build service; receiving a request from the adaptor application module for specific information relating to a most recent data build run by the build service; providing a response to the adaptor application module.

Data analytic systems

Systems and methods are provided for data migration. The system may comprise one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the system to migrate at least one first table of a first database schema to at least one second table of a second database schema, determine a query for modifying the first table during the migration, modify the second table based at least in part on the query, and update a mutation table to describe the modification, wherein the mutation table at least describes the modification.

System and methods for live data migration

A computer-implemented method comprises obtaining a first build task for building first source code in a first programming language of a plurality of programming languages; retrieving, by the processor, the first source code based on the first build task; building the first source code into one or more artifacts and one or more job specifications; storing the one or more artifacts in a cache shared across a cluster; and initializing an application module on the cluster based on the first programming language, the application module configured to receive a job specification of the one or more job specifications and execute a data transformation job using a reference to a location in the cache.

Extensible data transformation authoring and validation system

Disclosed herein are methods and systems for producing and processing transportation and shipping documents. Embodiments thereof effectively utilize a combination of accurate data collection of vehicle loaded packaging/parts departing a location at, e.g., an exit portal of dock, creation of an electronic BOL populated with that accurate data and other shipping information, which can launch an efficient over the road capability from initial journey to destination with enhanced visibility throughout.

System and method for producing and processing transportation and shipping documentation

A method is disclosed. The method comprises receiving, from one or more search nodes of a distributed system, one or more requests for log data, the one or more search nodes being associated with one or more hot storage systems; identifying, from an index catalog, an indexed portion of the log data stored in a cold storage system of one or more cold storage systems based on at least part of the particular request, the index catalog containing pointers to indexed portions of the log data in the one or more cold storage systems, the indexing being performed by one or more indexing nodes independently from the receiving by the one or more search nodes; and sending the indexed portion to the one or more search nodes for storage in the associated one or more hot storage systems, wherein the method is performed using one or more processors.

A computer-implemented system and method for data revision control in a large-scale data analytic systems. In one embodiment, for example, a computer-implemented method comprises the operations of storing a first version of a dataset that is derived by executing a first version of driver program associated with the dataset; and storing a first build catalog entry comprising an identifier of the first version of the dataset and comprising an identifier of the first version of the driver program.

An apparatus, and a method, performed by one or more processors are disclosed. The method receives a build request associated with performing an external data processing task on a first data set, the first data set being stored in memory associated with a data processing platform to be performed at a system external to the data processing platform. The method generates a task identifier for the data processing task, and providing, in association with the task identifier, the first data set to an agent associated with the external system with an indication of the data processing task, the agent being arranged to cause performance of the task at the external system, to receive a second data set resulting from performance of the task, and to provide the second data set and associated metadata indicative of the transformation. The method receives the second data set and metadata from the agent associated with the external system and stores the second data set and associated metadata.

Providing external access to a processing platform

Disclosed herein is a data structure which includes a sequence of events, each event associated with a sequence number indicating a temporal position of an event within the sequence of events; one or more read-offsets, each read-offset associated with a consumer, wherein each read-offset indicates a sequence number up to which a consumer has read events within the sequence of events; and at least one snapshot which represents events with sequence numbers smaller than the smallest read-offset in a compacted form. Disclosed herein is also a computer-implemented method of maintaining the data structure. Disclosed herein is a computer-implemented method, wherein the method is performed on a sequence of events accessible by a plurality of consumers, each event associated with a sequence number indicating a temporal position of an event within the sequence of events, each consumer associated with a read-offset indicating the sequence number up to which the consumer has read events within the sequence of events, the method includes determining a smallest read-offset of all read-offsets; compacting events with sequence numbers smaller than the smallest read-offset into a snapshot; and replacing the events with sequence numbers smaller than the smallest read-offset with the snapshot. Disclosed herein are corresponding computer-readable media and computing systems.

Data structure based on event compaction and read-offsets

A method is disclosed, comprising receiving, at a build service, an external dataset and an adaptor associated with the external dataset, the external dataset being in a format which is not managed by the build server. The method may also comprise querying the adaptor, by the build service, for information related to the external dataset. The method may also comprise receiving, at the build service, the information related to the external dataset; and causing, by the build service, the external dataset to be input into a build pipeline using the information related to the external dataset.

One or more processors examine source code of one or more software packages that produce output messages and identify, in the source code, one or more call expressions that each represent a logging call. The one or more processors generate a number of search patterns for parsing output messages produced by the one or more software packages, wherein each of the search patterns is based on one or more arguments of a corresponding call expression of the one or more call expressions. The one or more processors further reduce the number of search patterns to be applied to the output messages produced by the one or more software packages to identify log entries among the output messages.

Automatic configuration of logging infrastructure for software deployments using source code

A method, performed by one or more processors, is disclosed, the method comprising receiving a stream of log data from one or more applications and indexing a plurality of different portions of the received stream to respective locations of a cold storage system. The method may also comprise storing, in an index, catalog pointers to the respective locations of the indexed portions in the cold storage system. One or more requests for log data may be received, and the method may also comprise subsequently identifying from the index catalog one or more pointers to respective indexed portions appropriate to at least part of the one or more requests, and sending of the identified one or more indexed portions to one or more hot storage systems each associated with a respective search node for processing of one or more search requests.

Computer-implemented techniques for data extraction are described. The techniques include a method and system for retrieving an extraction job specification, wherein the extraction job specification comprises a source repository identifier that identifies a source repository comprising a plurality of data records; a data recipient identifier that identifies a data recipient; and a schedule that indicates a timing of when to retrieve the plurality of data records. The method and system further include retrieving the plurality of data records from the source repository based on the schedule, creating an extraction transaction from the plurality of data records, wherein the extraction transaction comprises a subset of the plurality of data records and metadata, and sending the extraction transaction to the data recipient.

Techniques for data extraction

Growing community of inventors

San Francisco, CA, United States of America

Robert Fink