The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Nov. 06, 2018
Filed:
Nov. 18, 2015
Linkedin Corporation, Mountain View, CA (US);
Lin Qiao, San Jose, CA (US);
Yinan Li, Los Gatos, CA (US);
Sahil Takiar, San Francisco, CA (US);
Ziyang Liu, San Mateo, CA (US);
Narasimha R. Veeramreddy, San Jose, CA (US);
Min Tu, Cupertino, CA (US);
Ying Dai, Mountain View, CA (US);
Issac Buenrostro, Sunnyvale, CA (US);
Kapil L. Surlaker, Sunnyvale, CA (US);
Shirshanka Das, San Jose, CA (US);
Chavdar Botev, Sunnyvale, CA (US);
Kenneth D. Goodhope, San Jose, CA (US);
Microsoft Technology Licensing, LLC, Redmond, WA (US);
Abstract
In order to ingest data from an arbitrary source in a set of sources, a computer system accesses predefined configuration instructions. Then, the computer system generates a dynamic data-ingestion pipeline that is compatible with a Hadoop file system based on the predefined configuration instructions. This dynamic data-ingestion pipeline includes a modular arrangement of operators from a set of operators that includes: an extraction operator for extracting the data of interest from the source, a converter operator for transforming the data, and a quality-checker operator for checking the transformed data. Moreover, the computer system receives the data from the source. Next, the computer system processes the data using the dynamic data-ingestion pipeline as the data is received without storing the data in memory for the purpose of subsequent ingestion processing.