Splunk Inc.

Yongxin Su

Ting Wang

John Robert Coates

Takuya Matsumoto

Guodong Wang

Gang Tao

Junqing Hao

Elias Haddad

Yiyun Zhu

Li Li

Ting Yuan

Qian Jie Zhong

Determining a set of extraction rules include clustering event segments into at least a first group of event segments, and determining, using first field data in the first group of event segments, a first set of extraction rules for extracting the first field data from each event segment of the first group of event segments. A determination is made that the first set of extraction rules fails to successfully extract all of the first field data. Responsive to the determination, the event segments are re-clustered into at least a second group of event segments and a third group of event segments until a successful set of extraction rules are identified. The successful set of extraction rules are stored in computer memory.

Extraction rule generation using clustering

The operation of an automatic data input and query system is controlled by well-defined control data. The system exposes user interfaces enabling an administrator to interact with control data to modify the ongoing operation of the system. Certain control data determines the collection and treatment of data from various technology sources. A robust control interface is provided enabling the efficient and reliable adding on of new technology data sources. Once established, control data for a new technology data source may be packaged in a form for archiving or distribution. The system may support the export and import of such packages. Such packages may be created independently of the system.

Technology add-on interface

Embodiments of the present invention are directed to enhancing extraction rules utilizing user feedback. In embodiments, a set of extraction rules relevant to an event set are provided for display. Thereafter, a selection of an extraction rule is received and, in response, a set of events matching the selected extraction rule is provided for display. A modification, for example provided by a user, in association with the extraction rule or the set of events is received. Such a modification is then used (e.g., via machine learning) to enhance extraction rules available for performing subsequent data extraction.

Enhancing extraction rules based on user feedback

Embodiments of the present invention are directed to validating extraction rules. In embodiments, a set of events for which field extraction is desired is obtained. Thereafter, an extraction rule is applied to the set of events to extract fields of the events. The application of the extraction rule can be monitored to determine that the applied extraction rule is invalid. Based on the applied extraction rule being invalid, a new extraction rule can be generated to apply to the set of events.

Inferring sourcetype based on match rates for rule packages

Technology add-on control console

The operation of an automatic data input and query system is controlled by well-defined control data. Certain control data may relate to data schemas and direct operations performed by the system to extract fields from machine data. Automatic methods may determine proper field extraction control information by analyzing a sample of data from a source, breaking the sample data into event segments, classifying the segments into groups based on a measure of similarity, determining an operable extraction rule for each group, and storing the resulting extraction model. Data patterns known by the system can be leveraged to perform the event breaking and field identification for the classifying. Embodiments may provide a user interface to view, interact with, and approve the computer-generated extraction model.

Field extraction rules from clustered data samples

Extraction rule validation

Growing community of inventors

Shanghai, China

Yongxin Su