Oracle International Corporation

Philip Ogren

Luis E Rivas

Alexander Sasha Stojanovic

Mark L Kreider

Edward A Green

Michael Malak

Kevin L Markey

Stephen Joseph Green

Glenn Allen Murray

Robert James Oberbreckling

A natural language identity classifier system is described, which employs a supervised machine learning (ML) model to perform language identity classification on input text. The ML model takes, as input, non-lexicalized features of target text derived from subword tokenization of the text. Specifically, these non-lexicalized features are generated based on statistics determined for tokens identified for the input text. According to an embodiment, at least some of the non-lexicalized features are based on natural language-specific summary statistics that indicate how often tokens were found within a corpus for each natural language. Use of such summary statistics allows for generation of natural language specific conditional probability-based features. Because of the inherent interpretability of a trained non-lexicalized ML model as described herein, it is possible to modify behavior of the trained ML model by adjusting summary statistics maintained for natural language tokens and/or by adjusting data for the subword tokenizers.

Non-lexicalized features for language identity classification using subword tokenization

The present disclosure describes techniques for entity classification and data enrichment of data sets. A data enrichment system is disclosed that can extract, repair, and enrich datasets, resulting in more precise entity resolution and classification for purposes of subsequent indexing and clustering. Disclosed techniques may include performing entity recognition to identify segments of interest that relate to an entity. Related data may be analyzed for classification, which can be used to transform the data for enrichment to its users.

Automated entity correlation and classification across heterogeneous datasets

The present disclosure relates generally to a data enrichment service that extracts, repairs, and enriches datasets, resulting in more precise entity resolution and correlation for purposes of subsequent indexing and clustering. As the data enrichment service can include a visual recommendation engine and language for performing large-scale data preparation, repair, and enrichment of heterogeneous datasets. This enables the user to select and see how the recommended enrichments (e.g., transformations and repairs) will affect the user's data and make adjustments as needed. The data enrichment service can receive feedback from users through a user interface and can filter recommendations based on the user feedback.

Declarative language and visualization system for recommended data transformations and repairs

Techniques are disclosed for standardization of data. According to a first technique, standard representation terms are determined for to-be-standardized data using the to-be-standardized data itself and without using any external reference data. According to a second technique, a combination of the to-be-standardized data and an external reference is used to determine standard representation terms for the to-be-standardized data.

Data standardization techniques

Embodiments perform string similarity analysis by receiving candidate strings of a collection to be searched and transforming each candidate string into one or more features. Embodiments generate a feature index that maps each of the features to one or more candidate strings which include the features, and transform the feature index into a low-memory index by byte encoding a sorted integer array into an encoded byte array. The transforming into a low-memory index further includes hashing each feature to an integer value and storing using a tightly-packed map a hashed value to feature identification ('ID') mapping, using a first tightly-packed list to map each of the feature IDs to one or more candidate strings which include the features, and using a second tightly-packed list to store the feature IDs.

Memory reduced string similarity analysis

A string analysis tool for calculating a similarity metric between an input string and a plurality of strings in a collection to be searched. The string analysis tool may include optimizations that may reduce the number of calculations to be carried out when calculating the similarity metric for large volumes of data. In this regard, the string analysis tool may represent strings as features. As such, analysis may be performed relative to features (e.g., of either the input string or plurality of strings to be searched) such that features from the strings may be eliminated from consideration when identifying candidate strings from the collection for which a similarity metric is to be calculated. The elimination of features may be based on a minimum similarity metric threshold, wherein features that are incapable of contributing to a similarity metric above the minimum similarity metric threshold are eliminated from consideration.

Scalable approach to information-theoretic string similarity using a guaranteed rank threshold

Computer-based tools and methods for conversion of data from a first form to a second form without reference to the context of data to be converted. The conversion may be facilitated by matching source data with external information (e.g., public and/or private schema) that contain rules (e.g., context specific rules) for conversion of the data. The matching may be performed based on an optimized index string matching technique that may be operable to match source data to external information that is context dependent without specific identification of the context of either the source data or the external information identified. Accordingly, the conversion of data may be performed in an unsupervised machine learning environment.

Contextually blind data conversion using indexed string matching

A string analysis tool for calculating a similarity metric between a source string and a plurality of target strings. The string analysis tool may include optimizations that may reduce the number of calculations to be carried out when calculating the similarity metric for large volumes of data. In this regard, the string analysis tool may represent strings as features. As such, analysis may be performed relative to features (e.g., of either the source string or plurality of target strings) such that features from the strings may be eliminated from consideration when identifying target strings for which a similarity metric is to be calculated. The elimination of features may be based on a minimum similarity metric threshold, wherein features that are incapable of contributing to a similarity metric above the minimum similarity metric threshold are eliminated from consideration.

Growing community of inventors

Boulder, CO, United States of America

Philip Ogren