International Business Machines Corporation

Christoph Adrian Miksovic Czasch

Paolo Scotton

Thomas Gschwind

Andreas Schade

Richard Obinna Osuala

Katsiaryna Mirylenka

A method, computer system, and a computer program product for searching are provided. The method may include receiving a word and a context of the word. The context may include additional words. A first word embedding may be generated by inputting a sequence into a word embedding model that resultantly outputs the first word embedding. The sequence may include the word and the context that are concatenated to each other in the sequence. The first word embedding may be compared with other word embeddings. The other word embeddings may have been generated by inputting respective text portions of other texts into the word embedding model. A candidate match of the other texts may be presented. A respective word embedding of the candidate match may be, of the other word embeddings, most similar to the first word embedding according to the comparing.

Selection-based searching using concatenated word and context

A system and a method for determining a similarity between a first string and a second string. A sequence of edit operations are performed on the first string in order to obtain the second string may be determined. The edit operation is of a first type or a second type. The first type operation comprises a character insertion operation or character removal operation. The second type operation comprises a character maintenance operation. The first type edit operation is associated with an operation score indicative of a cost for applying the edit operation. The first type edit operation is associated with a switching score indicative whether it is immediately followed by a second type edit operation. The switching scores and/or operation scores associated with the sequence of edit operations are combined in order to obtain a combined score that is indicative of the similarity level between the first and second strings.

String similarity determination

The present disclosure relates to a method for enabling a processing of a dataset of records having a set of attributes. The method comprises: selecting a first attribute of the set of attributes and a subset of one or more second attributes of the set of attributes. Distinct values of the subset of second attributes may be determined from the dataset. For each distinct value of the determined distinct values records of the dataset that have said each distinct value may be identified, and a group of words may be formed from values of the first attribute of the identified records. Distinct word sequences may be identified in the formed groups and a level of presence of each word sequence of the word sequences in each of the formed groups may be determined. At least part of the levels of presence may be provided as metadata.

Determining metadata of a dataset

A computer-implemented method and a related system for record linkage of an incoming record to a reference data set may be provided. The method comprises providing a reference data set comprising a plurality of records, each record comprising a plurality of attributes. The method comprises further assigning each of the plurality of records an initial surrogate identifier value, assigning a plurality of block identifiers to each of the records by applying a locality sensitive hashing function to a predefined attribute of the records, resulting in the plurality of the block identifiers, and determining a final surrogate identifier value to each of the records assigned to one of the blocks such that the final surrogate identifier values in each block are uniformly distributed.

Multicriteria record linkage with surrogate blocking keys

A given query entity of a query database and a set of reference entities from a master database are accessed; each entity accessed corresponds to an entry in a respective database, which is mapped to a set of words that are decomposed into tokens. For each reference entity, a closest token is identified therein for each token of the given query entity, via a given string metric. A number of closest tokens are thus respectively associated with highest scores of similarity between tokens of the query entity and tokens of each reference entity. An entity similarity score is computed based on said highest scores. A reference entity of the master database is identified, which is closest to said given query entity, based on the entity similarity score. Records of the given query entity are linked to records of the master database, based on the closest reference entity identified.

Growing community of inventors

Aeugst am Albis, Switzerland

Christoph Adrian Miksovic Czasch