International Business Machines Corporation

Naama Zwerdling

Boaz Carmeli

Naama Tepper

Ateret Anaby Tavor

George Kour

Esther Goldbraich

David Carmel

Elad Yom-Tov

Inbal Ronen

Amir Kantor

Sivan Yogev

Segev Shlomov

Erel Uziel

Balancing an imbalanced dataset, by: Receiving a balancing policy and the imbalanced dataset. Performing initial adjustment of the imbalanced dataset to comply with the balancing policy, by: oversampling one or more underrepresented classes, and, if one or more of the classes are overrepresented, undersampling them. Operating a generative machine learning model to generate samples for the one or more underrepresented classes, based on the initially-adjusted dataset. Operating a machine learning classification model to label the generated samples with class labels corresponding to the one or more underrepresented classes. Selecting some of the generated samples which, according to the labeling, have a relatively high probability of preserving their class labels. Composing a balanced dataset which complies with the balancing policy and comprises: the samples belonging to the one or more underrepresented classes, the selected generated samples, and an undersampling of the samples belonging to the one or more overrepresented classes.

Dataset balancing via quality-controlled sample generation

Embodiments of the present systems and methods may provide techniques for augmenting textual data that may be used for textual classification tasks. Embodiments of such techniques may provide the capability to synthesize labeled data to improve text classification tasks. Embodiments may be specifically useful when only a small amount of data is available, and provide improved performance in such cases. For example, in an embodiment, a method implemented in a computer system may comprise a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor, and the method may comprise fine-tuning a language model using a training dataset, synthesizing a plurality of samples using the fine-tuned language model, filtering the plurality of synthesized samples, and generating an augmented training dataset comprising the training dataset and the filtered plurality of synthesized sentences.

Language-model-based data augmentation method for textual classification tasks with little data

A method and system for providing relationships in search results are provided. The method includes indexing an entity in a search index as an entity index entry, the entity index entry including facets providing information on the entity type and related entities. Search results are obtained by a search engine in the form of ranked result entities, wherein the result entities include multiple types of entities. The method then includes retrieving index entries to determine relationships between search result entities and providing the relationships in search results. The method further includes, for each result entity, retrieving its entity index entry and cross-checking the facets of the retrieved entity index entry for other result entities.

Method and system for providing relationships in search results

A system and method including a simulator operating in conjunction with a search-engine, for improving document and site findability. Users input their content (pages or sites) and the simulator will analyze the site in terms of structure and content. It will then give the user a ranked list of suggestions about how the user might improve his/her site's findability. The user will then be able to apply some or all of these suggestions, or any other changes, by virtually modifying the site, and then immediately receive feedback both on how the pages look and a sense of the degree of findability improvement. The interactive process allows users to simulate modifications in their site structure and content in order to improve its findability. When the user completes the modifications and is satisfied with the new findability of his site, the user will be able then to replace his/her current site in the repository with the modified one.

Content analysis simulator for improving site findability in information retrieval systems

Growing community of inventors

Pardes Hana, Israel

Naama Zwerdling