International Business Machines Corporation

Obidul Islam

Mohammadreza Barouni Ebrahimi

Iosif Viorel Onut

Samaneh Bayat

Paul Ionescu

Eugenia Kondratova

Adi Sharabani

Nevon C Brake

Vishwanath Ramaswamy

A computer-implemented method identifies strings of data from a database. One or more processors receive data as an input string. The processor(s) generate a first binary code using a binary locality sensitive hashing of k-grams in the input string, where the binary locality sensitive hashing on the k-grams in the input string is derived from a first set of bi-grams in the input string, a second set of bi-grams in the input string, and a quantity of intersecting bi-grams from the first set of bi-grams and the second set of bi-grams. In response to receiving a search request for a particular string, the processor(s) generate a second binary code using a binary locality sensitive hashing on the particular string, and search a database in a query process. The processor(s) then rank and return a set of similar strings found in the database.

Matching strings in a large relational database

A method prevents source code from using an unsecure application program interface (API) due to a change to the source code. One or more processors and/or a user identify changes in source code. The processor(s) and/or user associate the changed code with affected APIs that are used by the changed code, and identify which commands are utilized by the changed code. The processor(s) and/or user determine that at least one command utilized by the changed code causes the source code to use an unsecure API. In response to determining that the at least one command utilized by the changed code causes the source code to use the unsecure API, the processor(s) and/or user modify the changed code to prevent the source code from using the unsecure API.

Security scanning of application program interfaces that are affected by changes to source code

Identifying unvisited portions of visited information to visit includes receiving information to crawl, wherein the information is representative of one of web based information and non-web based information, computing a locality sensitive hash (LSH) value for the received information, and identifying a most similar information visited thus far. Identifying unvisited portions of visited information further includes determining whether the LSH of the received information is equivalent to most similar information visited thus far and, responsive to a determination that the LSH of the received information is not equivalent to most similar information visited thus far, identifying a visited portion of the received information using information for most similar information visited thus far and crawling only unvisited portions of the received information.

Identifying unvisited portions of visited information

A computer-implemented method searches a database for a particular string. One or more processors receive data as an input string, and then identify multiple k-grams in, unique characters in, and a length of the input string. The one or more processors perform binary locality sensitive hashing on the k-grams, the unique characters, and the length for the input string, and then sum the binary locality sensitive hashings to create a first addition vector, which is used to generate a first binary vector. The same process is performed on a particular string being requested to generate a second binary vector. The one or more processors then search the database for the particular string that was requested using the second binary vector in a large scale hamming distance query process that determines a hamming distance between the first binary code and the second binary code.

A computer-implemented method improves the security of a system. One or more processors identify changes in source code, resulting in changed code in the source code. The one or more processors associate the changed code with affected application program interfaces (APIs) that are used by the changed code, and then manipulate parameters within the affected APIs based on the changes in the source code. The one or more processors execute a dynamic security scan on only the affected APIs using the manipulated parameters, in order to identify security vulnerabilities in only the affected APIs. The one or more processors then modify the source code to remove the security vulnerabilities in only the affected APIs, thus improving the security of the system on which the source code is running.

A computer program product for crawling URLs that are encoded and highly dynamic, the computer program product includes a non-transitory computer readable storage medium having computer readable program code embodied therewith. The computer readable program code includes computer readable program code configured to retrieve navigational state information corresponding to a URL and compare the navigational state information to previously stored navigational state information corresponding to one or more previously visited URLs. The computer readable program code also includes computer readable program code configured to determine if the URL has been previously visited and retrieve content associated with the URL if the URL has not been previously visited.

Automatic crawling of encoded dynamic URLs

A method includes retrieving navigational state information corresponding to a URL; analyzing which navigational state to consider for page identification; comparing the navigational state information to previously stored navigational state information corresponding to one or more previously visited URLs; determining if the URL has been previously visited; and retrieving content associated with the URL if the URL has not been previously visited.

A processor determines whether a DOM includes a repetitive pattern of a combination, formed by a tag of a leaf node and a tag of a parent node of the leaf node. Determining the repetitive pattern of the combination, the processor identifies a first inner cluster is identified by collapsing multiple instances of the repetitive pattern into a single instance. The processor generates a LSH signature for the single instance of the repetitive pattern. The processor determines an outer cluster, based on grouping one or more inner clusters, as part of a section rooted at a source node of the DOM, in which the source node is a parent node of the one or more inner clusters. Determining that a pair of outer clusters are near repetitive, the processor limits web content exploration to one of the pair of outer clusters.

Growing community of inventors

Kanata, Canada

Obidul Islam