International Business Machines Corporation

Ibm Canada Limited - Ibm Canada Limitee

Gregor Von Bochmann

Iosif Viorel Onut

Guy-Vincent Jourdan

Paul Ionescu

Seyed M Mir Taheri

Mustafa Emre Dincturk

Wayne Duncan Smith

Nevon C Brake

Khalil A Ayoub

Suryakant Choudhary

Omer Tripp

Kamara Akili Benjamin

Seyed Ali Moosavi Byooki

Aviv Ron

Ori Segal

Russell L Couturier

Qian Cui

Avishay Bartik

Seyed M Taheri

Nevon Christopher Brake

Seyed Ali Byooki

Sayed M Taheri

Khalil Andres Ayoub

A computer-implemented method and/or computer program product selectively assigns a task using a hybrid task assignment process. One or more processors direct a working hardware node in a network to crawl a particular application, thus causing the working hardware node to encounter a task in the particular application. The processor(s) selectively handle the task according to whether the task is reserved for dynamic assignment to an other hardware node in the network, such that in response to a determination that the task encountered is not reserved, the task is handled by the working hardware node that is crawling the particular application, and in response to a determination that the task encountered is reserved, the task encountered is sent to a central unit for dynamic assignment to the other hardware node in the network.

Hybrid task assignment for web crawling

A method for identifying client states, receives a set of paths representative of a document object model (DOM) associated with a web page of a rich internet application and for each path in the set of paths received, extracts a subtree, as subtree X, for a current path. The method traverses all known sub-paths under the current path and delete corresponding subtrees from subtree X and reads contents of and determines states of subtree X to form a state X. The state X is added to a set of current states and responsive to a determination no more paths exist, returns the set of current states of the rich internet application.

Identifying client states

Identifying equivalent JavaScript events includes receiving source code containing two JavaScript events for equivalency analysis, extracting an HTML element containing an event from each JavaScript event and analyzing the extracted HTML elements. Responsive to a determination that the HTML elements are of a same type according to equivalency criteria B, and responsive to a determination that the HTML elements have a same number of attributes according to equivalency criteria C, a determination is made whether JavaScript function calls of each JavaScript event are similar according to equivalency criteria A. Responsive to a determination that the JavaScript function calls are similar according to equivalency criteria A, and responsive to a determination that the other attributes of the HTML elements satisfy equivalency criteria D, the JavaScript events are identified as equivalent.

Identifying equivalent javascript events

An example operation may include one or more of identifying a page of a website for phishing testing, attempting each of a Hypertext Transfer Protocol (HTTP) GET request and a HTTP Secure (HTTPS) GET request via the identified page of the website, attempting each of a HTTP POST request and a HTTPS POST request via the identified page of the website, determining if the website is a phishing website based on server responses to the attempted HTTP and HTTPS GET requests and the attempted HTTP and HTTPS POST requests received from the website, and in response to determining the website is a phishing website, outputting an indication of the determination for display on a display device.

Phishing detection through secure testing implementation

A computer-implemented method for hybrid task assignment is presented. A working hardware node crawls a particular application and encounters a task. A mapping function is used to determine whether the task encountered is reserved. In response to a determination the task encountered is not reserved, the task is handled by the working node, and in response to a determination the task encountered is reserved, the task encountered is sent to a central unit. A determination is made as to whether the working node is idle. In response to a determination the working node is idle, another task is requested from the central unit by the working node. In response to a determination the working node is not idle, as determination is made as to whether all tasks are complete. In response to a determination all tasks are not complete, the task is handled by the working node.

Hybrid task assignment

Identifying equivalent JavaScript events

An embodiment for tracking JavaScript actions in a rich Internet application, receives a document object model (DOM) representative of a particular page of an application at a particular time and analyzes the DOM received to identify each JavaScript action on the particular page for which each JavaScript action identified, a JavaScript action characteristics ID is calculated and stored. Responsive to a determination multiple instances of a same ID exist, collecting a list of JavaScript actions corresponding to each ID corresponding to a multiple JavaScript action and removing from memory JavaScript action entries for the multiple instances of the same ID. A neighbor influence is computed for a member of the list of JavaScript actions remaining and the JavaScript action ID calculated for the member of the list of JavaScript actions remaining is stored. Responsive to a determination there are no more multiple JavaScript actions, return all JavaScript action IDs stored.

Tracking JavaScript actions

Embodiments are directed to computing, by an apparatus comprising a processing device, an identifier (ID) for an element in a page using an algorithm responsive to encountering the element a first time, causing, by the apparatus, the ID to be stored in a storage device, encountering, by the apparatus, the element a second time, determining, by the apparatus, that the element is encountered the second time, and responsive to determining that the element is encountered the second time, representing, by the apparatus, the element by the ID at the time of the second encounter by retrieving the ID from the storage device.

Representation of an element in a page via an identifier

An illustrative embodiment of a computer-implemented process for partitioning a crawling space computes an event identifier for each event in the set of events to form an identified set of events, segments the identified set of events into a number of partitions, assigns a partition to each node in a set of nodes and executes each event in each assigned partition by a respective node. In response to a determination that a new state is discovered, other nodes are notified of the new state, in which information associated with the new state is added to a respective assigned set of event IDs at each node. In response to a determination that no more notifications exist, the computer-implemented process determines whether more events to process exist and terminates in response to a determination that no more events to process exist.

Partitioning a search space for distributed crawling

A computer-implemented process, computer program product, and apparatus for computing excluded data. A web page of interest is identified to form an identified page. The identified page is loaded a first time to form a first load, and responsive to a determination that a delta has not been computed for the identified web page, the identified page is loaded a second time to form a second load. Whether portions of the first load differ from portions of the second load is determined. Responsive to a determination portions of the first load differ from portions of the second load, the portions that differ to form a delta are identified. The delta is stored to form stored delta and the stored delta is excluded from a document object model associated with the identified page to form a modified document object model.

System and method for exclusion of irrelevant data from a DOM equivalence

Embodiments relating to a computer-implemented process, an apparatus and a computer program product is provided for crawling rich Internet applications. In one aspect the method includes executing an event in a set of events discovered in a state exploration phase according to a predetermined priority of events in each set of events in the sets of events discovered, wherein the event from a higher priority is exhausted before an event from a lower priority is executed and determining any transitions. Responsive to a determination that there are at least one transition any remaining set of events is executed in a transition exploration phase. In addition the method determines the existence of any new states as a result of executing an event in the set of events and returns to the state exploration phase, responsive to a determination that a new state exists.

Crawling rich Internet applications

A computer-implemented process, computer program product, and apparatus for examining concurrent system state space models. A system has concurrent events to create a modeled system. A minimal chain decomposition of the modeled system is created to identify a set of chains. All chains in the set of chains are extended, and a first set of paths is generated, wherein the first set of paths is a minimal set of paths covering all states of the modeled system. The first set of paths is explored. It is optionally determined whether all transitions have been covered. Responsive to a determination that all transitions have not been covered, remaining paths are explored by traversing a graph induced by the modeled system using a depth-first strategy.

Growing community of inventors

Ottawa, Canada

Gregor Von Bochmann