Google Inc.

Arup Mukherjee

Arvind Jain

Anurag Acharya

Li Xiao

A system of reducing the possibility of crawling duplicate document identifiers partitions a plurality of document identifiers into multiple clusters, each cluster having a cluster name and a set of document parameters. The system generates an equivalence rule for each cluster of document identifiers, the rule specifying which document parameters associated with the cluster are content-relevant. Next, the system groups each cluster of document identifiers into one or more equivalence classes in accordance with its associated equivalence rule, each equivalence class including one or more document identifiers that correspond to a document content and having a representative document identifier identifying the document content.

Generating equivalence classes and rules for associating content with document identifiers

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for identifying web hosting entities. In one aspect, a system includes one or more computers programmed to perform operations including maintaining an Internet Protocol (IP) address history for each hostname in a plurality of hostnames. Each IP address history is a time series of IP addresses. The operations further include organizing the hostnames into a collection of groups so that each hostname of the plurality of hostnames is a member of exactly one group in the collection of groups. Each group has a kernel calculated from the IP address histories of the members of the group, and the IP address history of each member of the group is within a threshold distance of the kernel of the group.

Growing community of inventors

San Jose, CA, United States of America

Arup Mukherjee