Verizon Media Inc.

Jason Jinshui Qin

Denys P Kim

Yumei Tung

Systems and methods are disclosed for optimizing full-spectrum cardinality approximations on big data utilizing an optimized order statistics technique. To accomplish the foregoing, a multiset of objects that each corresponds to one of a plurality of objects associated with a resource are obtained. A compound data object is populated at least in part with data that is derived based on generated decimal fraction hash values that correspond to each object in the obtained multiset. The populated compound data object is processed with a full-spectrum arithmetic mean estimation operation that can accurately determine a cardinality estimate for the obtained multiset using less resources and time when compared to traditional techniques. The determination is further made without the need to employ linear counting or bias correction operations on low or high cardinalities. An estimated number of unique objects in the obtained multiset is determined as a result of the processing, and subsequently provided for display or further manipulation.

Optimized full-spectrum order statistics-based cardinality estimation

Systems and methods are disclosed for optimizing full-spectrum cardinality approximations on big data by exploiting an underlying relationship between LogLog counting estimation techniques and order statistics-based estimation techniques. To accomplish the foregoing, a multiset of objects that each corresponds to one of a plurality of objects associated with a resource are obtained by a computing device. A compound data object is populated by the computing device with data that is derived based on generated hash values that correspond to each object in the obtained multiset. The populated compound data object is processed utilizing a processor with a full-spectrum unified estimation operation that can accurately determine a cardinality estimate for the obtained multiset, utilizing considerably less resources when compared to traditional and state of the art techniques. The determination is made by the computing device without the need to employ linear counting for low cardinalities, bias correction operations, or angular correction terms, all while offering decreased memory usage, simpler implementation, improved performance, and comparable or improved accuracy. An estimated number of unique objects in the obtained multiset can be determined by the computing device, and subsequently provided for display, communication to another computing device, or further manipulation.

Optimized full-spectrum cardinality estimation based on unified counting and ordering estimation techniques

Systems and methods are disclosed for optimizing full-spectrum cardinality approximations on big data utilizing an optimized LogLog counting technique. To accomplish the foregoing, a multiset of objects that each corresponds to one of a plurality of objects associated with a resource are obtained. A compound data object is populated at least in part with data that is derived based on generated hash values that correspond to each object in the obtained multiset. The populated compound data object is processed with a full-spectrum harmonic mean estimation operation that can accurately determine a cardinality estimate for the obtained multiset using less resources and time when compared to traditional techniques. The determination is further made without the need to employ linear counting or bias correction operations on low or high cardinalities. An estimated number of unique objects in the obtained multiset is determined as a result of the processing, and subsequently provided for display or further manipulation.

Growing community of inventors

Great Falls, VA, United States of America

Jason Jinshui Qin