Sun Microsystems, Inc.

Oracle International Corporation

Oracle America, Inc.

Thinking Machines Corporation

T. M. Patents, L.p.

Guy Lewis Steele, Jr

Steven K Heller

Nir N Shavit

Paul A Martin

W Daniel Hillis

Ole Agesen

Daniel R Cassiday

David R Chase

Jean-Baptiste Tristan

David L Detlefs

Mark S Moir

Jon P Wade

Andrew Brownsword

Brewster A Kahle

David Lawrence Rager

Marc R Tremblay

Alexander T Garthwaite

Nils Gura

James Michael O'Connor

Sanjay Vishin

Derek R White

George G Robertson

Radia Joy Perlman

Victor M Luchangco

Yosef Lev

Mahesh N Ganmukhi

David C Douglas

Monica C Wong-Chan

Robert C Zak, Jr

Jeffrey V Hill

Shaw-Wen Yang

Robert D Lordi

Clifford A Lasser

John Reynders

Zahi S Abuhamdeh

Joseph Tassarotti

Margaret A St Pierre

Scott J Smith

William N Joy

Maurice P Herlihy

Miriam C Kadansky

Christine H Flood

Dah Ming Chiu

Stephen Joseph Green

John Patrick Connelly

Simon Doherty

Matthew Perry Abrams

Karl P Sims

Kenneth W Crouch

Murat Yuksel

Ron Nordan

Manzil Zaheer

Peter A Lawrence

Robert Otto Barnes, Ii

James B Salem

Rahul Dilip Jagad

Harry Mark Bromley

David Bruce Gingold

Michael Drumbeller

Guy Blelloch

Daniel E Huang

Abhiram Ranade

Disclosed are techniques for processing user profiles using data structures that are specialized for processing by a GPU. More particularly, the disclosed techniques relate to systems and methods for evaluating characteristics of user profiles to determine whether to offload certain user profiles to the GPU for processing or to process the user profiles locally by one or more central processing units (CPUs). Processing user profiles may include comparing the interest tags included in the user profiles with logic trees, for example, logic trees representing marketing campaigns, to identify user profiles that match the campaigns.

Enhanced processing of user profiles using data structures specialized for graphical processing units (GPUs)

Embodiments comprise construction of a collection of pseudorandom number generators (PRNGs), with either a known or unknown cardinality, using unique brine values that comprise a salt value for the collection and also different index values for each PRNG for the collection. The additive parameters of such PRNGs are based on the respective brine values of the PRNGs, thereby ensuring that the PRNGs in the collection have different state cycles. Embodiments make it likely that PRNGs from different collections have distinct additive parameters by choosing a pseudorandom salt value for each collection. According to embodiments, a stream of generators in a collection is created by a spliterator that carries a salt value for the collection and combines the salt value with index values for the generators to produce brined additive parameters for the PRNGs in the stream. According to embodiments, such a stream may be executed by multiple threads in parallel.

Parallel generation of pseudorandom number sequences using multiple generators with brined initial states

A method and apparatus is provided for generating pseudorandom numbers in a way that is deterministic (i.e., repeatable), that passes statistical tests, can have multiple instances of objects generating pseudorandom numbers at the same time. Also, the collection of pseudorandom numbers generated by multiple instances have the same statistical properties as numbers generated by a single instance (i.e., randomness). Embodiments described herein generate pseudorandom values by using a plurality of subsidiary linear congruential generators and combining their outputs nonlinearly. According to embodiments, after their outputs have been combined, a mixing function is applied. Embodiments include an on-demand split method in the style of the SplitMix algorithm.

Generating pseudorandom number sequences by nonlinear mixing of multiple subsidiary pseudorandom number generators

A lookup circuit evaluates hash functions that map keys to addresses in lookup tables. The circuit may include multiple hash function sub-circuits, each of which applies a respective hash function to an input key value, producing a hash value. Each hash function sub-circuit may multiply bit vectors representing key values by a sparse bit matrix and may add a constant bit vector to the results. The hash function sub-circuits may be constructed using odd-parity circuits that accept as inputs subsets of the bits of the bit vectors representing the key values. The sparse bit matrices may be chosen or generated so that there are at least twice as many 0-bits per row as 1-bits or there is an upper bound on the number of 1-bits per row. Using sparse bit matrices in the hash function sub-circuits may allow the lookup circuit to perform lookup operations with very low latency.

Systems and methods for implementing low-latency lookup circuits using sparse hash functions

A lookup circuit evaluates hash functions that map keys to addresses in lookup tables. The circuit includes multiple hash function sub-circuits, each of which applies a respective hash function to an input key, producing a hash value. Candidate pairs of hash functions to be implemented by the hash function sub-circuits may be generated and tested for suitability in hashing a particular collection of keys. The suitability testing may include computing hash value bit vectors by applying each hash function in a candidate pair to a given key, and determining (using a modified union-find type operation that organizes objects in each set as a directed graph whose root points to itself) whether the resulting hash value bit vectors belong to the same set. The union-find type operation may include a limited distance-from-root test, path compression, or exception handling for special cases, but not a rank test.

Systems and methods for generating bit matrices for hash functions using fast filtering

The present invention relates to a probabilistic programming compiler that (a) generates data-parallel inference code to sample from probability distributions in models provided to the compiler; and (b) utilizes a modular framework to allow addition and removal of inference algorithm information based on which the compiler generates the inference code. For a given model, the described compiler can generate inference code that implements any one or more of the inference algorithms that are available to the compiler. The modular compiler framework utilizes an intermediate representation (IR) that symbolically represents features of probability distributions. The compiler then uses the IR as a basis for emitting inference code to sample from the one or more probability distributions represented in the IR. Further, the compiler produces parallelized inference code that facilitates efficient parallel processing of inference computations in order to take advantage of highly data-parallel architectures, such as GPUs.

Data-parallel probabilistic inference

Herein is described an unsupervised learning method to discover topics and reduce the dimensionality of documents by designing and simulating a stochastic cellular automaton. A key formula that appears in many inference methods for LDA is used as the local update rule of the cellular automaton. Approximate counters may be used to represent counter values being tracked by the inference algorithms. Also, sparsity may be used to reduce the amount of computation needed for sampling a topic for particular words in the corpus being analyzed.

Learning topics by simulation of a stochastic cellular automaton

A system may perform work stealing using a dynamically configurable separation between stealable and non-stealable work items. The work items may be held in a double-ended queue (deque), and the value of a variable (index) may indicate the position of the last stealable work item or the first non-stealable work item in the deque. A thread may steal a work item only from the portion of another thread's deque that holds stealable items. The owner of a deque may add work items to the deque and may modify the number or percentage of stealable work items, the number or percentage of non-stealable work items, and/or the ratio between stealable and non-stealable work items in the deque during execution. For example, the owner may convert stealable work items to non-stealable work items, or vice versa, in response to changing conditions and/or according to various work-stealing policies.

Systems and methods for implementing work stealing using a configurable separation of stealable and non-stealable work items

An efficient parallel Gibbs sampler using butterfly-patterned partial sums is provided. Instead of building and searching a complete prefix sums table, an alternative 'butterfly patterned partial sums table' is described that integrates a lightweight transposition and partial sums operation. Accordingly, the usual full matrix transposition and full prefix sums table building operations can be omitted in favor of building the butterfly-patterned partial sums table, which requires less computational and communication effort. This butterfly-patterned partial sums table is used by a modified binary search phase that calculates the needed prefix-sum table values on-the-fly using the butterfly-patterned partial sums table. Transposed memory access is also provided while avoiding the full matrix transform, providing significant performance benefits for highly parallel architectures, such as graphics processing units (GPUs) where 1-stride or sequential memory accesses are important for optimization.

Parallel Gibbs sampler using butterfly-patterned partial sums

Herein is described a data-parallel algorithm for topic modeling in which the memory requirements are streamlined for implementation on a highly-parallel architecture, such as a GPU. Specifically, approximate counters are used in a large mixture model or clustering algorithm (e.g., an uncollapsed Gibbs sampler) to decrease memory usage over what is required when conventional counters are used. The decreased memory usage of the approximate counters allows a highly-parallel architecture with limited memory to process more computations for the large mixture model more efficiently. Embodiments describe binary Morris approximate counters, general Morris approximate counters, and Csr&#xf6;s approximate counters in the context of an uncollapsed Gibbs sampler, and, more specifically, for a Greedy Gibbs sampler.

Method and system for latent dirichlet allocation computation using approximate counters

Herein is described a data-parallel algorithm for topic modeling on a distributed system in which memory and communication bandwidth requirements are streamlined for distributed implementation. According to embodiments, a distributed LDA Gibbs sampling algorithm shares approximate counter values amongst the nodes of a distributed system. These approximate counter values are repeatedly aggregated and then shared again to perform the distributed LDA Gibbs sampling. In order to maintain the shared counter values as approximate counter values of sixteen bits or less, approximate counter values are summed to produce aggregate approximate counter values. These small aggregate approximate counter values are shared between the nodes of the distributed system. As such, the addition of various types of approximate counters is described herein. Specifically, addition of binary Morris approximate counters, general Morris approximate counters, and Cs&#x171;r&#xf6;s approximate counters are described in the context of distributed implementations of an LDA Gibbs sampling algorithm.

Method and system for distributed latent dirichlet allocation computation using addition of approximate counters

Herein is described a data-parallel and sparse algorithm for topic modeling. This algorithm is based on a highly parallel algorithm for a Greedy Gibbs sampler. The Greedy Gibbs sampler is a Markov-Chain Monte Carlo algorithm that estimates topics, in an unsupervised fashion, by estimating the parameters of the topic model Latent Dirichlet Allocation (LDA). The Greedy Gibbs sampler is a data-parallel algorithm for topic modeling, and is configured to be implemented on a highly-parallel architecture, such as a GPU. The Greedy Gibbs sampler is modified to take advantage of data sparsity while maintaining the parallelism. Furthermore, in an embodiment, implementation of the Greedy Gibbs sampler uses both densely-represented and sparsely-represented matrices to reduce the amount of computation while maintaining fast accesses to memory for implementation on a GPU.

Sparse and data-parallel inference method and system for the latent Dirichlet allocation model

Growing community of inventors

Lexington, MA, United States of America

Guy Lewis Steele, Jr