Yahoo Assets LLC

Yahoo! Inc.

Excalibur Ip, LLC

Verizon Patent and Licensing Inc.

Oath Inc.

Verizon Media Inc.

Yahoo Holdings, Inc.

Allie K Watfa

George Aleksandrovich

Dale Arthur Nussel

Jonathan David Kilroy

David Willcox

Michael Pippin

Robin Sahner

Mike Pippin

Anupam Seth

Mangesh Madan Pardeshi

Maulik Shah

Jacob Tolar

Nilam Jyoti Sharma

Michael D Hanson

Manikandan Murugesan

Rahul Chandran

Sergiu Cristian Vlas

Mumtaz Hannah Bee Vauhkonen

Lucas Saltz

Annie Y Wong

Sujata Walsh

Vinoth Kumar Rajasekar

Adabel Jasmin Marquina

Rajesh Pratabrai Lalwani

Disclosed embodiments are methods, apparatuses, and computer-readable media for annotating distributed data without redundant data copying. In one embodiment, a method is disclosed comprising reading a raw dataset, the raw dataset comprising a first set of columns and a first set of rows; generating an annotation dataset, the annotation dataset comprising a second set of columns and a second set of rows; assigning row identifiers to each row in the second set of rows, the row identifiers aligning the second set of rows with the first set of rows based on the underlying storage of the raw dataset and annotation dataset; and writing the annotation dataset to a distributed storage medium.

Annotating datasets without redundant copying

A device may receive image data identifying images of a network device, and may receive environmental data, historical outage data, performance data, customer data, and historical weather data associated with the network device. The device may process the image data, with a first model, to identify objects of the network device and to generate classifications for the objects, and may process the objects, the classifications, and the environmental data, with a second model, to determine relationships between an environment of the network device and the objects. The device may process the objects, the classifications, the environmental data, the relationships, the historical outage data, the performance data, the customer data, and the historical weather data, with a third model, to predict a probability of damage to the network device or maintenance issues for the network device, and may perform actions based on the probability of damage and/or the maintenance issues.

Systems and methods for utilizing images, computer vision, and empirical data to measure potential damage at network sites

Disclosed are embodiments for providing batch performance using a stream processor. In one embodiment, a method is disclosed comprising processing a plurality of events using a stream processor and executing a deduplication process on the plurality of events using the stream processor. The plurality of events is outputted to a streaming queue and a close of books (COB) of a data transport is detected. Then, an audit process is initiated in response to detecting the COB signal, the audit process comprising comparing a set of raw events to a set of events in the streaming queue to identify a set of missing events, and replaying a set of missing events through the stream processor.

Duplicate detection and replay to ensure exactly-once delivery in a streaming pipeline

Disclosed are embodiments for horizontally skimming composite datasets. In one embodiment, a method is disclosed comprising receiving a script, the script including commands to access a composite dataset; pre-processing the script to identify a set of columns; loading a metadata file associated with the composite dataset file; parsing the metadata file to identify one or more datasets that include a column in the set of columns; loading data from the one or more datasets; and executing the script on the one or more datasets.

Horizontal skimming of composite datasets

One or more computing devices, systems, and/or methods are provided. In an example, financial account information associated with a plurality of payment options may be determined. First financial account information of the financial account information is associated with a first payment option of the plurality of payment options. The first financial account information may be indicative of a first rewards profile associated with transactions performed using the first payment option. A request associated with a transaction may be received. A plurality of financial return scores associated with the plurality of payment options may be determined based upon the financial account information and transaction information associated with the transaction. A first financial return score of the plurality of financial return scores may be associated with the first payment option. The first payment option of the plurality of payment options may be selected for the transaction based upon the plurality of financial return scores.

Method and system for selecting payment option for transaction

Disclosed are embodiments for providing batch performance using a stream processor. In one embodiment, a method is disclosed comprising receiving, at a stream processor, an event, the stream processor including a plurality of processing stages; generating, by the stream processor, an augmented event based on the event, the augmented event including at least one additional field not appearing in the event, the additional field generated by an operation selected from the group consisting of a join or dimensional annotation operation; and emitting, by the stream processor, the augmented event to downstream consumer.

Joining and dimensional annotation in a streaming pipeline

Techniques for providing batch performance using a stream processor are described. In one technique, a method includes receiving an event that includes an event key and a timestamp. The method then generates a partition value and time bucket value, the partition value generated based on the event key and the time bucket value generated based on the timestamp. The event key, partition value, and time bucket value are then combined to form a database key and used in a command to a database.

Partitioned backing store implemented in a distributed database

Disclosed are embodiments for providing batch performance using a stream processor. In one embodiment, a method is disclosed comprising detecting a real close of books (COB) of a data transport, the real COB associated with a set of raw events transmitted over the data transport, flushing a stream processor in response to detecting the real COB, and retrieving a set of processed events from a distributed file system after the flushing is complete. A fact COB computation is then performed on the set of processed events and the set of raw events, the fact COB computation outputting a number of missing events, each missing event representing a raw event that is not present in the set of processed events. The processed events are then certified upon determining that the number of missing events is below a threshold.

Certifying events in a streaming pipeline

Disclosed are embodiments for sorting rows of a dataset after a JOIN operation. In one embodiment, a method is disclosed comprising performing a JOIN operation on an annotation dataset, the performing of the JOIN operation generating an unordered dataset; grouping a plurality of rows in the unordered dataset into a plurality of buckets, the grouping performed based on a root dataset associated with the annotation dataset; sorting each bucket, the sorting comprising sorting each bucket independently; and combining each sorted bucket into a sorted dataset.

Sorting unsorted rows of a composite dataset after a join operation

Disclosed are embodiments for providing batch performance using a stream processor. In one embodiment, a method is disclosed comprising receiving an event, such as a streaming event, from a client. The method determines that the event comprises a primary event and, if so, writes the primary event to a cache and returning the primary event to the client. The method later receives a second event from the client, the second event associated with the first event, annotates the second event based on the primary event, and returns the annotated second event to the client.

Pluggable join framework for stream processing

A method includes executing, on a processor of a computing device, instructions that cause the computing device to perform operations. The operations include executing, on a processor of a computing device, instructions that cause the computing device to perform operations. The operations include receiving site features associated with a communication site and receiving event features associated with a potential outage-causing event. A classifying engine is employed to generate an impact metric indicating an effect on the communication site from the potential outage-causing event based on the site features and the event features.

Communication site potential outage causing event impact prediction

Disclosed are embodiments for providing batch performance using a stream processor. In one embodiment, a method is disclosed comprising receiving an event that includes a plurality of fields and extracting needed fields from the plurality of fields. The method then serializes the plurality of fields and generates a new event that includes the set of needed fields and a hidden field, the value of the hidden field comprising the serialized fields. The method then transmits the new event for processing using at least one processing stage of a stream processor. In response, the method reserializes a processed event generated by the stream processor and outputs the reserialized event to a downstream consumer.

Automatic schema upgrade in a big data pipeline

Disclosed are embodiments for providing batch performance using a stream processor. In one embodiment, a method is disclosed comprising receiving a stream of events, where each event comprises a header including an event type field and a timestamp. and identifying that the header of a given event in the stream of events includes at least one secondary event in a list of secondary identifiers included in the header. A flag is set in a distributed database for the at least one secondary event, then writing the given event to a table in the distributed database occurs, the table selected based on a respective event type field and timestamp associated with the given event. Deduplicating the given event and the at least one secondary event in response to a close of books (COB) signal occurs, after which writing at least one deduplicated event to a storage device occurs, the at least one deduplicated event selected from the group consisting of the given event and the at least one secondary event.

Exactly-once performance from a streaming pipeline in a fault-vulnerable system

Disclosed are embodiments for providing batch performance using a stream processor. In one embodiment, a method is disclosed comprising completing a first audit for a primary event type, the first audit generating a set of primary events and completing a second audit for a secondary event type, the second audit generating a draft set of secondary events and an auxiliary feed of un-joined secondary events. The method then performs a join audit check on the auxiliary feed of un-joined secondary events and a set of flags, each flag in the set of flags indicating that a respective un-joined secondary event was properly joined. Based on the results of the join audit check, the method replays a subset of the un-joined secondary events in the auxiliary feed upon determining that the join audit check failed.

Growing community of inventors

Urbana, IL, United States of America

Allie K Watfa