Microsoft Technology Licensing, LLC

Microsoft Corporation

Clemens Alden Szyperski

Joshua Ryan Williams

Antony S Williams

Donald F Box

Craig Henry Wittenberg

Christopher L Anderson

Gregory Lawrence Hughes

Brad Olenick

Michael James Coulson

Anthony J Moore

Lev Novik

Geoffrey M Kizer

Quetzalcoatl Bradley

Martin James Gudgin

Boris Shulman

Jeffrey S Pinkston

Zhong Chen

Arshad F Ahmad

Erik B Christensen

Marius Dumitru

Randy S Kimmerly

Marc Todd Friedman

Balasubramanian Shyamsundar

John David Doty

Shi Qiao

Simeon John Cran

Terry D Zmrhal

Alexandre Tristan St-Cyr

Faisal Mohamood

Theresa Palmer-Boroski

David George Hunt

Henricus Johannes Maria Meijer

Anders Hejlsberg

Bradford H Lovering

Matthew J Warren

Dinesh Chandrakant Kulkarni

David Michael Miller

Mads Torgersen

Siddharth Jayadevan

Krzysztof J Cwalina

Olivier Nano

John Lawrence Hamby

Peter A Hallam

Alekh Jindal

Mircea Trofin

Hiren Shantilal Patel

Rathijit Sen

Yuri Rychikhin

Ivo Jose Garcia Dos Santos

William Emeric Aitken

William A Manis

Laurent Bussard

Simon David Kent

Jaliya Nishantha Ekanayake

Ziv Kasperski

Christopher Donald Hackmann

Jin Sun

Oleg G Lvovitch

Alex Bulankou

Blake W Stone

Anish Yatin Pimpley

Anubha Srivastava

Vishal Lalchand Rohra

Yi Zhu

Shuo Li

Dirk Siemer

Arthur H Watson

Vladimir Nedkov Hristov

Martin J Gudgin

Solutions for optimizing job runtimes via prediction-based token allocation includes receiving training data comprising historical run data, the historical run data comprising job characteristics, runtime results, and a token count for each of a plurality of prior jobs, and the job characteristics comprising an intermediate representation and job graph data; based at least on the training data, training a token estimator, the token estimator comprising a machine learning (ML) model; receiving job characteristics for a user-submitted job; based at least on the received job characteristics, generating, with the token estimator, token prediction data for the user-submitted job; selecting a token count for the user-submitted job, based at least on the token prediction data; identifying the selected token count to an execution environment; and executing, with the execution environment, the user-submitted job in accordance with the selected token count.

Optimizing job runtimes via prediction-based token allocation

Shuffling of into partitions by first grouping input vertices of a limited number. Each group of input vertices may then be simply shuffled into a corresponding group of intermediate vertices, such as by broadcasting. A second grouping occurs in which the intermediate vertices are grouped by partition. The intermediate vertices then shuffle into corresponding output vertices for the respective partitions of that group. If the intermediate vertices are still too large, then this shuffling may involve recursively performing the shuffling just described, until ultimately the number of intermediate vertices shuffling into the output vertices is likewise limited. Thus, the final shuffling into the output vertices might also be simply performed by broadcasting.

Grouped shuffling of partition vertices

Various technologies pertaining to construction of a query for execution over a cube are described. Tabular data is presented on a displayed on a display screen, where the tabular data represents at least a portion of a data cube. Input is received with respect to the tabular data, and responsive to the input being received, a query is constructed based upon the input. The query is executed over the data cube, resulting in provisioning of a new table.

Constructing queries for execution over multi-dimensional data structures

Stream processing can be performed using a pull-based, anchor-based methodology that guarantees once and only once processing and repeatability of the creation of output with no additional communication overhead during normal processing. Each node (computing device) in the graph (representing interconnected computing devices) establishes a system of anchors. An anchor describes a point in the output stream of the node, so that every event in the stream is either before or after any given anchor.

Using anchors for reliable stream processing

Embodiments described herein are directed to methods, and systems for generating event processing language code in a development environment using an event processing compiler. A query in event processing language is received in a development environment. The query can be associated with sample data from input files or an input data source. An event processing compiler compiles the query, where the compiler transforms the query from event processing language code to a development environment script language code. In particular, the event processing language code transforms the code based on event processing attributes that are intricately aligned in syntax and semantic between the event processing language and the development environment script language. The query as a development environment script is executed using sample data. Executing the query generates output comprising final results data, intermediate results data, and provides for display warnings when mismatches exist between the results data and output specifications.

Event processing development environment

Processing streaming data in accordance with policies that group data by source, enforce a maximum permissible late arrival value for streaming data, a maximum permissible early arrival for data and/or a maximum degree to which data can be out of order and still be compliant with the out of order policy is described. The correct starting point for reading a data stream so as to produce correct output from a given output start time can be enabled using the early arrival policy. Using combinations of policies, output can be generated promptly (with low latency). When input from a given source is not disrupted, output can be generated with low latency. Output can be generated even when the input stops by applying a late arrival policy.

Handling out of order events

Large amounts of memory can be consumed in streaming joins because events from one stream are held in memory while waiting for matching events from a second stream. Memory needs can be reduced by analyzing the join condition to determine the bounds on the time discrepancy between events in the two streams. When it is determined that an event from one stream must occur prior to the matching event from the other stream, the later-arriving stream data can be ingested with an intentional delay. When it is determined that regardless of input received from a first stream, no output will be produced when there is no input from the second stream, pulling data from the first stream can cease. A multi-stage join plan can be employed so that a less busy stream can be scanned with increasing amounts of intentional delay. Only unmatched data is stored.

Streaming joins in constrained memory environments

A dynamic, lazy type system is provided for a dynamic, lazy programming language. Consequently, programs can benefit from runtime flexibility and lightweight notation in combination with benefits afforded by a substantial type system.

Dynamic lazy type system

A nominal type framework can be configured to efficiently correlate different nominal types together based on a minimum set of common type shapes or structures. In one implementation, a developer identifies a number of different nominal types of interest (source types), and identifies the minimum set of common type shapes to be accessed by an application program. The minimum set of common type shapes can then be used to create an intermediate type (target type) to which each of the other different source types can be mapped. For example, one or more proxies can be created that map shapes of the one or more source types to corresponding shapes of the created target type. The application program created by the developer, in turn, can access, operate on, or otherwise use the mapped data of each different source type through a single target type.

Efficiently correlating nominally incompatible types

The present invention relates to an XML application framework (XAF). XAF applications are data driven such that all operations with the computer system are data focused. In addition, the components used in the XAF application are instantiated and connected according to how the data is displayed and what type of data is used. Applications within XAF comprise a user interface (UI) connector, an action module, and a data connector. UI connectors receive UI events and connect the UI event to an action module. The action module generates a standard format action from the UI event and sends it to the data connector. The data connector translates the standard format action into a data-specific action that changes data in a data store. A data connector then sends a standard format data representation corresponding to the changed data back to the UI connector to provide the changed data to the UI.

XML application framework

A common data type structure can be used to correlate access requests between applications that implement data in accordance with different types or type structures. In one implementation, a common data structure includes schemes for operations, sequences, records, and atoms (i.e., undefined). The system can then map any type structure to the schemes of the common data structure. In operation, a request for data by an application can involve identifying one or more proxies used by an application to map the data to the common data structure. The proxies map the data to the common data structure based on the shape of the data (to the extent it can be identified). The proxies then can return one or more data structures that comprise the identified mapping information. The application can then perform operations directly on the received data structures.

Accessing different application data via a common data structure

A framework for open, dynamic, and reflective applications contains high level metadata that define blocks of composable elements using a service definition, which defines specific keys for using the block of code. The service definition may be exported from the code using reflection without having to load the code into memory, and decisions on which block of code and how to load the code may be made at runtime. The composable elements may be grouped into composable parts. Composable part definitions may be used to instantiate or produce composable parts. At runtime, dynamic applications may search from many composable elements, and instantiate and execute the composable element as part of the application.

Framework for open dynamic reflective applications

Growing community of inventors

Bellevue, WA, United States of America

Clemens Alden Szyperski