Microsoft Technology Licensing, LLC

Eric I Horvitz

David Earl Heckerman

John S Breese

David O Hovel

Samuel D Hobson

Gregory L Shaw

Adrian C Klein

Jacobus A Rommelse

John Carlton Platt

Susan Theresa Dumais

Carl Myers Kadie

Erich S Finkelstein

Mehran Sahami

Ranveer Chandra

John Charles Krumm

David Maxwell Chickering

Nissanka Arachchige Bodhi Priyantha

Anirudh Badam

Andrew W Jacobs

Suryanarayanan V Raman

William J Bliss

Julia Louise Meinershagen

Jerome E Lengyel

Semiha Ece Kamar Eden

James Randall Flynn

Keiji Kanazawa

Karen Jensen

Besmira Nushi

Jerry R Hughson, Jr

Michael Lowell Roberts

Lois E Bauer

Christopher Dekmezian

Erich-So Slashedren Finkelstein

The present disclosure relates to systems, methods, and computer readable media that evaluate performance of a machine learning system in connection with a test dataset. For example, systems disclosed herein may receive a test dataset and identify label information for the test dataset including feature information and ground truth data. The systems disclosed herein can compare the ground truth data and outputs generated by a machine learning system to evaluate performance of the machine learning system with respect to the test dataset. The systems disclosed herein may further generate feature clusters based on failed outputs and corresponding features and generate a number of performance views that illustrate performance of the machine learning system with respect to clustered groupings of the test dataset.

Characterizing failures of a machine learning model based on instance features

A shared electrode battery includes multiple electrodes of one type (e.g., two or more cathodes) that share an electrode of another type (e.g., a shared anode). The multiple electrodes of the same type (e.g., the multiple cathodes) can have different characteristics, such as different chemistries, particle sizes and distributions, capacities, and so forth that are designed to provide particular features such as high energy density, high power density, high cycle life, fast charge, safety, and so forth. Multiple cathode-anode pairings of one of the multiple electrodes of the same type with the shared electrode are possible. Switching hardware is operable to select one of the multiple pairings at any given time, allowing the battery to provide power using the cathode having the desired characteristics at that given time. A single battery is thus able to provide these multiple different features.

Shared electrode battery

The present invention relates to a system () and methodology () to enable a plurality of information associated with electronic messages, for example, to be automatically prioritized by a priorities system () for transmittal to a user or system. The priorities system () can employ classifiers () that can be explicitly and/or implicitly trained to prioritize one or more received messages () according to a learned importance to the user. As an example, messages () can be classified as high, medium, low or other degrees of importance via a training set of examples () or types of messages having similar degrees of importance. A background monitor () can be provided to monitor a user's activities regarding message processing to further refine or tune the classifier () according to the user's personal decisions relating to message importance. Other priorities classifications can involve determinations relating to a loss associated with a time for delayed review or processing of the message.

Priorities generation and management

A location-centric signal shadow mapping and storing architecture that creates maps where signals, such as GPS, cannot be seen with ease because of natural or synthetic features such as groups of tall buildings. Such maps are used with other information, such as the dynamics of the sensed velocity that had been seen before the loss of the signals, to reason about the location and likely activities being carried out by one or more people. Inferences can be made based on information about organizations and services associated with structures and locations proximal to the locations where signals were lost. Also, such reasoning can be used to turn off or to reduce the power consumed by the receivers of the location information, potentially with the joint use of accelerometers to identify when significant accelerations occur.

Learning, storing, analyzing, and reasoning about the loss of location-identifying signals

Prioritizing text such as e-mail messages, given constrained resources, is disclosed. In one embodiment, a method first receives a plurality of messages, such as e-mail messages. The method prioritizes the messages, based on a predetermined criteria given a constrained resource. The method then outputs the messages to a user, as prioritized given the constrained resource. Specific constrained resources include limited bandwidth, limited storage space, and limited time in which to receive the texts. Embodiments include policies that include priority-sensitive truncation and summarization. Furthermore, in one embodiment, if context is detected, the priorities for caching are affected. In another embodiment, the likelihood that a user already knows all or part of the information being transmitted is considered.

Transmitting information given constrained resources

An intelligent user assistance system is disclosed. The system is constructed using a reasoning model to determine the probability of a user's intentions, goals, or informational needs in the context of a software program. This is accomplished through analysis of information about the user's actions, the program state, and spoken words. The system monitors user interaction with the software program and applies probabilistic reasoning to sense that the user may need assistance in using a particular feature or to accomplish a specific task. The inference system accesses a thorough and updateable user profile system to continually check for competencies and changes assistance that is given based on the competencies.

Intelligent user assistance facility

A general event composing and monitoring system that allows high-level events to be created from combinations of low-level events. An event specification tool allows for rapid development of a general event processor that creates high-level events from combinations of user actions. The event system, in combination with a reasoning system, is able to monitor and perform inference about several classes of events for a variety of purposes. The various classes of events include the current context, the state of key data structures in a program, general sequences of user inputs, including actions with a mouse-controlled cursor while interacting with a graphical user interface, words typed in free-text queries for assistance, visual information about users, such as gaze and gesture information, and speech information. Additionally, a method is provided for building an intelligent user interface system by constructing a reasoning model to compute the probability of alternative user's intentions, goals, or informational needs through analysis of information about a user's actions, program state, and words. The intelligent user interface system monitors user interaction with a software application and applies probabilistic reasoning to sense that the user may need assistance in using a particular feature or to accomplish a specific task. The intelligent user interface also accepts a free-text query from the user asking for help and combines the inference analysis of user actions and program state with an inference analysis of the free-text query. The inference system accesses a rich, updatable user profile system to continually check for competencies and changes assistance that is given based on user competence.

Intelligent user assistance facility for a software program

A decision-theoretic regulator employs a method for allocating computational resources to components of media content to create the highest quality output for a budget of rendering resources. The components of the content represent parts of the content that have independent quality parameters that the regulator can vary to trade-off quality for computational savings. For example, in multimedia content, the components might be objects in a 3D graphics scene. The method allocates computational resources by attempting to minimize the total expected cost of a rendering task. The method computes the raw error for a rendering action on a component and then maps the raw error to a perceived error based on empirical evidence of how users perceive errors in rendered output. The expected cost is computed from the perceived error or raw error by applying a model of attention that gives the probability that a user is focusing his or her attention on a component. The method minimizes the total expected cost by selecting a rendering action for each component that yields the lowest expected cost for a given rendering budget.

Decision-theoretic regulation for allocating computational resources among components of multimedia content to improve fidelity

A text classifier and building the text classifier by determining appropriate parameters for the text classifier.

Methods and apparatus for classifying text and for building a text classifier

A technique, specifically apparatus and accompanying methods for use therein, that, through continual computation, harnesses available computer resources during periods of low processing activity and low network activity, such as idle time, for prefetching, e.g., web pages, or pre-selected portions thereof, into local cache of a client computer. As the browser prefetches and stores each web page (or component thereof) in its local cache, the browser provides a suitable and preferably visual indication, through its graphical user interface, to a user that this item has been fetched and stored. Consequently, the user can quickly and visually perceive that a particular item (i.e., a &ldquo;fresh&rdquo; page or portion) has just been prefetched and which (s)he can now quickly access from local cache. As such additional items are cached, the browser can change the color of the displayed hotlink associated with each of the items then stored in cache so as, through color coding, to reflect their relative latency (&ldquo;aging&rdquo;) in cache.

Method and apparatus for display of information prefetching and cache status having variable visual indication based on a period of time since prefetching

A technique, specifically a method and apparatus that implements the method, which through a probabilistic classifier (370) and, for a given recipient, detects electronic mail (e-mail) messages, in an incoming message stream, which that recipient is likely to consider 'junk'. Specifically, the invention discriminates message content for that recipient, through a probabilistic classifier (e.g., a support vector machine) trained on prior content classifications. Through a resulting quantitative probability measure, i.e., an output confidence level, produced by the classifier for each message and subsequently compared against a predefined threshold, that message is classified as either, e.g., spam or legitimate mail, and, e.g., then stored in a corresponding folder (223, 227) for subsequent retrieval by and display to the recipient. Based on the probability measure, the message can alternatively be classified into one of a number of different folders, depicted in a pre-defined visually distinctive manner or simply discarded in its entirety.

Technique which utilizes a probabilistic classifier to detect 'junk'

A technique that, through continual computation, harnesses available computer resources during periods of low processing activity and low network activity, such as idle time, for prefetching, e.g., web pages, or pre-selected portions thereof, into local cache of a client computer. This technique utilizes a probabilistic user model to specify, at any one time, those pages or portions of pages that are likely to be prefetched given, e.g., a web page currently being rendered to a user, which promise to provide the largest benefit (expected utility) to the user. Specifically, once a user, at a client computer, enters an address of a desired web page, a set containing web addresses of web pages, that based on the user model are each likely to be accessed next by that user, are determined, with corresponding files therefor prefetched, in order of their expected utility to the user, by the client computer during intervals of low processing activity and low network activity. Expected utility of a page or portion is assessed as a product of rate of refinement in utility of that page or portion to the user multiplied by its transition probability. Once prefetched, these pages or portions are stored in local cache at the client computer for ready access should the user next select any such page or portion.

Method and apparatus for utility-directed prefetching of web pages into

A technique that, through continual computation, harnesses available computer resources during periods of low processing activity and low network activity, such as idle time, for prefetching, e.g., web pages, or pre-selected portions thereof, into local cache of a client computer. This technique utilizes, e.g., a probabilistic user model, which specifies, at any one time, those pages or portions of pages that are likely to be prefetched given, e.g., a web page currently being rendered to a user; these pages being those which promise to provide the largest benefit (expected utility) to the user. Advantageously, this technique prematurely terminates or retards a current information download for a user in favor of prefetching a web page of future interest to that user whenever the latter page exhibits greater current incremental benefit to that user, in terms of a discounted expected rate of refinement in value with time for the future page, than a current incremental benefit being obtained for that user by continuing the current download.

Technique for prefetching a web page of potential future interest in

An event composing and monitoring system that allows high-level events to be created from combinations of low-level events. An event specification tool, contained in the system, allows for rapidly developing a general event processor that creates high-level events from combinations of user actions. An event system, in combination with an inference system, monitors and infers, for various purposes, about several classes of events including: current program context; state of key data structures; user input sequences, including actions with a mouse-controlled cursor while interacting with a graphical user interface; words typed in free-text help queries; visual user information, such as gaze and gesture information; and user speech information. Additionally, an intelligent user interface is provided by constructing a reasoning model that computes probability of alternative user intentions, goals or information needs through analyzing information regarding program state, and that user's actions and free-text query words. Specifically, the interface monitors user interaction with a program and probabilistically reasons to sense that a user may need assistance in using a particular feature or to accomplish a specific task. This interface accepts a free-text help query from the user and combines the inference analysis of user actions and the program state with an inference analysis of the query. The inference system, using an updateable user profile, continually checks for user competencies and, based on such competencies, changes assistance that is offered.

Growing community of inventors

Kirkland, WA, United States of America

Eric I Horvitz