Microsoft Technology Licensing, LLC

David J Kurlander

Yuan Kong

Joel P Dehlin

David W Williams

Zhengyou Zhang

Steven Nabil Bathiche

Andrew David Wilson

Greg D Schechter

Shanku Shivabrata Niyogi

Christina Summer Chen

Zicheng Liu

John D Rinaldo, Jr

Xuedong D Huang

Dawson Yee

Nigel Stuart Keam

Silviu-Petru Cucerzan

Donald Marcus Gillett

Daniel Ryan Lehenbauer

Robert Allan Brigham, Ii

Chun-Fu Ricky Yeung

Regis L F Brid

Mike Sinclair

Regis Lucien Francis Brid

Kris N Nye

Daniel T Ling

A multimodal system that employs a plurality of sensing modalities which can be processed concurrently to increase confidence in connection with authentication. The multimodal system and/or set of various devices can provide several points of information entry in connection with authentication. Authentication can be improved, for example, by combining face recognition, biometrics, speech recognition, handwriting recognition, gait recognition, retina scan, thumb/hand prints, or subsets thereof. Additionally, portable multimodal devices (e.g., a smartphone) can be used as credit cards, and authentication in connection with such use can mitigate unauthorized transactions.

Multimodal authentication

An interactive table has a display surface on which a physical object is disposed. A camera within the interactive table responds to infrared (IR) light reflected from the physical object enabling a location of the physical object on the display surface to be determined, so that the physical object appear part of a virtual environment displayed thereon. The physical object can be passive or active. An active object performs an active function, e.g., it can be self-propelled to move about on the display surface, or emit light or sound, or vibrate. The active object can be controlled by a user or the processor. The interactive table can project an image through a physical object on the display surface so the image appears part of the object. A virtual entity is preferably displayed at a position (and a size) to avoid visually interference with any physical object on the display surface.

Interaction between objects and a virtual environment display

A multi-modal multi-lingual mobile device that facilitates intelligently automating an action. The device can automatically synchronize a user schedule based upon a user state, intention, preference and/or limitation. The device can employ sensors to automatically detect criteria by which to automatically implement an action. Moreover, the system can interrogate a user thus converging upon a user intention and/or preference. An analyzer component can intelligently evaluate the compiled criterion in order to automatically perform an action. The multi-modal multi-lingual mobile device can automatically facilitate identification of an individual. Other actions that are automatically performed can include modifying personal information manager data, translating languages into a language comprehendible to a user, etc. Implementation of these actions can be based at least in part upon an environmental factor, a conversation, a location factor and a temporal factor.

Multi-modal device capable of automated actions

A multimodal, multilanguage mobile device which can be employed to enhance note taking and/or annotation of a document, and gaming. Input data types such as optical character recognition (OCR), speech, handwriting, and visual information (e.g., image and/or video), etc., can be fused to generate rich documents with a multidimensional level of data to provide an increased level of context over conventional documents. Such architecture can be utilized by students for homework management, as well as entertainment (e.g., gaming).

Multimodal note taking, annotation, and gaming

Portable computing devices automatically interface with other computing devices to interact in a collaborative effort toward providing a single, seamless computing experience for a user. As a user walks into a room with a cellular telephone certain functionality and data can be automatically unloaded to a desktop computer or other device based on a user or device identification or state. For example, a conversation on a cellular telephone can be automatically migrated to a desktop telephone as a user sits down. As a user is about to leave a room for a meeting, the desktop computer can update the telephone with latest versions of certain files. Thus, devices can automatically aggregate and/or decouple to provide a user with a single computing experience. These portable devices can broadcast an extensible set of services to other devices as well as to a host computer or server.

Seamless integration of portable computing devices and desktop computers

An object placed on an interactive display surface is detected and its position and orientation are determined in response to IR light that is reflected from an encoded marking on the object. Upon detecting the object on an interactive display surface, a software program produces a virtual entity or image visible through the object to perform a predefined function. For example, the object may appear to magnify text visible through the object, or to translate a word or phrase from one language to another, so that the translated word or phrase is visible through the object. When the object is moved, the virtual entity or image that is visible through the object may move with it, or can control the function being performed. A plurality of such objects can each display a portion of an image, and when correctly positioned, together will display the entire image, like a jigsaw puzzle.

Using clear-coded, see-through objects to manipulate virtual objects

A light pointer is selectively activated to direct a light beam onto an interactive display surface, forming a pattern of light that is detected by a light sensor disposed within an interactive display table. The waveband of the light produced by the light pointer is selected to correspond to a waveband to which the light sensor responds, enabling the light sensor to detect the position of the pattern on the interactive display surface, as well as characteristics that enable the location and orientation of the light pointer to be determined. Specifically, the shape and size of the pattern, and the intensity of light forming the pattern are detected by the light sensor and are processed to determine the orientation of the light pointer and its distance from the interactive display surface. The pattern may comprise various shapes, such as circles, arrows, and crosshairs.

Using a light pointer for input on an interactive display surface

A system that facilitates consolidating resources (e.g., devices, services) based at least in part upon an established context. More particularly, a context determination component can be employed to establish a context by processing sensor inputs or learning/inferring a user action/preference. Once the context is established via a context determination component, a consolidation component can be employed to aggregate and/or disaggregate disparate resources (e.g., devices, services) in accordance with the established context. These resources can work autonomously and/or in conjunction with a subset of the other resources to provide for a highly rich multi-modal user experience.

Aggregation of multi-modal devices

One or more objects placed on a display surface of an interactive display system are identified by a camera that receives infrared light reflected back by the object(s) through the display surface. The interactive display system displays images on the display surface that are produced by software applications, as well as sensing objects placed proximate to the display surface. If object(s) that were placed on the display surface are identified and determined to be associated with a predefined software application, the application is automatically executed. Objects can be identified based upon shape, a pattern applied to the objects, a size of the object, or a location on the display surface where the object is placed. The object may be used in the software application after it is executed or may simply be bear a functional relationship to the software application.

Invoking applications with virtual objects on an interactive display

A server-based application includes at least one page file that identifies one or more server objects. The page file describes the particular layout and interaction of the server objects, such as controls, in such a way that a page of information may be transmitted to a target device. When designing the page file a developer may use declarative statement, so that certain properties of the specified controls may be altered based on the particular type of target device.

Inserting devices specific content

A method and system for interacting with devices having different capabilities is described. Intelligent server-side objects (referred to as adapters) are used to translate information and commands to and from various formats depending on the requirements and capabilities of the target device. Using an interface to the adapters, a software developer may create a form without knowing the exact details or features of the device upon which the form will be displayed. Adapters are logically grouped together in adapter sets wherein a set of adapters may be used to transform information to and from a set of devices with common capabilities. An adapter may inherit attributes and methods from another adapter in the same or another adapter set. An adapter set may inherit adapter associations, i.e., which server objects should be mapped to which adapters, from another adapter set. To make a new or existing device compatible with existing server applications, a device developer may relatively quickly create one or more adapters.

Method and system for interacting with devices having different capabilities

A system and/or method that generates user interface output sequences controlled by a user interface output system. The user interface output system can provide event definitions to an application pro that specify high-level actions to be performed by the sequence and can issue low-level commands to direct the actions of the user interface output sequence. The user interface output system provides a user interface output controller, which acts as an interface between an application program and the low-level commands which specify tasks for the user interface output sequence to perform. The user interface output controller is generated from a specification, using a planning methodology.

Method and system for generating user-interface output sequences

A system retrieves a first template associated with data identified in a data request. A second template is generated using the first template and a device description, which is associated with a device generating the data request. The first template is device independent and the second template is associated with the specific device generating the data request as well as the first template. The second template defines a data presentation format for displaying the requested data on the type of device that generated the data request. The second template is stored in a cache, which allows the second template to be used with multiple sets of data without regenerating the template. The requested data is retrieved from a data source and formatted based on the second template. The formatted data is then transmitted to the device generating the data request.

Method and apparatus for applying an adaptive layout process to a layout template

A comic generation system controls the generation of comic panels. A comic panel is a graphical representation of an instance in time of a sequential course of events. Each graphical representation typically includes a graphical representation of at least one character. The character has gestures that reflect a body pose and expressions that reflect a facial pose. Moreover, the character may have text associated with it (i.e., the character &ldquo;speaks&rdquo;), which is displayed in the comic panel within a bounded area called a &ldquo;balloon.&rdquo; The comic generation system receives an interaction event and generates a comic panel. Examples of interaction events include text input by a user and selection of gestures and expressions by a user. When the interaction event provides text, the comic generation system automatically searches the text to identify any words or phrases that give rise to gestures and expressions for the character representing the user who provided the interaction event. Moreover, the comic generation system searches the text for trigger words that indicate alterations to the comic panel. The comic generation system generates the comic panel based on the results of its searches. The comic generation system also determines which characters to place in a comic panel, the positions of the characters, and the orientations of the characters. The comic generation system additionally determines the placement of balloons containing text. Also, the comic generation system determines when to generate a new comic panel and when to modify a comic panel that has already been generated. Furthermore, the comic generation system maintains a graphical history of the comic panels.

Growing community of inventors

Bellevue, WA, United States of America

David J Kurlander