Livescribe Inc.

Leapfrog Enterprises, Inc.

Anoto Ab

Tracy L Edgecomb

Jim Marggraff

James L Marggraff

Andy Van Schaack

Alexander Sasha Pesic

Vinaitheerthan Meyyappan

Dan Zacharias Gärdenfors

Andrew J Van Schaack

Teresa Cameron

Alexander Chisholm

Erica Leverett

Gabriel Acosta-Mikulasek

Anders Svensson

Nicole Wrubel

Steve Baldzikowski

A 'self-addressing' writing surface, such as paper, includes an encoded identifier that is uniquely associated with the recipient or a group of recipients. A pen-based computing system is used to capture writing made on the writing surface. The captured writing and the recipient identifier are sent electronically to a routing system, which identifies the recipient to which the content is to be routed based on the recipient identifier. The routing system forwards the message to the identified recipient, thereby enabling communication from the writer to the recipient associated with the writing surface.

Self-addressing paper

In a pen-based computing system, a user-specific smart pen application is created from a template application using customer authoring tools. The template application contains computer program code that is to be executed by a processor of a smart pen. Application content and a representation for printed content are received. The application content, provided by user or customer, defines functional interactions between the printed content representation and a smart pen. The template application is combined with application content to generate a user-specific application comprising instructions for being executed on a processor of a smart pen. The user-specific application is stored on a storage medium.

Customer authoring tools for creating user-generated content for smart pen applications

Embodiments of the invention present a system and method for identifying relationships between different types of data captured by a pen-based computing system, such as a smart pen. The pen-based computing system generates one or more sessions including different types of data that are associated with each other. In one embodiment, the pen-based computing system generates an index file including captured audio data and written data, where the written data is associated with a temporal location of the audio data corresponding to the time the written data was captured. For example, the pen-based computing system applies one or more heuristic processes to the received data to identify relationships between various types of the received data, used to associated different types of data with each other.

Grouping variable media inputs to reflect a user session

Embodiments of the invention present a system and method for controlling audio capture by a smart pen based computing system. An audio capture mechanism that is independent from a gesture capture system is included on the smart pen to control audio capture by one or more microphones included on the smart pen. In one embodiment, the audio capture mechanism comprises a shared function button, such as a power button. For example, a user interaction with the shared function button initiates audio capture by the one or more microphones on the smart pen and a second user interaction with the shared audio function stops audio capture. Alternatively, audio capture is stopped after completion of a predefined time interval after the user interaction with the shared function button.

Quick record function in a smart pen computing system

Paper-based playback of media can be performed by electronically recording handwritten notes including pen strokes using a digital pen on a position-coded paper. A plurality of bounding areas is identified, e.g. by generating bounding areas around the pen strokes as they are developed. Each bounding box is provided with a time stamp which indexes a media file, which was recorded simultaneously with the handwritten notes. By placing the digital pen close to a handwritten note on the paper, the part of the media that was recorded when the specific note was written can be recalled. More specifically, the bounding box corresponding to the position of the digital pen is identified and the associated time stamp is used to find the media to recall. This paper-based playback of media can be performed in e.g. a stand-alone device or by a combination of a digital pen and a mobile phone.

System and method for recalling media

In a pen-based computing system, multi-modal data is transferred between a paper domain and a digital domain. Data initially generated in the paper domain is captured by a smart pen and a digital file including the captured data is generated. For example, a computing system coupled to the smart pen generates a digital file including the captured data. A paper representation of the digital file is subsequently generated. The digital file can subsequently be modified by editing the paper representation of the digital file using the smart pen. Edits to the paper representation of the digital file are captured by the smart pen and converted to the digital domain where they are used to edit the content of the digital file.

Cyclical creation, transfer and enhancement of multi-modal information between paper and digital domains

In a pen-based computing system, use of paradigms similar to those used with physical paper to organize user generated content captured by a smart pen is disclosed. Data, such as handwriting gestures, is captured by the smart pen and transferred to a digital domain, such as by being transferred to a computing system. Once in the digital domain, the captured content is organized as virtual pages or virtual notebooks. Hence, content captured from various sources, such as different notebooks or different physical pieces of paper, is assembled into a virtual page or virtual notebook. User input or automatic application of rules can be used to assemble the captured content into a virtual page or virtual notebook.

Organization of user generated content captured by a smart pen computing system

In a pen-based computing system, a microphone on the smart pen device records audio to produce audio data and a gesture capture system on the smart pen device records writing gestures to produce writing gesture data. Both the audio data and the writing gesture data include a time component. The audio data and writing gesture data are combined or synchronized according to their time components to create audio ink data. The audio ink data can be uploaded to a computer system attached to the smart pen device and displayed to a user through a user interface. The user makes a selection in the user interface to play the audio ink data, and the audio ink data is played back by animated the captured writing gestures and playing the recorded audio in synchronization.

Animation of audio ink

In a pen-based computing system, a printed version of a document having preexisting content is annotated using a smart pen. The smart pen captures handwriting gestures to obtain an electronic representation of the annotations. The smart pen computing system identifies a digital version of the document having the preexisting content and stores the electronic representation of the annotations in association with the digital document. The smart pen computing system may overlay the electronic representation of the annotations with the preexisting content to provide a digital representation of the annotated document.

Electronic annotation of documents with preexisting content

A device user interface in which computer functionality is invoked by user interaction with dynamically generated interface regions of a writing surface. A computer system identifies a marking written on the writing surface or a user selection of an existing written marking. Adjacent to the marking, the computer system automatically generates one or more interface regions associated with the marking. User interaction with one of these regions automatically invokes computer functionality related to the interacted region. A different function may be invoked by each region. The computer system dynamically positions and may dynamically size the interface regions based on the position (and size) of the marking. Multiple markings yield multiple regions, with different regions associated with respective markings. In one embodiment, the regions are established in front of and/or after a written word. Regions may also be established on top of and/or below the written word, for example. In another embodiment, the computer system is a pen-based interactive computer system.

Method and system for invoking computer functionality by interaction with dynamically generated interface regions of a writing surface

A method and system for implementing a user interface for a device through user created graphical elements. The method includes recognizing a graphical element icon created by a user. Once recognized, a function related to the graphical element icon is accessed and an output in accordance with the function is provided. The function is persistently associated with the graphical element icon. Menu selection and navigation is implemented through interaction with the graphic element icon. A listing of options associated with the graphical element icon is audibly rendered. In response to a selection of one of the options, the selected option is invoked.

Method and system for implementing a user interface for a device employing written graphical elements

Embodiments of the invention recite a method, apparatus and system for conveying cartridge notification. In one embodiment, a permanently encoded pattern of location information embedded on a printed medium is recognized using an electronic interactive device. Note that the encoded pattern identifies an application. A determination can be made as to whether the electronic interactive device has access to the application based on its memory and the memory of an attached cartridge. In response to the electronic interactive device not having access to the application, it is determined based on a cartridge lookup table of which cartridge includes the application. A message can be conveyed regarding the cartridge to the user. The cartridge lookup table may be populated with information regarding the application contents of various cartridges in response to these cartridges being inserted into the interactive device.

Growing community of inventors

Berkeley, CA, United States of America

Tracy L Edgecomb