Livescribe Inc.

Andy Van Schaack

Jim Marggraff

Tracy L Edgecomb

Francis James Canova, Jr

James L Marggraff

Vinaitheerthan Meyyappan

Stefan Burstrom

Rick Lewis

Byron Connell

The described embodiments present a system and method for obtaining a portion of a digital document from a printed version of the document. Locations where a smart pen device interacts with a digital bookclip aligned relative the printed document are identified via coordinate sets. A digital document associated with the printed document and the coordinate sets from the smart pen device and digital bookclip are used to identify locations in the digital documents corresponding to locations in the printed document. Data from the identified locations in the digital documents is retrieved and presented to a user and can then be copied, saved, mailed electronically, and/or printed to a new paper document.

Digital bookclip

In a pen-based computing system, a user-specific smart pen application is created from a template application using customer authoring tools. The template application contains computer program code that is to be executed by a processor of a smart pen. Application content and a representation for printed content are received. The application content, provided by user or customer, defines functional interactions between the printed content representation and a smart pen. The template application is combined with application content to generate a user-specific application comprising instructions for being executed on a processor of a smart pen. The user-specific application is stored on a storage medium.

Customer authoring tools for creating user-generated content for smart pen applications

In a pen-based computing system, use of paradigms similar to those used with physical paper to organize user generated content captured by a smart pen is disclosed. Data, such as handwriting gestures, is captured by the smart pen and transferred to a digital domain, such as by being transferred to a computing system. Once in the digital domain, the captured content is organized as virtual pages or virtual notebooks. Hence, content captured from various sources, such as different notebooks or different physical pieces of paper, is assembled into a virtual page or virtual notebook. User input or automatic application of rules can be used to assemble the captured content into a virtual page or virtual notebook.

Organization of user generated content captured by a smart pen computing system

In a pen-based computing system, a microphone on the smart pen device records audio to produce audio data and a gesture capture system on the smart pen device records writing gestures to produce writing gesture data. Both the audio data and the writing gesture data include a time component. The audio data and writing gesture data are combined or synchronized according to their time components to create audio ink data. The audio ink data can be uploaded to a computer system attached to the smart pen device and displayed to a user through a user interface. The user makes a selection in the user interface to play the audio ink data, and the audio ink data is played back by animated the captured writing gestures and playing the recorded audio in synchronization.

Animation of audio ink

In a pen-based computing system, a pair of microphones captures audio concurrently with a smart pen device capturing handwriting gestures. The audio captured by the pair of microphones is processed to produce an enhanced audio recording. Noise cancellation techniques may be applied to reduce noise generated by the smart pen device interacting with the writing surface to produce a higher quality audio recording. In addition, beam steering techniques may be applied to adjust the gain of portions of the audio signal from different directions and allow the user to focus between multiple audio sources.

Enhanced audio recording for smart pen computing systems

In a pen-based computing system, a printed version of a document having preexisting content is annotated using a smart pen. The smart pen captures handwriting gestures to obtain an electronic representation of the annotations. The smart pen computing system identifies a digital version of the document having the preexisting content and stores the electronic representation of the annotations in association with the digital document. The smart pen computing system may overlay the electronic representation of the annotations with the preexisting content to provide a digital representation of the annotated document.

Electronic annotation of documents with preexisting content

A pen based computing system concurrently captures handwriting gestures and records audio using binaural recording. A binaural headset communicatively coupled to the smart pen device uses at least two microphones. A left microphone is placed in or near the left ear and the right microphone is placed in or near the right ear, each facing outward. Speakers are integrated into a shared housing with the microphones facing inward towards the ear canal to play back the audio recordings. By recording audio with microphones placed close to the ears, the system provides realistic sounding playback and allows users to more easily differentiate between multiple sources of audio.

Binaural recording for smart pen computing systems

Interactive digital content for printed materials is provided using applications executing in a smart pen computing system. The applications and the printed materials may be developed by separate entities but appear well integrated from the perspective of a user of the smart pen computing system. Printed content produced by a first entity is obtained. The printed content is associated with a dot pattern. An application associated with the printed content and linked to the dot pattern is produced by a separate second entity. The application includes computer program instructions for, responsive to a selection within the dot pattern, invoking a function related to the printed content. The selection within the dot pattern may be made using a smart pen device, and the function may include presenting output to a user through an output device (e.g., a display or speaker) of the smart pen device.

Growing community of inventors

Nashville, TN, United States of America

Andy Van Schaack