International Business Machines Corporation

Michelle Yoonkyung Kim

Junehwa Song

Wlodek Wlodzimierz Zadrozny

Dimitri Kanevsky

David Nahamoo

Michael Alan Picheny

Asser Nasreldin Tantawi

Peter Hans Westerink

Marc Hubert Willebeek-LeMair

Edward Cholchin So

Yitzhak Dishon

Hamed A Ellozy

Judith L Klavans

Daniel Tunkelang

Leo Joskowicz

Lisa C Braden-Harder

Francisco Corella

Environments with unreliable delivery may result in faltering presentation of multimedia objects, due to missing time stamp deadlines. This may be alleviated by introducing more flexible time stamping. To avoid this, additional MPEG-4 object time information is sent to the client. This requires a new dedicated descriptor, carried in the Elementary Stream Descriptor. The new more flexible timing information will have two features. First, instead of fixed start and end times, the duration of an object can be given a range. And second, the start and end times are made relative to other multimedia object start and end times. This information can then be used by the client to adapt the timing of the ongoing presentation to the environment, while having more room to stay within the presentation author's intent and expectations.

Progressive adaptive time stamp resolution in multimedia authoring

A method and an apparatus adaptively and dynamically deliver multimedia content based on continuously monitored network conditions. Given the temporal relationships among the multimedia items in the multimedia data stream and also the current network condition, the method and apparatus compute a transmission schedule such that the transmission delay and the burstyness of the network bandwidth utilization are minimized, while maintaining the integrity of the multimedia presentation.

Method and apparatus for delivering multimedia content based on network connections

An authoring system and method create a knowledge base for an expert system that comprises communication cells for storing external data, assertion cells for storing data acquired from a user in the course of a user dialog, and interaction cells that specify (multimedia) interactions with the user, which are carried out as the cells are invoked by the run-time program. Each interaction cell has an enabling condition that must be satisfied before it can be invoked. At authoring time, the knowledge base is constructed by an expert system author who allocates the knowledge base cells on a rectangular grid similar to a spreadsheet. The author can try out various what-if scenarios by setting the values of assertion and communication cells and recalculating the knowledge base or simulating the created expert system. The grid is only an authoring tool, which is not visible or present at run time. A preferred application of the system is to create a health care expert system that will exchange information with a health care provider.

System and method for authoring an expert system

The design, creation, organization, and playing of multimedia stories is accomplished by first defining a story with episodes scheduled on a temporal layout. The organization of the story episodes is then separated into two parts: (1) a temporal organization of story episodes using temporal cliques, and (2) a spatial organization which spatially organizes the members of each temporal clique independently of the members of other temporal cliques. A complicated multimedia problem that exists in time and space is reduced to a number of smaller problems that exist only in space. Stories that are organized according this way can be related by a directed graph into a hyperstory. Using the directed graph, different stories in the hyperstory can be played by satisfying defined asynchronous conditions. The conditions are used in the graph to define the relationships among the stories in the graph.

Hyperstories: organizing multimedia episodes in temporal and spatial

A computer-implemented method of drawing a multimedia story including at least one episode is provided which represents a plurality of multimedia fries (e.g., text, sound, video, and picture files) graphically in a 'time-box' which can be connected to other time boxes to for a time box diagram. A user can easily stretch or shrink, reposition, delete, or otherwise manipulate the fries graphically using the time boxes in order to produce a final multimedia story within given temporal (time) constraints. The method according to the invention includes steps of inputting to a processor story data having at least one episode, and, for each episode, generating first and second events and temporal constraints. Thereafter, from the temporal constraints, first coordinates of each of the first and second events for each of the episodes are determined, and the first and second events are assigned to layers based on a temporal position of the first and second events such that for each episode the first and second events are temporally connected. Thereafter, an order of events on each layer of the layers is permuted, and, from the order of the events on each layer and the temporal constraints, second coordinates of each of the first and second events for each of the episodes are determined. Finally, a layout of the story is generated.

Multimedia document using time box diagrams

A knowledge based computing system is authored using a directed graph presented to the author at an authoring time. The directed graph has one or more question nodes and one or more answer nodes. The question nodes and the answer nodes connected to one another on one or more paths by links between each of the question nodes and each of the answer nodes so that question nodes and answer nodes alternate as the path is traversed in a path direction so that a 'QA directed graph' is created. Each question node associated with a question data structure and each answer node associated with a answer data structure that are populated with information accessed from the author using question and answer templates associated with respective question and answer nodes on the QA directed graph. After the expert system is created (authored) using the QA directed graph it can be run at a run time to provide and access information from a service user. The system has many applications particularly in the health care and teaching fields.

Authoring knowledge-based systems using interactive directed graphs

A computer implemented method reduces the complexities of obtaining a spatial design in a multimedia presentation. The method is an interactive process which assists an author in readily arriving at a suitable spatial design in the multimedia presentation. A set of objects that appear on the computer display screen simultaneously is termed a clique. The spatial design for a multimedia clique is defined in terms of three dimensions, two dimensions defining the plane of the computer display screen and the third dimension defining the depth of overlapping objects. The locations of the associated episodes and their respective depths on the screen are found. This process involves resolving space requirements between the several episodes in the clique according to a defined set of constraints. Assuming that a resolution can be reached which provides a consistent layout, the layout is displayed. The user is then given the opportunity to change the predefined set of constraints, resulting in the display of the layout with the changed set of constraints. If the layout is still not satisfactory, the user can change the sizes and locations of individual episodes in the clique. Based on the changed constraints for the episodes in the clique, the process again automatically determines a layout which is displayed for review and possible acceptance by the user. When the spatial design is finally accepted by the user, the process outputs the spatial design for the clique. Given a series of cliques which constitute an entire multimedia story, a technique of constraint propagation is provided. More particularly, an object in one clique may be displayed with other objects in a succeeding clique. Rather than moving that object around, the option is provided to fix the position of the object and let the other objects move around the fixed one.

Method for automatically obtaining spatial layout for multimedia

A multimedia system represents multimedia objects as ranges of time intervals, each bounded by a minimum and a maximum, and temporal relationships are given to a set of objects which are to be integrated. In this system, where there are uncertainties in time, a stretchable time-line is provided. The stretchable time-line is modeled after a spring system such that an object (or a spring) is associated not only with a minimum and a maximum length but also with a length at rest. As a spring rests at a certain length and stretches and shrinks by a certain degree when a force is applied, multimedia objects placed on the stretchable time-line may also rest at a certain length, and stretch or shrink if necessary. As a spring has a tendency to return to the length at rest, a multimedia object may stretch or shrink when necessary and by a smallest degree possible. The system according to the invention can answer a question like, 'Can I show this multimedia presentation in ten minutes, and if so, how should all the objects be scheduled?' If there is a solution that satisfies all the constraints given, the solution consists of a set of time intervals which 'minimally' deviate from the corresponding lengths at rest and also evenly distribute the variation of the difference between the optimum duration and the scheduled duration for each episode.

System and method for globally scheduling multimedia stories

A method of automatically aligning a written transcript with speech in video and audio clips. The disclosed technique involves as a basic component an automatic speech recognizer. The automatic speech recognizer decodes speech (recorded on a tape) and produces a file with a decoded text. This decoded text is then matched with the original written transcript via identification of similar words or clusters of words. The results of this matching is an alignment of the speech with the original transcript. The method can be used (a) to create indexing of video clips, (b) for 'teleprompting' (i.e. showing the next portion of text when someone is reading from a television screen), or (c) to enhance editing of a text that was dictated to a stenographer or recorded on a tape for its subsequent textual reproduction by a typist.

Automatic indexing and aligning of audio and text using speech

To archive information, a phrase or sentence describing the information, typically expressed in a natural language and conforming to the rules of a grammar (like a natural language grammar) is used to create a structured index which also conforms to the natural language grammar. The structured index has structure because the words in the index have a function and a relationship among each other as determined by the grammar. The index is combined with a location pointer of information to be cataloged, preferably multimedia information, to form a matched pair, i.e., a structured index and a segment (or a pointed to a multimedia object). The matched pair is stored in a database for later retrieval. Retrieval is based on parsing a query into a query structure that is identical or similar to the structured index. A heuristic interface presents the multimedia information along with a template in order to prompt the user into describing the multimedia information with a string of words so that they conform to the grammar.

Archiving and retrieving multimedia objects using structured indexes

A floor planning technique is provided herein wherein an initial floor plan having a predefined area and a number of overlapping modules is used to produce a final floor plan having the same predefined area and no overlapping modules. In order to eliminate the overlaps, a two stage process is performed. The first stage includes minimizing the overlaps by performing a series of move and/or reshape operations on the overlapping modules. The amount an overlapping module is moved and/or reshaped is based upon a computed net repelling force. After the move and/or reshape operations no longer minimize overlaps, the second stage is performed. The second stage, referred to as fitting, fits each overlapping module into the predefined area by awarding the overlapping area to the module or protruding the overlapping module into available space surrounding the module. The space that a module extends into is determined by calculating attracting forces. Once the second stage is complete, the resulting floor plan is legal having no overlapping modules.

Method and system for providing a non-rectangular floor plan

At least two direct access storage devices (DASDs), which are predetermined to record the same data from a central processing unit (CPU), are normally kept synchronzied with each other except during the power up phase. The DASD synchronization is controlled and maintained by synchronization control means independent of any commands from the CPU. When one or more commands such as SEEK and SET SECTOR or LOCATE commands are transferred from the CPU to a control unit over a single data transfer path between them, desired identical records on the synchronized DASDs are concurrently located while the DASDs are disconnected from the data transfer path. Upon locating the desired identical records, the DASDs are reconnected to the data transfer path. Then, a WRITE command is transferred from the CPU to the control unit for concurrently recording the same data onto the synchronized DASDs at the desired record locations. Thus, at the conclusion of this recording, when status information is reported to the host CPU, it is assured that multiple copies of the same data are recorded onto the synchronized DASDs.

Growing community of inventors

Scarsdale, NY, United States of America

Michelle Yoonkyung Kim