Koninklijke Philips Corporation N.V.

Hugo Strubbe

Srinivas Gutta

Larry James Eshelman

John G Milanski

Mi-Suen Lee

Daniel Pelletier

Tomas Brodsky

Antonio Colmenarez

Jorge Caviedes

Bartel Marinus Van De Sluis

Miroslav Trajkovic

Peter J Janssen

Eric Cohen-Solal

Mark Henricus Verberkt

Yun-Ting Lin

Vasanth Philomin

Mohamed S Abdel-Mottaleb

Michael D Pashley

Serhan Dagtas

Elmo M A Diederiks

Ahmed Elgammal

Max Wu

Soumitra Sengupta

Adrian Martel

Pelletier Daniel

James A Hoekema

Nermeen M Ismail

A system and apparatus are disclosed for modeling patterns of behavior of humans or other animate objects and detecting a violation of a repetitive pattern of behavior. The behavior of one or more persons is observed over time and features of the behavior are recorded in a multi-dimensional space. Over time, the multi-dimensional data provides an indication of patterns of human behavior. Activities that are repetitive in terms of time, location and activity, such as sleeping and eating, would appear as a Gaussian distribution or cluster in the multi-dimensional data. Probability distribution functions can be analyzed using known Gaussian or clustering techniques to identify repetitive patterns of behavior and characteristics thereof, such as a mean and variance. Deviations from repetitive patterns of behavior can be detected and an alarm can be triggered, if appropriate.

Method and apparatus for detecting an event based on patterns of behavior

A system and method for automatically controlling systems and devices in a local environment, such as a home. The system comprises a control unit that receives images associated with one or more regions of the local environment. The one or more regions are each serviced by one or more servicing components. The control unit processes the images to identify, from a group of known persons associated with the local environment, any one or more known persons located in the regions. For the regions in which one or more known person is identified, the control unit automatically generates a control signal for at least one of the servicing components associated with the region, the control signal reflecting a preference of at least one of the known persons located in the respective region.

System and method for adapting the ambience of a local environment according to the location and personal preferences of people in the local environment

A video conferencing system and method which automatically determines the appropriate preset camera parameters corresponding to participants participating in the video conference. A camera zooms out or pans the video conference space and looks for participants based on their faces. When a participant is detected, the preset camera parameters for that participant are calculated for when the center of the participant is in the center of the camera's view. This is continued for all the participants in the room. The optimal position for each participant and corresponding camera parameters are determined based on cultural preferences. Updates in the presets can be made periodically by the camera zooming out or panning the room. Multiple cameras can be used to continually update the presets.

Conferencing system and method for the automatic determination of preset positions corresponding to participants in video-mediated communications

An image processing system includes a combined display-camera having an array of interspersed display elements and camera elements, arranged substantially in a common plane of a flat panel or other display. Each of at least a subset of the camera elements has one or more imaging angles associated therewith, with the one or more imaging angles being selected to provide a desired imaging operation for the combined display-camera. The imaging angles of the camera elements can be selected to provide an imaging operation which approximates that of a lens-based single-camera system, a pin-hole camera system or other type of system. Each of the camera elements may include multiple image sensors, such that different imaging angles can be set for the different image sensors of a given camera element, and different perspectives of a scene can be generated in the image processing system.

Combined display-camera for an image processing system

An animated virtual creature is generated on a display such as a television set, in order to simulate a pet or to facilitate user interface functions. In one embodiment, one or more signals corresponding to the program being watched are monitored. When certain events are detected based on this monitoring, the animated character is controlled in a manner that corresponds to the detected event. In another embodiment, the animated virtual creature responds to program selections made by a user in a program recommendation system.

Virtual creature displayed on a television

Methods and apparatus are disclosed for predicting events using acoustic and visual cues. The present invention processes audio and video information to identify one or more (i) acoustic cues, such as intonation patterns, pitch and loudness, (ii) visual cues, such as gaze, facial pose, body postures, hand gestures and facial expressions, or (iii) a combination of the foregoing, that are typically associated with an event, such as behavior exhibited by a video conference participant before he or she speaks. In this manner, the present invention allows the video processing system to predict events, such as the identity of the next speaker. The predictive speaker identifier operates in a learning mode to learn the characteristic profile of each participant in terms of the concept that the participant 'will speak' or 'will not speak' under the presence or absence of one or more predefined visual or acoustic cues. The predictive speaker identifier operates in a predictive mode to compare the learned characteristics embodied in the characteristic profile to the audio and video information and thereby predict the next speaker.

Method and apparatus for predicting events in video conferencing and other applications

A home control system for automatic detection and warning of abnormal behavior includes a unit for observing behavior in a predetermined area under surveillance, a unit for processing an output of observed behavior from the unit for observing, and a pattern recognition module for recognizing whether the observed behavior is associated with predefined normal behaviors. The detection of predetermined normal behavior in progress leads to a provision of an anticipatory action. Upon recognition that the observed behavior is abnormal, an alarm signal is triggered to remind the user.

System and method of keeping track of normal behavior of the inhabitants of a house

A video processing system tracks a moving person or other object of interest using a combined audio-video tracking system. The audio-video tracking system comprises an audio locator, a video locator, and a set of rules for determining the manner in which settings of a camera are adjusted based on outputs of the audio locator and video locator. The set of rules may be configured such that only the audio locator output is used to adjust the camera settings if the audio locator and video locator outputs are not sufficiently close and a confidence indicator generated by the audio locator is above a specified threshold. For example, in such a situation, the audio locator output alone may be used to direct the camera to a new speaker in a video conference. If the audio locator and video locator outputs are sufficiently close, the system determines if a confidence indicator generated by the video locator is above a specified level, and if so, the video locator output may be used to adjust the camera settings. For example, the camera may be zoomed in such that the face of a video conference participant is centered in and occupies a designated portion of a video frame generated by the camera.

Method and apparatus for tracking moving objects using combined video and audio information in video conferencing and other applications

An interaction simulator, such as a chatterbot, is connected with an external database, such as an electronic program guide. The information gathered during interaction, particularly conversational, is parsed and used to augment the database data. The interaction simulator may be guided by the data residing in the database so as to help fill in recognizable gaps by, for example, intermittently asking questions relating to the subject data requirement. The interaction simulator may be provided with specific response templates based on the needs of the database and a corresponding set of templates to extract the information required by the database. Another example database may be for recording and indexing by key word stories or other free-form verbal data uttered by the user. The interaction simulator may be programmed to help the user develop the story using templates designed for this purpose.

User interface/entertainment device that simulates personal interaction and charges external database with relevant data

The appearance of a cabinet for a device (e.g., a television) is dynamically controlled. One way of controlling the appearance of the cabinet is by providing a display panel on the cabinet. At least one input is obtained from an input device and processed by a controller. Based on these inputs, the controller sends signals to the display panel to change the appearance of the display panel. Lamps and heat-sensitive materials may be used to effectuate the desired change in appearance. The cabinet display features may be controlled to give the device the appearance of having a persona or of being an independent animate being. For example, lights in the cabinet could flash to suggest the device is laughing in response to the detection of a laugh track in a sitcom being displayed on the device. Patterns and colors can be generated to reflect moods according to the preferences of an interaction-design programmer.

Device cabinet with dynamically controlled appearance

An electronic assistant, incorporates electronic functions in a personal object that the user finds indispensable. Examples of personal objects include a cane, a walking stick, a walker, a wheelchair, a personal transportation vehicle, a purse, a key holder, a watch, a pendant, a hearing aid, an eyeglass frame, or a crutch. Electronics may include a PDA, a cell phone, a navigation module, a biosensor module, and an emergency alert module. The navigation module might include a GPS, an altimeter, an electronic compass, and/or a stored map. Biosensors may be directly incorporated into the personal device or communicate with a receiver therein if the biosensor is required to be attached directly to the user's body or surgically implanted. The emergency alert module may include an emergency button, and an emergency notification sequence stored in the electronic assistant capable of calling for assistance through the cell phone.

Electronic assistant incorporated in personal objects

An interaction simulator uses computer vision, and inputs of other modalities, to analyze the user's mental state and/or personality. The mental state and/or personality are classified and this information used to guide conversation and other interaction. In a chatterbot embodiment, the substance of the conversation may be altered in response to the mental state and/or personality class, for example, by changing the topic of conversation to a favorite subject when the user is sad or by telling a joke when the user is in a good mood.

User interface/entertainment device that simulates personal interaction and responds to user's mental state and/or personality

An interaction simulator has the ability to dynamically update response templates and response data by retrieving current information from a data source. The interaction simulator may interact verbally or by other means such as gesture, typed text, or other means to generate a &ldquo;dialogue. &rdquo; The interaction simulator, by dynamically updating its responses, is able to be more interesting and useful to users and also more personally relevant than prior art, so-called chatterbots. The device also reduces the required number of rules and library of response data that must be retained compared to prior art conversation simulators. The criteria used to create new response data can come from fixed specifications, for example, the current weather, or from the content of previous conversations (for example, the system creating a new template about a topic indicated by the user to be of interest to him/her).

Self-updating user interface/entertainment device that simulates personal interaction

A system and method for controlling the movement of a cursor on a monitor screen are provided. The system comprising at least one remote control unit having a plurality of push buttons for remotely controlling the moving direction of the cursor on the monitor screen; at least one light emitting element for emitting light that indicates a signal generated by the remote control unit; a light detector for extracting the light movement that is transmitted sequentially from the remote control unit; and, a control unit for displaying the moving position of the cursor on the monitor screen corresponding to the extracted movement of the light from the remote control unit, and also adapted to stop the moving position of the cursor upon releasing the push button of the remote control unit. The movement of the cursor on the monitor screen also can be stopped if the light movement transmitted from the remote control unit changes in the opposite direction.

Remote control system and method for a television receiver

An interaction simulator, such as a chatterbot, is enabled to simulate an awareness of the user to generate an interaction that is more natural and appropriate than prior art chatterbots. For example, the device may employ machine vision to detect the number of persons present or the activity of the user and respond accordingly by interrupting its output or by inviting a conversation or other interaction when a user approaches. The device may modify its responses according to the user's activity, for example, by playing music when the user falls asleep or requesting an introduction when another user speaks. The device may also respond to unrecognized changes in the situation by inquiring about what is going on to stimulate interaction or generate new responses.

Growing community of inventors

Yorktown Heights, NY, United States of America

Hugo Strubbe