System and method for haptic interaction with virtual objects

Abstract

The invention relates to a system and a method for haptic interaction with virtual objects. The system of the invention is based on the idea that, for the haptic representation of any virtual three-dimensional object—synchronized with the visual representation—only the tangible haptic characteristics of the surface area of the object at the collision point or the area of contact are important. The haptic display system (morphling) of the invention that displays these characteristics comprises in principle three functional groups, which operate in synchronism with one another and with a visual subsystem. The individual subsystems can be operated statically or dynamically. In dynamic operation, users can change the position and/or shape, for example, of the tactile display by touching it. With regard to the real-time representation of the three-dimensional data of the user, at least the body part that is interacting with the object, e.g. the hand, must be represented in the visual system; the body part may also be directly visible, as with a head-up display. Ideally, all body parts of the user that lie within the user's field of view are displayed visually. With the synchronized real-time combination in the visual subsystem, users and objects that are actually spatially separated are displayed virtually in a defined spatial proximity. In this virtual representation, the user thus sees himself touching the virtual object. At the time of collision and at the collision point, however, he does not actually interact with the object represented in the visual system, but with a haptic display subsystem, the morphling, in a tactile subsystem which, in the tangible portion of the virtual object, displays the object's haptic characteristics, e.g. its surface condition and shape, synchronized in real time with the visual representation. The interaction between the at least one body part of the visualized user and the visualized object are represented in the visual subsystem simultaneously with the interaction of the user with the tactile display system, and upon collision of the at least one part of the visualized user with the visualized object, a collision point is determined. The three-dimensional data of the object at the collision point of the at least one part of the visualized user with the visualized object are presented in the tactile subsystem, the haptic element having a surface with structuring that is designed such that the haptic element displays the three-dimensional structure of the object at the collision point based on the captured three-dimensional data of the object, at least in the area of the collision point.