Controlling a 2d screen interface application in a mixed reality application

Abstract

The present invention relates to controlling a 2D application in a mixed reality application. In order to provide further simplification in terms of operating devices and equipment for the medical personnel, a device () for controlling a 2D application in a mixed reality application is provided. The device comprises a 2D application interface (), a processor (), a mixed reality application interface () and a control interface (). The 2D application interface receives data representative of a 2D application. The application comprises interactive elements providing commands for user control of the 2D application. The processor identifies and extracts the interactive elements based on the data representative of the 2D application. The processor also translates the interactive elements to user interface features of a mixed reality display application. The processor further generates data for presenting the mixed reality display application and maps user input to commands for user control of the 2D application. The mixed reality application interface provides the mixed reality display application to a user as mixed reality. The control interface receives input from the user for the user interface features and provides the commands to the 2D application in order to control the 2D application. Hence, augmented reality interfaces are mapped to traditional 2D screen interfaces. This allows to integrate applications that only provided the interaction on a monitor with a mouse or the like to be operated in today's augmented reality environment.