Telefonaktiebolaget Lm Ericsson (Publ)

AmirHossein Taher Kouhestani

José Araújo

Ioannis Karagiannis

Ananya Muddukrishna

Diego Gonzalez Morin

Lars M Andersson

Leonard Bruns

Volodya Grancharov

Lars Andersson

Simona Gugliermo

A device is disclosed that is configured to perform localization using one or both of a monocular camera and a depth sensor that are transportable with the device. The device includes at least one processor operationally connected to the monocular camera and the depth sensor. The device also includes at least one memory storing program code that is executed by the at least one processor to perform operations to receive image data from the monocular camera. The operations determine a benefit level of activating the depth sensor for localization, based on the image data, and activate the depth sensor for localization based on a determination that the benefit level of activating the depth sensor satisfies an activation rule. Related methods and computer program products are also disclosed.

Depth sensor activation for localization based on data from monocular camera

A method () for 3D modeling of one or more regions of interest. The method includes obtaining (s) information indicating that a person's gaze is fixed. Advantageously, the method further includes, in response to obtaining the information indicating that the person's gaze is fixed, initiating (s) an active scan of a region of interest (ROI), wherein the ROI is a region in which the person's gaze is directed.

Three-dimensional (3D) modeling

A method performed by a device for occlusion handling in augmented reality is provided. The device can generate at least one pixel classification image in a frame including an occluding object and having foreground, background, and unknown pixels. Generation of the at least one pixel classification image can include (1) calculating an initial foreground pixel probability image, and an initial background pixel probability image, and (2) calculating a normalized depth image based on depth information of the occluding object. The device can obtain an alpha mask to blend a virtual object and the foreground of the at least one pixel classification image based on determining a color of the unknown pixels. The device can render a final composition of an augmented reality image containing the virtual object occluded by the occluding object based on applying the alpha mask to pixels in the at least one pixel classification image.

Methods for handling occlusion in augmented reality applications using memory and device tracking and related apparatus

A mobile electronic device is provided for use with a headset. A camera outputs digital pictures of a portion of the headset. A display device displays information for viewing by a user wearing the headset. A processor retrieves calibration parameters that characterize at least a pose of the camera relative to the display device, and processes a digital picture from the camera to identify a pose of an optically identifiable feature within the digital picture. A pose of the mobile electronic device is identified relative to the holder based on the identified pose of the optically identifiable feature within the digital picture and based on at least the pose of the camera relative to the display device as characterized by the calibration parameters. The processor controls where graphical objects are rendered on the display device based on the identified pose of the mobile electronic device relative to the holder.

Calibration of mobile electronic devices connected to headsets wearable by users

A device is disclosed that is configured for performing localization using one or both of a depth sensor and a monocular camera that are transportable with the device. The device is adapted to receive depth data from the depth sensor, determine a benefit level of activating the monocular camera for localization, based on the depth data, and activate the monocular camera for localization based on a determination that the benefit level of activating the monocular camera satisfies an activation rule. Related methods and computer program products are also disclosed.

Monocular camera activation for localization based on data from depth sensor

A method of controlling a portable device including a camera, the method including: performing localization of the portable device using a localization algorithm having as an input image data representing images captured by the camera; identifying a current focus area of the camera; determining a desired focus area for the camera; determining that a change of focus is required when the desired focus area is different to the current focus area; and generating a control signal configured to cause the focus of the camera to change in respect of a new focus area.

Method of controlling a camera

A method for determining a transformation between a first coordinate system of an ultrasonic haptic device and a second coordinate system of a visual sensor device. The method includes: triggering generation of an ultrasonic focal point; obtaining images depicting a body part of a user while the ultrasonic focal point is active; determining a position of the ultrasonic focal point in the second coordinate system when the body part of the user is in the ultrasonic focal point; repeating the triggering, obtaining and determining; and calculating a first transformation between the first coordinate system and the second coordinate system based on the positions of the ultrasonic focal points in the first coordinate system and the second coordinate system to thereby calibrate a relationship between the ultrasonic haptic device and the visual sensor device.

Determining a transformation between coordinate systems in an ultrasonic haptic device and a visual sensor device

A method () of controlling a portable device comprising a first camera and a second camera facing in the same direction. The method comprises: selecting () one of the first camera and the second camera as a visualization camera; initializing () a localization algorithm having as an input image data representing images captured by one of the first camera and the second camera; determining () a respective focus score for at least one of the first camera and the second camera, said focus score indicating a focus quality of features identified from images captured by one of the respective camera; selecting (,) one of the first camera and the second camera as an enabled camera based on the at least one focus score; and generating a control signal configured to cause the selected camera to be enabled such that the image data representing images captured by the enabled camera are provided as the input to the localization algorithm.

Method of controlling a portable device and a portable device

A portable electronic device is used with a mixed reality (MR) headset. The MR headset includes a lens and a holder that retains the portable electronic device in a defined orientation relative to the lens. The portable electronic device includes a camera, a display device, and a processor. The camera outputs video frames and is arranged to view the lens. The display device is arranged to display information that is projected on the lens for reflection directly or indirectly toward the user's eyes and the camera. The processor performs operations that include processing the video frames from the camera to identify locations of at least one real-world feature, displaying information on the display device, and controlling at least one of the processing of the video frames and the displaying of the information on the display device to at least partially reduce occurrence in the video frames of the displayed information.

Portable electronic device for mixed reality headset

A portable electronic device is used with a mixed reality headset. The MR headset includes a lens and a holder that retains the portable electronic device in a defined orientation relative to the lens. The portable electronic device includes a camera, a display device, and a processor. The camera outputs video frames and is arranged to view the lens. The display device is arranged to display information that is projected on the lens for reflection directly or indirectly toward the user's eyes and the camera. The processor performs operations that include processing the video frames from the camera to identify locations of at least one real-world feature, displaying information on the display device, and controlling at least one of the processing of the video frames and the displaying of the information on the display device to at least partially reduce occurrence in the video frames of the displayed information.

Growing community of inventors

Solna, Sweden

AmirHossein Taher Kouhestani