Hong Kong Applied Science and Technology Research Institute Company, Limited

Yat Cheung Ngai

Yin Yee Chan

Fu Tuen Leung

A method of aligning and overlaying an AR virtual object in an electronic 3D viewing environment, comprising: receiving a motion comprising: a surrounding real-world scene image containing the real-world object captured by a camera of the electronic 3D viewing environment, and a camera real-world pose; recognizing a reference feature for aligning the virtual object to the real-world object; recording the motion and its corresponding reference feature; estimating a real-world object pose using the camera real-world pose and the reference feature; refining the estimated real-world object pose by pose estimation refinement by features and pose estimation refinement by images; and rendering the virtual object according to the estimated and refined real-world object pose in the 3D viewing environment.

Apparatus and method for aligning virtual objects in augmented reality viewing environment

A method of generating an AR user manual in an electronic 3D viewing environment, comprising: recording a moving trajectory of the 3D viewing environment's optical sensor; receiving a landmark location information; executing an iterative target object pose estimation comprising: estimating an estimated target object pose from each of the optical sensor poses in the recorded moving trajectory and the landmark location; calculating an estimation error from a 3D model being arranged in the estimated target object pose and projected onto the target object in the real-world scene; calculating a mean of the estimation errors; and reiterating the iterative target object pose estimation to optimize the estimated target object pose for a minimum mean estimation error; if the minimum mean estimation error is within a predefined estimation error threshold, then rendering the AR user manual onto the target object according to the optimized estimated target object pose.

Apparatus and method for augmented reality user manual

A method of tracking a point of interest (POI) in an electronic three-dimensional (3D) viewing environment, comprising: capturing via an optical sensor and recording motions of an onsite user, wherein each motion comprises an image surrounding and a pose of the onsite user; displaying a snapshot to a remote user, wherein the snapshot is one of the recorded motions; receiving a POI indicator in the snapshot from the second user; estimating a 3D position of the POI in the electronic 3D viewing environment using the POI indicator data, a moving trajectory from each of the recorded motions to the snapshot, and an estimation of distance between the optical sensor to the POI center; and rendering and superimposing the POI indicator in the electronic 3D viewing environment to be displayed to the onsite user using the estimated 3D position of the POI, the moving trajectory, and the recorded motions.

Apparatus and method of three-dimensional interaction for augmented reality remote assistance

A method of generating an electronic 3D walkthrough environment of a targeted environment, comprising: capturing a panorama at a first location point in the targeted environment; measuring an initial orientation of the camera whenever a panorama is captured; moving through the targeted environment and at each location point, capturing a live image of surrounding, and measuring an orientation of the camera; determining whether a new panorama is required to be captured at the current location point at where the camera is with the current orientation of the camera, comprising: identifying a sub-image in the panorama last captured that matches the live image captured; and if the sub-image cannot be identified, then a new panorama is required; and generating the electronic 3D walkthrough environment by connecting the panoramas captured.

Apparatus and method of generating electronic three-dimensional walkthrough environment

An apparatus and method for performing real object detection and control using a Virtual Reality Head Mounted Display System, and more particularly, when the real object detection and control are detecting hand presence and control in free space, in a virtual reality environment, the control being, for example, gestures or movement, in the free space.

Apparatus and method for performing real object detection and control using a virtual reality head mounted display system

Real-time managing multiple display windows on a screen for multiple users each allowed to request reserving an area on the screen and each having a priority value indicating a level of right to use the screen is considered. In a method, the screen is partitioned into grid-area units. Upon receiving a request from a requesting user, determine a target window formed by one or more of these units to cover the area to be reserved. Whether the requesting user is permitted to use the target window is then determined. If permitted, the target window is allocated to the requesting user as a new window. If the target window overlaps any already-allocated display window, the already-allocated window is adjusted (resized/removed) to not overlap the target window. All the allocated windows are then tiled one by one in an order according to the users' priority values for maximizing the screen's usage percentage.

Growing community of inventors

Hong Kong, China

Yat Cheung Ngai