Inha University Research and Business Foundation

Other

Inha-industry Partnership Institute

Geun Sik Jo

Ui Nyoung Yoon

Seung Bo Park

Young Hoon Yu

Myung Duk Hong

Disclosed are a method and apparatus for converting 3D manuals into 2D interactive videos for a cloud service. The method for converting 3D manuals into 2D interactive videos for a cloud service, proposed in the present disclosure, includes the steps of performing labeling for annotations in a 3D manual for a cloud service and machine learning for recognizing the annotations; automatically recognizing the annotations, fine-adjusting automatically unrecognized annotations, and generating a file for converting the 3D manual into a 2D interactive video; linking the 3D manual to a corresponding 2D manual using the file; extracting and handling artificial intelligence knowledge based on work instructions in the 2D manual; and performing neural network tasks and linguistic inferences for processing voice requests, and informing the user about the handling and completion of a requested task by sending visual and audio feedback, in order to execute commands based on the artificial intelligence knowledge, the digital twin, and the annotations in the 3D manual.

Method and apparatus for converting 3D manuals into 2D interactive videos for cloud service

Aspects of the present invention provide an approach for reinforcement learning for unsupervised video summarization with piecewise linear interpolation. A candidate importance score is anticipated by training a video summarization network using a set of video frames in a video. The candidate importance score is interpolated with respect to other candidate importance scores in the video in order to generate a selection probably for each frame of the video. A video summarization that includes a set of selected frames is generated frame selection probability.

Unsupervised video summarization with piecewise linear interpolation

Disclosed are a method and apparatus for summarization of unsupervised video with efficient key frame selection reward functions. Frame-level visual features are extracted from an input video. An attention weight is computed and an importance score is represented as a frame tracking probability for selecting a key frame using the attention weight. A temporal consistency reward function and a representativeness reward function are obtained so as to select the key frame, based on a visual similarity distance and temporal distance between key frames, and an attention-based video summarization network is trained to predict an importance score for selecting a key frame of a video summary by using the temporal consistency reward function and the representativeness reward function. A video summary is created by selecting a corresponding key frame based on the predicted importance score, the quality of the created video summary is evaluated, and policy gradient learning is performed for the attention-based video summarization network. Regularization and reconstruction loss is calculated for controlling the probability to select a key frame by using the importance score of the selected key frame. A video summary is created based on the calculated regularization and reconstruction loss.

Method and apparatus for summarization of unsupervised video with efficient key frame selection reward functions

Approaches presented herein enable implementation of augmented reality in a smart glasses device. More specifically, two-dimensional (2-D) image data of a real-world object is collected from a 2-D camera of the smart glasses device. From the collected 2-D image data, 2-D coordinate data is generated. Based on location and rotation data of the smart glasses device and 2-D depth information from a viewing angle of the smart glasses device, three-dimensional (3-D) coordinates are generated from the generated 2-D coordinate data. An augmented object is displayed, on a holographic display of the smart glasses device, at an apparent location of the real-world object utilizing the 3-D coordinates.

Implementing three-dimensional augmented reality in smart glasses based on two-dimensional data

Approaches presented herein enable automatic creation of metadata for contents of a video. More specifically, a video and a script corresponding to the video are obtained. A location corresponding to an object in at least one shot of the video is extracted. This at least one shot includes a series of adjacent frames. The extracted location is saved as an annotation area in an annotation knowledge base. An element of a plot of the video is extracted from the script. This element of the plot is derived from content of the video in combination with content of the script. The extracted element of the plot is saved in a narrative knowledge base.

Automatic creation of metadata for video contents by in cooperating video and script data

Provided is a caption replacement service system and method for servicing an interactive advertisement on a video on demand (VOD). A system for making an annotation may include an annotation creator configured to create annotation information on a moving object included in a video and an annotation player configured to play a preview on the annotation information based on a frame unit or a shot unit of the video. The video and the annotation information may be used to provide an interactive advertisement by replacing a caption system of the video with a meaningful annotation.

Caption replacement service system and method for interactive service in video on demand

Embodiments of the present invention provide a client terminal (e.g., user terminal or advertiser terminal) for displaying a video annotated with advertisement information. Specifically, among other things, embodiments of the present invention provide a client terminal including a video player module configured to receive video based on a user request. The client terminal also further includes an advertising information search module configured to receive an advertising information overlay. The client terminal further includes an advertising information display module configured to simultaneously display the requested video and the advertising information overlay on a display of the user terminal.

System and method for annotating a video with advertising information

Embodiments of the present invention relate to unifying augmented reality technology and big data. An interactive operation element may be defined. The interactive operation element is associated with an event and a location on an augmented reality (AR) screen. An action may be performed based on the event using a predefined communication protocol. The action may be associated with an information artifact which is derived from big data.

Unifying augmented reality and big data

A system and method for making a semantic annotation for a moving object in an interactive video and an interface for the system is described. The annotation creating system may include an annotation creating unit adapted to create annotation information on a moving object contained in a video, and an annotation replaying unit adapted to execute a preview of the annotation information by the video frame or shot. The video and the annotation information may be registered in a platform for sharing the video.

System and method for making semantic annotation for objects in interactive video and interface for the system

Disclosed is a sampling method for estimating a homography matrix. This sampling method, for estimating a homography matrix that represents conversion correlations between pluralities of images by means of Constraint Satisfaction-Random Sample Consensus (CS-RANSAC), includes the steps of: sampling to divide an input image into a form of grids (N by N), select features, which are used for calculating a homography matrix, from features, which are abstracted from the input image, by means of a random sampling, and inspect whether the features selected by the random sampling satisfy predefined constraints; and executing model estimation to calculate the homography matrix from the features only if the selected features satisfy the constraints.

Sampling method and image processing apparatus of CS-RANSAC for estimating homography

Embodiments of the present invention provide a visual interlock system for providing product information by displaying video content annotated with virtual information (stored as metadata) such as advertisement information or a social network service (SNS) message from a friend. Specifically, among other things, embodiments of the present invention provide a system that allows a video playing on a television to be simultaneously played on a mobile device. The system searches for virtual information related to a visual object in the video. The virtual information is stored as metadata. The virtual object in the video is annotated with the virtual information. The annotated video is displayed on the user terminal.

Annotating an object in a video with virtual information on a mobile terminal

The present invention relates to a time slot allocation apparatus and method for preventing collisions between time slots in a TDMA-based RFID network. The time slot allocation apparatus allocates time slots, obtained by dividing a same frequency band into unit time slots, to RFID readers for reading identification codes of RFID tags () in the TDMA-based RFID network. A time slot allocation device () allocates time slots required for communication with the RFID tags to a plurality of RFID readers () provided in the RFID network. A collision prevention device () performs control such that the time slot allocation device allocates time slots to the plurality of RFID readers while distributing different and non-successive time slots to adjacent RFID readers, thus preventing collisions from occurring between time slots allocated to the RFID readers.

Growing community of inventors

Incheon, South Korea

Geun Sik Jo