Tianjin University

Ning Xu

Anan Liu

Weizhi Nie

Guokai Zhang

Yuting Su

Yongdong Zhang

Hongshuo Tian

Yanhui Wang

Dan Song

Lanjun Wang

The present disclosure discloses a cross-modal image-watermark joint generation and detection device and method thereof. The device includes a multimodal encoder, an image-watermark feature co-embedding module, an image-watermark feature fusion module, an up-sampling generator, a non-cooperative game decoupling module configured to decouple an unwatermarked image and a reconstructed watermark from a composite image through two decoders by developing allocation strategies according to a non-cooperative game theory and a Shannon information theory; a strategy allocation module configured to set an composite image discriminator, keeping the consistency between the composite image and input text by multi-specification down-sampling convolution kernels and set the objective functions to constrain watermark reconstruction and unwatermarked image decoding; and a post-processing attack module configured to simulate various attacks for ensuring the robustness of watermarks.

Cross-modal image-watermark joint generation and detection device and method thereof

The present disclosure discloses a visual relationship detection method based on adaptive clustering learning, including: detecting visual objects from an input image and recognizing the visual objects to obtain context representation; embedding the context representation of pair-wise visual objects into a low-dimensional joint subspace to obtain a visual relationship sharing representation; embedding the context representation into a plurality of low-dimensional clustering subspaces, respectively, to obtain a plurality of preliminary visual relationship enhancing representation; and then performing regularization by clustering-driven attention mechanism; fusing the visual relationship sharing representations and regularized visual relationship enhancing representations with a prior distribution over the category label of visual relationship predicate, to predict visual relationship predicates by synthetic relational reasoning. The method is capable of fine-grained recognizing visual relationships of different subclasses by mining latent relationships in-between, which improves the accuracy of visual relationship detection.

Visual relationship detection method and system based on adaptive clustering learning

The present invention discloses a visual relationship detection method based on a region-aware learning mechanism, comprising: acquiring a triplet graph structure and combining features after its aggregation with neighboring nodes, using the features as nodes in a second graph structure, and connecting in accordance with equiprobable edges to form the second graph structure; combining node features of the second graph structure with features of corresponding entity object nodes in the triplet, using the combined features as a visual attention mechanism and merging internal region visual features extracted by two entity objects, and using the merged region visual features as visual features to be used in the next message propagation by corresponding entity object nodes in the triplet; and after a certain number of times of message propagations, combining the output triplet node features and the node features of the second graph structure to infer predicates between object sets.

Visual relationship detection method and system based on region-aware learning mechanisms

Growing community of inventors

Tianjin, China

Ning Xu