International Business Machines Corporation

Zhenfang Chen

Chuang Gan

Bo Wu

Dakuo Wang

Pin-Yu Chen

A system may include a memory and a processor in communication with the memory. The processor may be configured to perform operations. The operations may include receiving an input, extracting features from the input, and mining object relations using the features. The operations may include determining feature vectors using the object relations and generating, using the feature vectors, an output indicating a target region, wherein the target region corresponds to the input.

Image grounding with modularized graph attentive networks

According to one embodiment, a method, computer system, and computer program product for identifying one or more intrinsic physical properties of one or more objects is provided. The present invention may include identifying one or more objects in a video set, extracting observable physical properties of the identified one or more objects from the video set, including one or more trajectories, and inferring, by a property-based graph neural network, intrinsic properties of the one or more objects based on the trajectories.

Object-centric and relation-centric graph neural networks for physical property discovery

Mechanisms are provided for performing artificial intelligence-based video question answering. A video parser parses an input video data sequence to generate situation data structure(s), each situation data structure comprising data elements corresponding to entities, and first relationships between entities, identified by the video parser as present in images of the input video data sequence. First machine learning computer model(s) operate on the situation data structure(s) to predict second relationship(s) between the situation data structure(s). Second machine learning computer model(s) execute on a received input question to predict an executable program to execute to answer the received question. The program is executed on the situation data structure(s) and predicted second relationship(s). An answer to the question is output based on results of executing the program.

Neural-symbolic action transformers for video question answering

A processor may receive a video including a plurality of video frames in sequence and a question regarding the video. For a video frame in the plurality of video frames, a processor may parse the video frame into objects and relationships between the objects, and create a subgraph of nodes representing objects and edges representing the relationships, where parsing and creating are performed for each video frame in the plurality of video frames, where a plurality of subgraphs can be created. A processor may create a hypergraph connecting subgraphs by learning relationships between the nodes of the subgraphs, where a hyper-edge is created to represent a relationship between at least one node of one subgraph and at least one node of another subgraph in the plurality of subgraphs. A processor may generate an answer to the question based on the hypergraph.

Transformers for real world video question answering

One or more computer processors improve action recognition by removing inference introduced by visual appearances of objects within a received video segment. The one or more computer processors extract appearance information and structure information from a received video segment. The one or more computer processors calculate a factual inference (TE) for the received video segment utilizing the extracted appearance information and structure information. The one or more computer processors calculate a counterfactual debiasing inference (NDE) for the received video segment. The one or more computer processors calculate a total indirect effect (TIE) by subtracting the calculated counterfactual debiased inference from the calculated factual inference. The one or more computer processors action recognize the received video segment by selecting a classification result associated with a highest calculated TIE.

Growing community of inventors

Cambridge, MA, United States of America

Zhenfang Chen