Microsoft Technology Licensing, LLC

Mei Chen

Gaurav Mittal

Ye Yu

Victor Manuel Fragoso Rojas

Nikolaos Karianakis

Gabriel Takacs

Junwen Chen

Jedrzej Jakub Kozerawski

Zicheng Liu

Yinpeng Chen

Dongdong Chen

Lu Yuan

Chang Liu

Xiyang Dai

Mengchen Liu

Yunsheng Li

Jialin Yuan

Jay Sanjay Patravali

Example solutions for video object segmentation (VOS) use a bilateral attention transformer in motion-appearance neighboring space, and perform a process that includes: receiving a video stream comprising a plurality of video frames in a sequence; receiving a first object mask for an initial video frame of the plurality of video frames; selecting a video frame of the plurality of video frames as a current query frame, the current query frame following, in the sequence, a reference frame of a reference frame set, wherein each reference frame has a corresponding object mask; using the current query frame and a video frame in the reference frame set, determining a bilateral attention; and using the bilateral attention, generating an object mask for the current query frame.

Bilateral attention transformer in motion-appearance neighboring space for video object segmentation

A method of balancing a dataset for a machine learning model includes identifying confusing classes of few-shot classes for a machine learning model during validation. One of the confusing classes and an image from one of the few-shot classes are selected. An image perturbation is computed such that the selected image is classified as the selected confusing class. The selected image is modified with the computed perturbation. The modified selected image is added to a batch for training the machine learning model.

Image generation using adversarial attacks for imbalanced datasets

A computer device for automatic feature detection comprises a processor, a communication device, and a memory configured to hold instructions executable by the processor to instantiate a dynamic convolution neural network, receive input data via the communication network, and execute the dynamic convolution neural network to automatically detect features in the input data. The dynamic convolution neural network compresses the input data from an input space having a dimensionality equal to a predetermined number of channels into an intermediate space having a dimensionality less than the number of channels. The dynamic convolution neural network dynamically fuses the channels into an intermediate representation within the intermediate space and expands the intermediate representation from the intermediate space to an expanded representation in an output space having a higher dimensionality than the dimensionality of the intermediate space. The features in the input data are automatically detected based on the expanded representation.

Dynamic matrix convolution with channel fusion

Example solutions for video frame action detection use a gated history and include: receiving a video stream comprising a plurality of video frames; grouping the plurality of video frames into a set of present video frames and a set of historical video frames, the set of present video frames comprising a current video frame; determining a set of attention weights for the set of historical video frames, the set of attention weights indicating how informative a video frame is for predicting action in the current video frame; weighting the set of historical video frames with the set of attention weights to produce a set of weighted historical video frames; and based on at least the set of weighted historical video frames and the set of present video frames, generating an action prediction for the current video frame.

Video frame action detection using gated history

The disclosure herein describes preparing and using a cross-attention model for action recognition using pre-trained encoders and novel class fine-tuning. Training video data is transformed into augmented training video segments, which are used to train an appearance encoder and an action encoder. The appearance encoder is trained to encode video segments based on spatial semantics and the action encoder is trained to encode video segments based on spatio-temporal semantics. A set of hard-mined training episodes are generated using the trained encoders. The cross-attention module is then trained for action-appearance aligned classification using the hard-mined training episodes. Then, support video segments are obtained, wherein each support video segment is associated with video classes. The cross-attention module is fine-tuned using the obtained support video segments and the associated video classes. A query video segment is obtained and classified as a video class using the fine-tuned cross-attention module.

Leveraging unsupervised meta-learning to boost few-shot action recognition

A scale and pose estimation method for a camera system is disclosed. Camera data for a scene acquired by the camera system is received. A scale prior parameter characterizing scale of the camera system is received. A cost of a cost function is calculated for a similarity transformation. The cost of the cost function is influenced at least by the scale prior parameter. Based at least on the cost function being less than a threshold cost, an estimated scale and pose of the camera system is output based on the similarity transformation.

Prior informed pose and scale estimation

Providing a task-aware recommendation of hyperparameter configurations for a neural network architecture. First, a joint space of tasks and hyperparameter configurations are constructed using a plurality of tasks (each of which corresponds to a dataset) and a plurality of hyperparameter configurations. The joint space is used as training data to train and optimize a performance prediction network, such that for a given unseen task corresponding to one of the plurality of tasks and a given hyperparameter configuration corresponding to one of the plurality of hyperparameter configurations, the performance prediction network is configured to predict performance that is to be achieved for the unseen task using the hyperparameter configuration.

Task-aware recommendation of hyperparameter configurations

A scale and pose estimation method for a camera system is disclosed. Camera data for a scene acquired by the camera system is received. A rotation prior parameter characterizing a gravity direction is received. A scale prior parameter characterizing scale of the camera system is received. A cost of a cost function is calculated for a similarity transformation that is configured to encode a scale and pose of the camera system. The cost of the cost function is influenced by the rotation prior parameter and the scale prior parameter. A solved similarity transformation is determined upon calculating a cost for the cost function that is less than a threshold cost. An estimated scale and pose of the camera system is output based on the solved similarity transformation.

Growing community of inventors

Bellevue, WA, United States of America

Mei Chen