Microsoft Technology Licensing, LLC

Future Wei Technologies, Inc.

Wenjun Zeng

Cuiling Lan

Junliang Xing

Chong Luo

Shipeng Li

Feng Wu

Wei Liu

Jun Tian

Hong Heather Yu

Tao Mei

Zhiwei Xiong

Lina Dong

Wentao Zhu

Yanghao Li

Suman Deb Roy

Yingnan Zhu

Lizhi Wang

Dongliang He

Sijie Song

Wenxuan Xie

Various implementations of the subject matter described herein relate to a neural network-based action detection. There is provided an action detection scheme using a neural network. The action detection scheme can design and optimize the neural network model based on respective importance of different frames such that frames that are more important or discriminative for action recognition tend to be assigned with higher weights and frames that are less important or discriminative for action recognition tend to be assigned with lower weights.

Neural network-based action detection

In accordance with implementations of the subject matter described herein, a hybrid digital-analog coding scheme is proposed. In general, in accordance with implementations of the subject matter described herein, digital or analog encoding is selected at a level of chunks of a frame rather than at a level of the whole frame. Further, the encoding of a chunk is based on an expected distortion to be caused by analog transmission of the chunk over a communication channel, the distortion being estimated based on a constraint on available transmission resources over the transmission channel.

Hybrid digital-analog coding

In implementations of the subject matter described herein, an action detection scheme using a recurrent neural network (RNN) is proposed. Representation information of an incoming frame of a video and a predefined action label for the frame are obtained to train a learning network including RNN elements and a classification element. The representation information represents an observed entity in the frame. Specifically, parameters for the RNN elements are determined based on the representation information and the predefined action label. With the determined parameters, the RNN elements are caused to extract features for the frame based on the representation information and features for a preceding frame. Parameters for the classification element are determined based on the extracted features and the predefined action label. The classification element with the determined parameters generates a probability of the frame being associated with the predefined action label. The parameters for the RNN elements are updated according to the probability.

On-line action detection using recurrent neural network

High-speed hyperspectral (HSHS) video reconstruction is implemented in an imaging system. A hyperspectral snapshot is determined. A panchromatic video is determined corresponding to a frame time of the hyperspectral snapshot, the panchromatic video including a plurality of panchromatic frames. An HSHS video is generated by building a patch dictionary based at least in part on the panchromatic video and generating at least one frame of the HSHS video based at least in part on the patch dictionary, the panchromatic video, and the hyperspectral snapshot. A computing device can include a dictionary component configured to determine the patch dictionary and a reconstruction component configured to generate at least one HSHS frame based at least in part on the patch dictionary, the panchromatic video, and the hyperspectral snapshot.

Dual-sensor hyperspectral motion imaging system

In this disclosure, a hybrid digital-analog video coding scheme is described. For a video including at least two associated sequences of frames such as a stereo video, some of the frames are digitally encoded, while the others are encoded in an analog way with reference to the digital frames. For an analog frame, a previous frame in the same sequence/view and a temporally consistent frame in another sequence/view will be encoded as digital frames. These two digital frames are used to provide side information in encoding the analog frame. At decoding side, inverse operations are performed. Such 'zigzag' hybrid coding significantly improves the coding efficiency while providing robustness and good visual quality.

Hybrid digital-analog coding of stereo video

In implementations of the subject matter described herein, a current captured frame of a video is compared with a respective reference frame to determine a correlation therebetween. The correlation is used to indicate a change degree of the current frame. If the correlation for the current frame is below a predetermined threshold, the current frame may not be directly determined as including a motion of an object. Instead, correlations between one or more frames before or after the current frame and their respective reference frames are took into account. If the correlations of the frames under consideration are below the predetermined threshold, it may be detected that the current frame includes a motion of an object. In this way, incorrect detection of the object motion is reduced in the cases when larger changes in frames of a video are caused by factors such as noise and error, and the accuracy of the object motion detection is improved.

Motion detection of object

In implementations of the subject matter described herein, an action detection scheme using a recurrent neural network (RNN) is proposed. Joint locations for a skeleton representation of an observed entity in a frame of a video and a predefined action label for the frame are obtained to train a learning network including RNN elements and a classification element. Specifically, first weights for mapping the joint locations to a first feature for the frame generated by a first RNN element in a learning network and second weights for mapping the joint locations to a second feature for the frame generated by a second RNN element in the learning network are determined based on the joint locations and the predefined action label. The first and second weights are determined by increasing a first correlation between the first feature and a first subset of the joint locations and a second correlation between the second feature and the first subset of the joint locations. Based on the joint locations and the predefined action label, a parameter for a classification element included in the learning network is also determined by increasing a probability of the frame being associated with the predefined action label. The probability is generated by the classification element at least based on the first and second features.

Skeleton-based action detection using recurrent neural network

An apparatus comprising one or more processors configured to process a first image at a first resolution to generate a first solved image at the first resolution, determine a difference between the first solved image and a function of the first image, and generate a second solved image at a second resolution higher than the first resolution based at least in part on the difference between the first solved image and the function of the first image.

Delta interpolation for upsampling imaging solution

Some examples include receiving a microblog entry from a social stream domain. Further, some implementations include determining, based on a topic space associated with the social stream domain and a media domain, a topic that is associated with the microblog entry. Some implementations include determining, based on the topic space, one or more media items that are associated with the topic.

Cross-domain topic space

An apparatus comprising a first node configured to communicate with a second node, wherein the first node is configured to receive a first stream from the second node, receive a bridging stream from the second node, and receive the second stream from the second node, and wherein the bridging stream has a lower bit-rate than the second stream. Included is an apparatus comprising at least one processor configured to implement a method comprising transmitting a first stream at a first time, transmitting a bridging stream that has a bit-rate greater than the first stream and less than the second stream at a second time, and transmitting the second stream at a third time.

Growing community of inventors

Beijing, China

Wenjun Zeng