Samsung Electronics Co., Ltd.

Caleb Ryan Phillips

Iqbal Ismail Mohomed

Afsaneh Fazly

Zhiming Hu

Ning Ye

Allan Jepson

Timothy Paul Capes

Gary Leung

Krish Perumal

Minfan Zhang

Alborz Rezazadeh Sereshkeh

A method of real-time video event detection includes: obtaining, based on a natural language query, a query vector; performing multimodal feature extraction on a video stream to obtain a video vector, obtaining a similarity score by comparing the query vector to the video vector; comparing the similarity score to a predetermined threshold; and activating, based on the similarity score being above the predetermined threshold, an action trigger. The multimodal feature extraction is performed using a plurality of overlapping windows that include sequential frames of the video stream.

Method of live video event detection based on natural language queries, and an apparatus for the same

A method may include receiving, by a virtual assistant of a user device, an input from a user, the virtual assistant being based on software. The method may include obtaining, by the virtual assistant of the user device and via a sensor of the user device, audio information or video information of the user. The method may include determining, by the virtual assistant of the user device, an identity of the user based on the audio information or the video information of the user and a set of facial embeddings and speech embeddings that is correlated with the user, the set of facial embeddings and speech embeddings being generated using a facial embedding model, a speech embedding model, and a sound source localization model. The method may include performing, by the virtual assistant of the user device, an action based on the input and the identity of the user.

System and method for personalization in intelligent multi-modal personal assistants

Training a classifier using embeddings and building a latent space is disclosed. The embeddings may be based on weights in a trained machine learning model. Also, operation of the classifier to process video segments in real-time using the using the weights and the latent space is disclosed. The embeddings and the latent space allow the classification to be performed at an overall reduced dimensionality. The latent space is designed to efficiently scale with an increasing number of queries to permit fast search through the space. Embodiments permit real-time operation on video with dynamic features. The classifier reduces the bandwidth demand of video camera-equipped devices at a network edge by setting aside, accurately, non-informative video sequences rather than uploading video too many things over the network. Applications include security cameras, robots and augmented reality glasses.

System and method for language-guided video analytics at the edge

An apparatus for video searching, includes a memory storing instructions, and a processor configured to execute the instructions to split a video into scenes, obtain, from the scenes into which the video is split, one or more textual descriptors describing each of the scenes, encode the obtained one or more textual descriptors describing each of the scenes into a video scene vector of each of the scenes, encode a user query into a query vector having a same semantic representation as that of the video scene vector of each of the scenes into which the one or more textual descriptors describing each of the scenes are encoded, and identify whether the video scene vector of at least one among the scenes corresponds to the query vector into which the user query is encoded.

Apparatus for video searching using multi-modal criteria and method thereof

An apparatus for vision and language-assisted smartphone task automation, includes a processor configured to execute instructions to identify whether an utterance of a user refers to one among at least one existing automation script, and based on the utterance being identified to not refer to the one among the at least one existing automation script, capture one or more screenshots and one or more coordinates of a display, the one or more screenshots and the one or more coordinates corresponding to one or more interactions of the user with the display while the user is demonstrating a new automation script, perform object detection and optical character recognition on the captured one or more screenshots and the captured one or more coordinates to obtain user interface (UI) elements corresponding to the one or more interactions, and obtain the new automation script, based on the obtained UI elements.

Growing community of inventors

Toronto, Canada

Caleb Ryan Phillips