Nex Team Inc.

City University of Hong Kong

The University of Hongkong

Qi Zhang

Keng Fai Lee

Wing Hung Chan

On Loy Sung

Man Hon Chan

Tin Yin Lam

Ian C Smith

Long Mak

Wang Fai Ng

Arron Mollet

Jorge S Fino

Bradford Lewis Farkas

David Richard Schmierer

Kin Ming Law

Alexander Wu

Daniel Dejos

Xinge Yu

Kening Zhu

Hongzhe Sun

Runming Wang

Antoni Bert Chan

Hongyan Li

Shing Fat Mak

Shaoyu Cai

Haichen Gao

Pak Leung Ho

Methods and systems for object trajectory reconstruction are disclosed. The methods and systems perform steps of first, receiving a first trajectory in a first image space and a user's location. Next, generating at least one trajectory line based on the user's location and the first trajectory, where the trajectory line can include at least one projection point representing a projection of the object's position. And finally, for the trajectory line, determining an associated second trajectory by determining, for the projection point in the trajectory line, a distance between the object and a plane. Some embodiments of the present invention enable a resource-limited mobile device, such as a smartphone, to efficiently perform this method. Also disclosed are benefits of the new methods, and alternative embodiments of implementation.

Methods and systems for object trajectory reconstruction

A computer-implemented method for video processing is disclosed. The method includes receiving an input video of one or more persons from a camera; detecting a sequence of human poses in the input video using an artificial intelligence (AI) based technique; selecting a proper pose from among multiple poses in a given frame of the input video, to generate a sequence of proper poses; detecting one or more key points in the sequence of proper poses; computing changes in coordinates of the one or more key points; computing a function of the changes in the coordinates of the one or more key points in the sequence of proper poses; counting a given user movement as a repetitive motion of an activity based on the function; and computing a plurality of statistics about the activity based on the counting. In some embodiments, the activity is running, jogging, walking jumping, performing jumping jacks, squatting, and/or dribbling.

Video-based motion counting and analysis systems and methods for virtual fitness application

Disclosed are pharmaceutical compositions containing (a) polymyxin antibiotic and (b) a mobile colistin resistant proteins (MCRs) inhibitor. The inhibitor relates to silver/gold-compounds or the pharmaceutically acceptable salts thereof. Also disclosed are methods of making silver/gold-compounds or the pharmaceutically acceptable salts thereof and methods for treating MCR-producing bacterial infection via multiple mechanisms.

Silver-/gold-compounds and methods thereof

Methods and systems are disclosed for pose comparison, interactive physical gaming, and remote fitness training on a user computing device. The methods and systems are configured to first receive a reference feature generated from a frame of a reference video, the reference feature computed from a reference posture of a reference person in the frame of the reference video. Next, receive a frame of a user video, the frame of the user video comprising a user. Next, extract a user posture from the frame of the user video, by performing a machine learning-based computer vision algorithm that detects one or more body key points of the user in an image plane of the user video. Finally, generate a user feature from the user posture; and determine an output score based on a distance between the reference feature and the user feature.

Pose comparison systems and methods using mobile computing devices

A wearable haptic modulation device for real-time simulation of texture properties of a material surface in mixed reality environments is provided. Particularly, it includes a U-shaped pneumatic actuator surrounds a user's fingertip, a finger-housing support creating a space for fingertip containment, a resonant actuator on the finger-housing support generating vibrotactile feedback to enhance perceived roughness on touched physical surfaces, and a pneumatic control system generating pneumatic actuation to reduce the perceived roughness on touched physical surface. Notably, the U-shaped pneumatic actuator allows direct contact of the user's fingerpad with the touched physical surface.

Wearable haptic devices for modulating perceived roughness of physical surfaces in real-time simulation in mixed reality environments

Methods and systems for remote multiplayer gameplay are disclosed. An example method can include joining a multiplayer interactive physical game including first and second players competing in an activity; capturing a first game video of a first player in a first play of the multiplayer interactive physical game, using a first camera on a first mobile computing device in a first gaming area; determining a first object flow associated with the first player from the first game video, by performing an algorithm on one or more frames of the first game video to detect one or more key points of the first player; generating a first analytic based on the first object flow; receiving a second analytic associated with the second player in a second gaming area; and generating a feedback to the first player based on the first analytic or the second analytic.

Remote multiplayer interactive physical gaming with mobile computing devices

A computer implemented method for facilitating training of body-eye coordination using a computing device having access to a camera is disclosed. The method includes receiving a training video of a player from the camera; superimposing a visual cue onto the training video; extracting a body posture flow of the player from the training video by performing a computer vision algorithm on one or more frames of the training video; determining whether the player has responded to the visual cue by analyzing the body posture flow of the player; and generating a feedback to the player in response to determining that the player has responded to the visual cue. Multi-player embodiments of the present invention are also disclosed.

Methods and systems for facilitating interactive training of body-eye coordination and reaction time

Methods and systems for performing location and movement determination in a three-dimensional environment using a device with a camera are disclosed. The methods and systems perform the steps of capturing an image of a user using the camera of the computing device, identifying the user from the image using a first algorithm, determining a first physical information associated with the user using a first algorithm, determining a second physical information of the user based on the first physical information and an input parameter of the user using a search process, and determining the location of the user in the environment based on the first physical information and the second physical information.

User analytics using a camera device and associated systems and methods

A system and method for implementing a virtual fitness application are disclosed. Embodiments may be built for various platforms, including web browsers and mobile phones. One embodiment uses pose estimation to count the repetitive motions of a fitness activity (e.g., running). In one embodiment, to count the repetitive motions, a repetitive motion counting process is used that is based on computing differences of Y-coordinates of key points of the user. The repetitive motion counting process includes selecting a proper pose of the user; computing one or more delta values (corresponding to Y-coordinate changes of the key points); and counting a given user movement as a repetitive motion based on a function (e.g., average) of the delta values. One embodiment uses the pose estimation and repetitive motion count results to provide a gamified experience, for example, a leaderboard, a stats report, instant visual feedback, badges, coins, and a social experience.

Methods and systems for image clustering are described, and include first receiving a plurality of images from a video of a game. Next, determining a first feature vector of a first player from the plurality of images by inputting the plurality of images to an artificial intelligence (AI)-based process. Next, dividing the plurality of images into a first subset of images and a second subset of images based on the first feature vector. Next, receiving a first identifier tagging the first player in a first image of the first subset. Finally, identifying a second player in a second image in the second subset by propagating the first identifier of the first subset, based on a distance measure associated with the first feature vector from the artificial intelligence (AI)-based process. Running on a mobile computing device, the method allows multiplayer tagging to be easily performed in almost any environment.

Methods and systems for multiplayer tagging using artificial intelligence

Methods and systems for multi-player training of body-eye coordination using mobile computing devices each having a camera are provided. The methods and systems execute instructions to capture a first training video of a first of player, using a first camera on a first mobile computing device; superimpose a visual cue onto the first training video at a visual cue location and for a cue period starting from a first time instant; determine whether the first player has responded to the visual cue at a second time instant within the cue period; receive a notification from a second mobile computing device, where the notification was generated in response to determining that a second player has responded to the visual cue at a third time instant within the cue period; and in response to a determination that the first player has responded to the visual cue and to the notification, generate a feedback to the first player.

Interactive training of body-eye coordination and reaction times using multiple mobile device cameras

Methods and systems for real-time generation of ball shot analytics are disclosed. The methods and systems perform steps for ball and posture detection, ball and posture flow generation, shot event identification and classification, and shot analytics generation based on identified shot events and shooter posture flow. Embodiments of the present invention use computer vision techniques to enable a resource-limited mobile device such as a smartphone to conduct the aforementioned steps. Also disclosed are benefits of the new methods, and alternative embodiments of implementation.

Methods and systems for generating sports analytics with a mobile device

Methods and systems for determining a ball shot attempt location on a ball court are disclosed. The method includes receiving an input video, where the input video includes the ball shot attempt and one or more players; receiving a declared ball shot attempt; determining a potential shooter by detecting a moving object flow in the input video; determining a potential shooter image region in the image plane by analyzing an overlapping image area of the ball trajectory and the potential shooter; determining a shooter foot-on-court time; determining a most probable pair of shooter legs in the potential shooter image region; determining a shooter foot location based on the most probable pair of shooter legs; and determining a ball shot attempt location on the ball court by applying a camera projection to the shooter foot location.

Methods and systems for determining ball shot attempt location on ball court

Methods and systems for determining and classifying a number of repetitive motions in a video are described, and include the steps of first determining a plurality of images from a video, where the images are segmented from at least one video frame of the video. Next, performing a pose detection process on a feature of the images to generate one or more landmarks. Next, determining one or more principle component axes on points associated with a given landmark. Finally, determining at least one repetitive motion based on a pattern associated with a projection of the points onto the one or more principle components. In some embodiments, the disclosed methods can classify the repetitive motions to respective types. The present invention can be implemented for convenient use on a mobile computing device, such as a smartphone, for tracking exercises and similar repetitive motions.

Repetition counting and classification of movements systems and methods

Methods and systems for determining a ball shot attempt result, the ball shot attempt captured by a camera on a mobile computing device, are disclosed. The methods and systems perform steps comprising receiving an input video captured using the camera on the mobile computing device, wherein the input video comprises a ball shot attempt at a goal; receiving a declared ball shot attempt, the declared ball shot attempt detected by analyzing a ball trajectory in the input video; detecting an element movement associated with an element of the goal; and determining the ball shot attempt result based on the element movement and the ball trajectory. In some embodiments, the element of the goal is a net, and the declared ball shot attempt is detected by analyzing the ball trajectory in a ball shot attempt Region of Interest (ROI) within frames of the input video.

Growing community of inventors

Hong Kong, China

Qi Zhang