Image Metrics Limited

Optasia Medical Limited

Kevin Walker

Michael Rogers

Gareth Edwards

Tomos G Williams

Jane Haslam

Anthony Holmes

Mike Rogers

Vincent Drouard

Martin Tosas

Mike Taylor

Steven Caulkin

Matthew Oakes

Salil Deena

Chris Bowles

Martin De La Gorce

Carl Moore

Martin DeLa Gorce

Steve Caulkin

In one or more implementations, body features of a subject can be tracked and a pose estimation produced using an unconstrained video sequence. The video sequence constitutes a physical record of the body features of the subject. A set of 2D coordinates corresponding to the body features of the subject can be received from a first neural network and 2D Keypoints are detected and transmitted to a second neural network. The second neural network can return corresponding depth estimation values, which can be processed to generate a set of 3D coordinates that correspond to the body features of the subject in camera space. Furthermore, one or more non-linear optimizations can be applied to fit a predetermined skeleton to the generate 3D Keypoints, and produce the pose estimation corresponding to the body features of the subject.

Real-time body pose estimation system and method in unconstrained video

A method for locating and tracking facial features in an unconstrained video sequence includes: in a face-detecting process, delineating, with region-bounding coordinates, the face of the subject within an image selected from the sequence; detecting, in the selected image, a small set of landmarks, corresponding to the face of the subject, using a convolutional neural network, trained to take as input an image region corresponding to the face of the&#x2014;subject and to return a set of coordinates at computational speeds approximating real time; projectively fitting a three-dimensional character model to the detected landmarks, and using the fitted model to estimate physical locations of additional landmarks, so as to provide a complete hypothesized set of facial landmarks; and in a feature tracker process, updating the hypothesized set of facial landmarks to improve convergence between predicted feature locations and their actual physical locations based on data sampled from the selected image.

Real-time tracking of facial features in unconstrained video

Computer-implemented methods for vending at least one cosmetic product to a first consumer. A real-time image of a portion of a body of the consumer is captured using a personal computational device and the consumer provides a desired makeup look via the personal computational device. The desired makeup look is then applied to an image of the portion of the body of the consumer, thereby generating a composite image of the portion of the body of the first consumer with the desired makeup look, and a retail order for the cosmetic product is received from the consumer. The order may be received before or after the consumer's image is captured. Another consumer, or a product specialist, may be involved in the aforesaid process via a computer communications link.

Monetization using video-based simulation of cosmetic products

Computer-implemented methods for real-time simulation of the application of one or more cosmetic products to a portion of the body of a user being imaged in an image-capture environment that is subject to uncertain lighting conditions. A real-time image of a portion of the user is captured and used to derive both image-capture environment parameters and user visual characteristics. Based on a desired look received from the user, a correspondence is extracted between the desired look and the pertinent portion of the user so that the desired look can be adapted to the image-capture environment parameters and the user's visual characteristics, and the adapted look is applied to an image of the user.

Adaptive, calibrated simulation of cosmetic products on consumer devices

Computerized methods for creating tracks of locations across frames of a video corresponding to a facial feature of a human. A set of feature location hypotheses is generated, as applied to images derived from the sequence of frames representing images of the human. Each hypothesis is refined, and a first set of confidence measures is associated with each hypothesis. A second set of confidence measures is associated with interframe transition, and a cost function that is a combination of hypotheses and transition confidence measures is minimized. A set of tracks is generated, characterizing each of a plurality of facial features within each frame of the sequence of frames. Performance analysis data may further be derived in a performance driven animation production pipeline, based on the generated tracks.

Optimal automatic capture of facial movements and expressions in video sequences

Computer implemented methods for generating a non-transient record of feature locations and/or facial expression parameters characterizing a person's face. A video sequence of a specified individual person is received and a feature locator update model is applied to the video sequence. The feature locator update model is derived by defining a set of training images, generating a set of facial feature displacements for each training image with associated image sample vectors, and training a regularized linear regression which maps from image sample vectors to displacement vectors, wherein the regularization includes a spatial smoothness term within the shape-free sample space. A feature location and/or a facial expression parameter is then extracted, based on the feature update model, characterizing the location, and/or the expression, of the feature of the face of the specified individual person.

Building systems for tracking facial features across individuals and groups

Computer implemented methods for generating a non-transient record of feature locations and/or facial expression parameters characterizing a person's face. A video sequence of a specified individual person is received and a feature locator update model is applied to the video sequence. The feature locator update model is derived by defining a set of training images, generating a set of facial feature displacements for each training image with associated image sample vectors, and training a regularized linear regression which maps from image sample vectors to displacement vectors, wherein the regularization includes a spatial smoothness term within the shape-free sample space. A feature location and/or a facial expression parameter is then extracted, based on the feature update model, characterizing the location, and/or the expression, of a selected set of features of the face of the specified individual person that correspond to an adaptive set of feature locations.

Building systems for adaptive tracking of facial features across individuals and groups

Computer-implemented methods and computer program products for automatically transferring expressions between rigs with consistent joint structure, and for automatically transferring skin weights between different skin meshes based on joint positioning. A method is provided for transferring an expression between a plurality of source rigs and a target rig, where each rig characterizes an animated character, and each rig, in turn, is characterized by a set of joints and a skin mesh having a plurality of vertices, with each vertex characterized by a matrix of weights relating a response of the vertex to movement of associated joints. A set of offsets is calculated of joint positions of a goal expression of each source rig relative to a neutral expression of the source rig. A scaling transformation is then applied to the set of offsets to produce a scaled set of offsets, which are added, in turn, to a neutral expression of the target rig. Methods are also provided for transferring a set of skin weights between the source rigs and the target rig.

Development tools for animated character rigging

Computer-implemented methods and computer program products for automatically transferring expressions between rigs with consistent joint structure, and for automatically transferring skin weights between different skin meshes based on joint positioning. A method is provided for transferring an expression between a source rig and a target rig, where each rig characterizes an animated character, and each rig, in turn, is characterized by a set of joints and a skin mesh having a plurality of vertices, with each vertex characterized by a matrix of weights relating a response of the vertex to movement of associated joints. A set of offsets is calculated of joint positions of a goal expression of the source rig relative to a neutral expression of the source rig. A scaling transformation is then applied to the set of offsets to produce a scaled set of offsets, which are added, in turn, to a neutral expression of the target rig. Methods are also provided for transferring a set of skin weights between a source rig and a target rig.

A method and system for characterizing a knee joint in terms of its skeletal morphology. A plurality of loci associated with a model of skeletal structure of a knee joint are fitted and used to parameterize positions of the plurality of loci in a given subject and, thereby, to derive parameters of a deformable statistical template. The skeletal morphology is then characterized on the basis of the derived parameters of the deformable statistical template.

Method and system for characterization of knee joint morphology

A method and system for driving a graphical object based on a performance of an actor in a video stream. A plurality of key frames are selected in a calibration video stream, allowing animation control values to be defined, in accordance with artistic intent, for each of a set of control variables, corresponding to each of the key frames. Features of the actor in the calibration video stream are parameterized so as to obtain a vector of values of parameters for each frame of the calibration video stream, and, then, an array of distance vectors is derived, characterizing a distance between each pair of vectors of values of parameters among the video stream frames. The space of distance vectors is mapped into the set of control variables according to a mapping operation which is then applied to distance vectors derived from an actual performance video stream to obtain a time sequence of animation control values. Finally, an animation is created on the basis of applying the sequence of animation control values to the graphical object.

Animating graphical objects using input video

Methods and computer program code computer program codes for modifying an expression of an actor in at least one video frame based on performance of the actor in a sequence of frames of video footage. A plurality of control points are identified for characterizing the shape of the actor in each frame of the video footage by a shape vector comprising positions of the control points. The shape vectors of each frame are then warped to a common standard frame thereby generating an aligned shape vector and a transformed image for each frame. The aligned shape vectors and attendant texture vectors are transformed in accordance with specified pose and expression parameters to create new shape and texture vectors which are reinserted into the sequence of frames so as to reanimate the actor.

Replacement of faces in existing video

A system and method for cleaning a set of motion capture data, wherein the data correspond to a set of point coordinates relating to a model but may include missing or inaccurate values. A subset of the motion capture data is identified that corresponds to valid points matching locations of corresponding points of the model, and a best set of model parameters is computed in such a manner as to minimize a norm of the valid points with respect to the model. A complete set of point coordinates is then predicted based on the best set of model parameters and a subsequent set of cleaned motion capture data. Actual points are inserted where validated, and the process is iterated until a specified criterion is met.

Growing community of inventors

Sheffield, United Kingdom

Kevin Walker