Sportvision, Inc.

Sportsvision, Inc.

Vidya Elangovan

Richard H Cavallaro

Timothy P Heidmann

Kenneth Alan Milnes

Marvin S White

Marlena Fecho

James Orville McGuffin

Alina Alt

Reuben Halper

Video frames of a baseball game are analyzed to determine a track for the participants in the game, e.g., players, coaches and umpires, and to update a digital record of the game. The start and/or end of a half-inning can be indicated by factors such as: the participants leaving the dugout region and entering the playing field, including the outfield, a participant leaving the dugout region and entering the region of a base coach's box, a pitcher throwing pitches when no batter is present, and players on the playing field throwing the ball back and forth to one another. In one approach, the combination of detecting an event which is known to occur most often or always between half-innings followed by detecting another event which is known to occur most often or always during a half-inning, can be used to signal that a half-inning has started.

Video tracking of baseball players to determine the end of a half-inning

Video frames of a baseball game are analyzed to determine a track for the participants in the game and to update a digital record of the game. The merging of participants in a video frame is resolved by associating the participants' tracks before and/or after the merging with a most likely participant role, such as a player, coach or umpire role. The role of one merged participant can be used to deduce the role of the other merged participant. In this way, the digital record can be completed even for the merged period. The role of a participant can be based, e.g., on the location of the participant relative to a base, a coach's box region, a pitcher's mound, a dugout, or a fielding position, or by determining that a participant is running along a path to a base or performing some other movement.

Video tracking of baseball players which identifies merged participants based on participant roles

Video tracking of baseball players to determine the start and end of a half-inning

A video broadcast of a live event is enhanced by providing graphics in the video in real time to depict the fluid flow around a moving object in the event and to provide other informative graphics regarding aerodynamic forces on the object. A detailed flow field around the object is calculated before the event, on an offline basis, for different speeds of the object and different locations of other nearby objects. The fluid flow data is represented by baseline data and modification factors or adjustments which are based on the speed of the object and the locations of the other objects. During the event, the modification factors are applied to the baseline data to determine fluid flow in real time, as the event is captured on video. In an example implementation, the objects are race cars which transmit their location and/or speed to a processing facility which provides the video.

Providing graphics in images depicting aerodynamic flows and forces

An object is detected in images of a live event by storing and indexing templates based on representations of the object from previous images. For example, the object may be a vehicle which repeatedly traverses a course. A first set of images of the live event is captured when the object is at different locations in the live event. A representation of the object in each image is obtained, such as by image recognition techniques, and a corresponding template is stored. When the object again traverses the course, for each location, the stored template which is indexed to the location can be retrieved for use in detecting the object in a current image. The object's current location may be obtained from GPS data from the object, for instance, or from camera sensor data, e.g., pan, tilt and zoom, which indicates a direction in which the camera is pointed.

Detecting an object in an image using templates indexed to location or camera sensors

A representation of an object in an image of a live event is obtained by determining a color profile of the object. The color profile may be determined from the image in real time and compared to stored color profiles to determine a best match. For example, the color profile of the representation of the object can be obtained by classifying color data of the representation of the object into different bins of a color space, in a histogram of color data. The stored color profiles may be indexed to object identifiers, object viewpoints, or object orientations. Color data which is common to different objects or to a background color may be excluded. Further, a template can be used as an additional aid in identifying the representation of the object. The template can include, e.g., a model of the object or pixel data of the object from a prior image.

Identifying an object in an image using color profiles

A representation of an object in a live event is detected in an image of the event. A location of the object in the live event is translated to an estimated location in the image based on camera sensor and/or registration data. A search area is determined around the estimated location in the image. A direction of motion of the object in the image is also determined. A representation of the object is identified in the search area by detecting edges of the object, e.g., perpendicular to the direction of motion and parallel to the direction of motion, performing morphological processing, and matching against a model or other template of the object. Based on the position of the representation of the object, the camera sensor and/or registration data can be updated, and a graphic can be located in the image substantially in real time.

Detecting an object in an image using edge detection and morphological processing

A representation of an object in an image of a live event is detected by matching potential representation of the object against multiple types of templates. For example, the templates can include monochrome data, chrominance and/or luminance data, pixel data of the object from an earlier image, e.g., as a video template, an edge and morphology based template, a model of the object, or a predetermined static texture which is based on an appearance of the object. A weighting function may also be used. In one possible approach, a first type of template is used in an initial search area, and a second type of template is used in a smaller region of the initial search area. Based on a position of the optimum representation of the object in the image, a graphic can be provided in the image, or sensor and/or registration data of a camera can be updated.

Detecting an object in an image using multiple templates

An object is detected in images of a live event by storing and indexing camera registration-related data from previous images. For example, the object may be a vehicle which repeatedly traverses a course. A first set of images of the live event is captured when the object is at different locations in the live event. The camera registration-related data for each image is obtained and stored. When the object again traverses the course, for each location, the stored camera registration-related data which is indexed to the location can be retrieved for use in estimating a position of a representation of the object in a current image, such as by defining a search area in the image. An actual position of the object in the image is determined, in response to which the camera registration-related data may be updated, such as for use in a subsequent traversal of the course.

Detecting an object in an image using camera registration data indexed to location or camera sensors

Camera registration and/or sensor data is updated during a live event by determining a difference between an estimated position of an object in an image and an actual position of the object in the image. The estimated position of the object in the image can be based on an estimated position of the object in the live event, e.g., based on GPS or other location data. This position is transformed to the image space using current camera registration and/or sensor data. The actual position of the object in the image can be determined by template matching which accounts for an orientation of the object, a shape of the object, an estimated size of the representation of the object in the image, and the estimated position of the object in the image. The updated camera registration/sensor data can be used in detecting an object in a subsequent image.

Growing community of inventors

Sunnyvale, CA, United States of America

Vidya Elangovan