Elastic sequence correlation for human action analysis.

Abstract

This paper addresses the problem of automatically analyzing and understanding human actions from video footage. An "action correlation" framework, elastic sequence correlation (ESC), is proposed to identify action subsequences from a database of (possibly long) video sequences that are similar to a given query video action clip. In particular, we show that two well-known algorithms, namely approximate pattern matching in computer and information sciences and dynamic time warping (DTW) method in signal processing, are special cases of our ESC framework. The proposed framework is applied to two important real-world applications: action pattern retrieval, as well as action segmentation and recognition, where, on average, its run time speed (in matlab) is about 3.3 frames per second. In addition, comparing with the state-of-the-art algorithms on a number of challenging data sets, our approach is demonstrated to perform competitively.