Elastic functional coding of human actions: From vector-fields to latent variables

Rushil Anirudh, Pavan Turaga, Jingyong Su, Anuj Srivastava

Research output: Chapter in Book/Report/Conference proceedingConference contribution

46 Citations (Scopus)

Abstract

Human activities observed from visual sensors often give rise to a sequence of smoothly varying features. In many cases, the space of features can be formally defined as a manifold, where the action becomes a trajectory on the manifold. Such trajectories are high dimensional in addition to being non-linear, which can severely limit computations on them. We also argue that by their nature, human actions themselves lie on a much lower dimensional manifold compared to the high dimensional feature space. Learning an accurate low dimensional embedding for actions could have a huge impact in the areas of efficient search and retrieval, visualization, learning, and recognition. Traditional manifold learning addresses this problem for static points in ℝn, but its extension to trajectories on Riemannian manifolds is non-trivial and has remained unexplored. The challenge arises due to the inherent non-linearity, and temporal variability that can significantly distort the distance metric between trajectories. To address these issues we use the transport square-root velocity function (TSRVF) space, a recently proposed representation that provides a metric which has favorable theoretical properties such as invariance to group action. We propose to learn the low dimensional embedding with a manifold functional variant of principal component analysis (mfPCA). We show that mf-PCA effectively models the manifold trajectories in several applications such as action recognition, clustering and diverse sequence sampling while reducing the dimensionality by a factor of ∼ 250×. The mfPCA features can also be reconstructed back to the original manifold to allow for easy visualization of the latent variable space.

Original languageEnglish (US)
Title of host publicationProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
PublisherIEEE Computer Society
Pages3147-3155
Number of pages9
Volume07-12-June-2015
ISBN (Print)9781467369640
DOIs
StatePublished - Oct 14 2015
EventIEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015 - Boston, United States
Duration: Jun 7 2015Jun 12 2015

Other

OtherIEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015
CountryUnited States
CityBoston
Period6/7/156/12/15

Fingerprint

Trajectories
Principal component analysis
Visualization
Invariance
Sampling
Sensors

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition

Cite this

Anirudh, R., Turaga, P., Su, J., & Srivastava, A. (2015). Elastic functional coding of human actions: From vector-fields to latent variables. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Vol. 07-12-June-2015, pp. 3147-3155). [7298934] IEEE Computer Society. https://doi.org/10.1109/CVPR.2015.7298934

Elastic functional coding of human actions : From vector-fields to latent variables. / Anirudh, Rushil; Turaga, Pavan; Su, Jingyong; Srivastava, Anuj.

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Vol. 07-12-June-2015 IEEE Computer Society, 2015. p. 3147-3155 7298934.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Anirudh, R, Turaga, P, Su, J & Srivastava, A 2015, Elastic functional coding of human actions: From vector-fields to latent variables. in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. vol. 07-12-June-2015, 7298934, IEEE Computer Society, pp. 3147-3155, IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, United States, 6/7/15. https://doi.org/10.1109/CVPR.2015.7298934
Anirudh R, Turaga P, Su J, Srivastava A. Elastic functional coding of human actions: From vector-fields to latent variables. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Vol. 07-12-June-2015. IEEE Computer Society. 2015. p. 3147-3155. 7298934 https://doi.org/10.1109/CVPR.2015.7298934
Anirudh, Rushil ; Turaga, Pavan ; Su, Jingyong ; Srivastava, Anuj. / Elastic functional coding of human actions : From vector-fields to latent variables. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Vol. 07-12-June-2015 IEEE Computer Society, 2015. pp. 3147-3155
@inproceedings{fae48b6c5fcb48b2ac93822a889c8299,
title = "Elastic functional coding of human actions: From vector-fields to latent variables",
abstract = "Human activities observed from visual sensors often give rise to a sequence of smoothly varying features. In many cases, the space of features can be formally defined as a manifold, where the action becomes a trajectory on the manifold. Such trajectories are high dimensional in addition to being non-linear, which can severely limit computations on them. We also argue that by their nature, human actions themselves lie on a much lower dimensional manifold compared to the high dimensional feature space. Learning an accurate low dimensional embedding for actions could have a huge impact in the areas of efficient search and retrieval, visualization, learning, and recognition. Traditional manifold learning addresses this problem for static points in ℝn, but its extension to trajectories on Riemannian manifolds is non-trivial and has remained unexplored. The challenge arises due to the inherent non-linearity, and temporal variability that can significantly distort the distance metric between trajectories. To address these issues we use the transport square-root velocity function (TSRVF) space, a recently proposed representation that provides a metric which has favorable theoretical properties such as invariance to group action. We propose to learn the low dimensional embedding with a manifold functional variant of principal component analysis (mfPCA). We show that mf-PCA effectively models the manifold trajectories in several applications such as action recognition, clustering and diverse sequence sampling while reducing the dimensionality by a factor of ∼ 250×. The mfPCA features can also be reconstructed back to the original manifold to allow for easy visualization of the latent variable space.",
author = "Rushil Anirudh and Pavan Turaga and Jingyong Su and Anuj Srivastava",
year = "2015",
month = "10",
day = "14",
doi = "10.1109/CVPR.2015.7298934",
language = "English (US)",
isbn = "9781467369640",
volume = "07-12-June-2015",
pages = "3147--3155",
booktitle = "Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition",
publisher = "IEEE Computer Society",

}

TY - GEN

T1 - Elastic functional coding of human actions

T2 - From vector-fields to latent variables

AU - Anirudh, Rushil

AU - Turaga, Pavan

AU - Su, Jingyong

AU - Srivastava, Anuj

PY - 2015/10/14

Y1 - 2015/10/14

N2 - Human activities observed from visual sensors often give rise to a sequence of smoothly varying features. In many cases, the space of features can be formally defined as a manifold, where the action becomes a trajectory on the manifold. Such trajectories are high dimensional in addition to being non-linear, which can severely limit computations on them. We also argue that by their nature, human actions themselves lie on a much lower dimensional manifold compared to the high dimensional feature space. Learning an accurate low dimensional embedding for actions could have a huge impact in the areas of efficient search and retrieval, visualization, learning, and recognition. Traditional manifold learning addresses this problem for static points in ℝn, but its extension to trajectories on Riemannian manifolds is non-trivial and has remained unexplored. The challenge arises due to the inherent non-linearity, and temporal variability that can significantly distort the distance metric between trajectories. To address these issues we use the transport square-root velocity function (TSRVF) space, a recently proposed representation that provides a metric which has favorable theoretical properties such as invariance to group action. We propose to learn the low dimensional embedding with a manifold functional variant of principal component analysis (mfPCA). We show that mf-PCA effectively models the manifold trajectories in several applications such as action recognition, clustering and diverse sequence sampling while reducing the dimensionality by a factor of ∼ 250×. The mfPCA features can also be reconstructed back to the original manifold to allow for easy visualization of the latent variable space.

AB - Human activities observed from visual sensors often give rise to a sequence of smoothly varying features. In many cases, the space of features can be formally defined as a manifold, where the action becomes a trajectory on the manifold. Such trajectories are high dimensional in addition to being non-linear, which can severely limit computations on them. We also argue that by their nature, human actions themselves lie on a much lower dimensional manifold compared to the high dimensional feature space. Learning an accurate low dimensional embedding for actions could have a huge impact in the areas of efficient search and retrieval, visualization, learning, and recognition. Traditional manifold learning addresses this problem for static points in ℝn, but its extension to trajectories on Riemannian manifolds is non-trivial and has remained unexplored. The challenge arises due to the inherent non-linearity, and temporal variability that can significantly distort the distance metric between trajectories. To address these issues we use the transport square-root velocity function (TSRVF) space, a recently proposed representation that provides a metric which has favorable theoretical properties such as invariance to group action. We propose to learn the low dimensional embedding with a manifold functional variant of principal component analysis (mfPCA). We show that mf-PCA effectively models the manifold trajectories in several applications such as action recognition, clustering and diverse sequence sampling while reducing the dimensionality by a factor of ∼ 250×. The mfPCA features can also be reconstructed back to the original manifold to allow for easy visualization of the latent variable space.

UR - http://www.scopus.com/inward/record.url?scp=84959193616&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84959193616&partnerID=8YFLogxK

U2 - 10.1109/CVPR.2015.7298934

DO - 10.1109/CVPR.2015.7298934

M3 - Conference contribution

AN - SCOPUS:84959193616

SN - 9781467369640

VL - 07-12-June-2015

SP - 3147

EP - 3155

BT - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

PB - IEEE Computer Society

ER -