Elastic Functional Coding of Riemannian Trajectories

Rushil Anirudh, Pavan Turaga, Jingyong Su, Anuj Srivastava

Research output: Contribution to journalArticle

12 Citations (Scopus)

Abstract

Visual observations of dynamic phenomena, such as human actions, are often represented as sequences of smoothly-varying features. In cases where the feature spaces can be structured as Riemannian manifolds, the corresponding representations become trajectories on manifolds. Analysis of these trajectories is challenging due to non-linearity of underlying spaces and high-dimensionality of trajectories. In vision problems, given the nature of physical systems involved, these phenomena are better characterized on a low-dimensional manifold compared to the space of Riemannian trajectories. For instance, if one does not impose physical constraints of the human body, in data involving human action analysis, the resulting representation space will have highly redundant features. Learning an effective, low-dimensional embedding for action representations will have a huge impact in the areas of search and retrieval, visualization, learning, and recognition. Traditional manifold learning addresses this problem for static points in the euclidean space, but its extension to Riemannian trajectories is non-trivial and remains unexplored. The difficulty lies in inherent non-linearity of the domain and temporal variability of actions that can distort any traditional metric between trajectories. To overcome these issues, we use the framework based on transported square-root velocity fields (TSRVF); this framework has several desirable properties, including a rate-invariant metric and vector space representations. We propose to learn an embedding such that each action trajectory is mapped to a single point in a low-dimensional euclidean space, and the trajectories that differ only in temporal rates map to the same point. We utilize the TSRVF representation, and accompanying statistical summaries of Riemannian trajectories, to extend existing coding methods such as PCA, KSVD and Label Consistent KSVD to Riemannian trajectories or more generally to Riemannian functions. We show that such coding efficiently captures trajectories in applications such as action recognition, stroke rehabilitation, visual speech recognition, clustering and diverse sequence sampling. Using this framework, we obtain state-of-the-art recognition results, while reducing the dimensionality/complexity by a factor of 100-250 ×. Since these mappings and codes are invertible, they can also be used to interactively visualize Riemannian trajectories and synthesize actions.

Original languageEnglish (US)
Article number7466117
Pages (from-to)922-936
Number of pages15
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume39
Issue number5
DOIs
StatePublished - May 1 2017

Fingerprint

Coding
Trajectories
Trajectory
Square root
Velocity Field
Dimensionality
Euclidean space
Nonlinearity
Action Recognition
Manifold Learning
Invariant Metric
Rehabilitation
Vector spaces
Speech Recognition
Feature Space
Stroke
Speech recognition
Patient rehabilitation
Invertible
Vector space

Keywords

  • activity recognition
  • dimensionality reduction
  • Riemannian geometry
  • visualization

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

Cite this

Elastic Functional Coding of Riemannian Trajectories. / Anirudh, Rushil; Turaga, Pavan; Su, Jingyong; Srivastava, Anuj.

In: IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, No. 5, 7466117, 01.05.2017, p. 922-936.

Research output: Contribution to journalArticle

Anirudh, Rushil ; Turaga, Pavan ; Su, Jingyong ; Srivastava, Anuj. / Elastic Functional Coding of Riemannian Trajectories. In: IEEE Transactions on Pattern Analysis and Machine Intelligence. 2017 ; Vol. 39, No. 5. pp. 922-936.
@article{950d23e398d54f5c8bbd94b56416db42,
title = "Elastic Functional Coding of Riemannian Trajectories",
abstract = "Visual observations of dynamic phenomena, such as human actions, are often represented as sequences of smoothly-varying features. In cases where the feature spaces can be structured as Riemannian manifolds, the corresponding representations become trajectories on manifolds. Analysis of these trajectories is challenging due to non-linearity of underlying spaces and high-dimensionality of trajectories. In vision problems, given the nature of physical systems involved, these phenomena are better characterized on a low-dimensional manifold compared to the space of Riemannian trajectories. For instance, if one does not impose physical constraints of the human body, in data involving human action analysis, the resulting representation space will have highly redundant features. Learning an effective, low-dimensional embedding for action representations will have a huge impact in the areas of search and retrieval, visualization, learning, and recognition. Traditional manifold learning addresses this problem for static points in the euclidean space, but its extension to Riemannian trajectories is non-trivial and remains unexplored. The difficulty lies in inherent non-linearity of the domain and temporal variability of actions that can distort any traditional metric between trajectories. To overcome these issues, we use the framework based on transported square-root velocity fields (TSRVF); this framework has several desirable properties, including a rate-invariant metric and vector space representations. We propose to learn an embedding such that each action trajectory is mapped to a single point in a low-dimensional euclidean space, and the trajectories that differ only in temporal rates map to the same point. We utilize the TSRVF representation, and accompanying statistical summaries of Riemannian trajectories, to extend existing coding methods such as PCA, KSVD and Label Consistent KSVD to Riemannian trajectories or more generally to Riemannian functions. We show that such coding efficiently captures trajectories in applications such as action recognition, stroke rehabilitation, visual speech recognition, clustering and diverse sequence sampling. Using this framework, we obtain state-of-the-art recognition results, while reducing the dimensionality/complexity by a factor of 100-250 ×. Since these mappings and codes are invertible, they can also be used to interactively visualize Riemannian trajectories and synthesize actions.",
keywords = "activity recognition, dimensionality reduction, Riemannian geometry, visualization",
author = "Rushil Anirudh and Pavan Turaga and Jingyong Su and Anuj Srivastava",
year = "2017",
month = "5",
day = "1",
doi = "10.1109/TPAMI.2016.2564409",
language = "English (US)",
volume = "39",
pages = "922--936",
journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence",
issn = "0162-8828",
publisher = "IEEE Computer Society",
number = "5",

}

TY - JOUR

T1 - Elastic Functional Coding of Riemannian Trajectories

AU - Anirudh, Rushil

AU - Turaga, Pavan

AU - Su, Jingyong

AU - Srivastava, Anuj

PY - 2017/5/1

Y1 - 2017/5/1

N2 - Visual observations of dynamic phenomena, such as human actions, are often represented as sequences of smoothly-varying features. In cases where the feature spaces can be structured as Riemannian manifolds, the corresponding representations become trajectories on manifolds. Analysis of these trajectories is challenging due to non-linearity of underlying spaces and high-dimensionality of trajectories. In vision problems, given the nature of physical systems involved, these phenomena are better characterized on a low-dimensional manifold compared to the space of Riemannian trajectories. For instance, if one does not impose physical constraints of the human body, in data involving human action analysis, the resulting representation space will have highly redundant features. Learning an effective, low-dimensional embedding for action representations will have a huge impact in the areas of search and retrieval, visualization, learning, and recognition. Traditional manifold learning addresses this problem for static points in the euclidean space, but its extension to Riemannian trajectories is non-trivial and remains unexplored. The difficulty lies in inherent non-linearity of the domain and temporal variability of actions that can distort any traditional metric between trajectories. To overcome these issues, we use the framework based on transported square-root velocity fields (TSRVF); this framework has several desirable properties, including a rate-invariant metric and vector space representations. We propose to learn an embedding such that each action trajectory is mapped to a single point in a low-dimensional euclidean space, and the trajectories that differ only in temporal rates map to the same point. We utilize the TSRVF representation, and accompanying statistical summaries of Riemannian trajectories, to extend existing coding methods such as PCA, KSVD and Label Consistent KSVD to Riemannian trajectories or more generally to Riemannian functions. We show that such coding efficiently captures trajectories in applications such as action recognition, stroke rehabilitation, visual speech recognition, clustering and diverse sequence sampling. Using this framework, we obtain state-of-the-art recognition results, while reducing the dimensionality/complexity by a factor of 100-250 ×. Since these mappings and codes are invertible, they can also be used to interactively visualize Riemannian trajectories and synthesize actions.

AB - Visual observations of dynamic phenomena, such as human actions, are often represented as sequences of smoothly-varying features. In cases where the feature spaces can be structured as Riemannian manifolds, the corresponding representations become trajectories on manifolds. Analysis of these trajectories is challenging due to non-linearity of underlying spaces and high-dimensionality of trajectories. In vision problems, given the nature of physical systems involved, these phenomena are better characterized on a low-dimensional manifold compared to the space of Riemannian trajectories. For instance, if one does not impose physical constraints of the human body, in data involving human action analysis, the resulting representation space will have highly redundant features. Learning an effective, low-dimensional embedding for action representations will have a huge impact in the areas of search and retrieval, visualization, learning, and recognition. Traditional manifold learning addresses this problem for static points in the euclidean space, but its extension to Riemannian trajectories is non-trivial and remains unexplored. The difficulty lies in inherent non-linearity of the domain and temporal variability of actions that can distort any traditional metric between trajectories. To overcome these issues, we use the framework based on transported square-root velocity fields (TSRVF); this framework has several desirable properties, including a rate-invariant metric and vector space representations. We propose to learn an embedding such that each action trajectory is mapped to a single point in a low-dimensional euclidean space, and the trajectories that differ only in temporal rates map to the same point. We utilize the TSRVF representation, and accompanying statistical summaries of Riemannian trajectories, to extend existing coding methods such as PCA, KSVD and Label Consistent KSVD to Riemannian trajectories or more generally to Riemannian functions. We show that such coding efficiently captures trajectories in applications such as action recognition, stroke rehabilitation, visual speech recognition, clustering and diverse sequence sampling. Using this framework, we obtain state-of-the-art recognition results, while reducing the dimensionality/complexity by a factor of 100-250 ×. Since these mappings and codes are invertible, they can also be used to interactively visualize Riemannian trajectories and synthesize actions.

KW - activity recognition

KW - dimensionality reduction

KW - Riemannian geometry

KW - visualization

UR - http://www.scopus.com/inward/record.url?scp=85018476593&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85018476593&partnerID=8YFLogxK

U2 - 10.1109/TPAMI.2016.2564409

DO - 10.1109/TPAMI.2016.2564409

M3 - Article

VL - 39

SP - 922

EP - 936

JO - IEEE Transactions on Pattern Analysis and Machine Intelligence

JF - IEEE Transactions on Pattern Analysis and Machine Intelligence

SN - 0162-8828

IS - 5

M1 - 7466117

ER -