Abstract

High-order parametric models that include terms for feature interactions are applied to various data mining tasks, where ground truth depends on interactions of features. However, with sparse data, the high-dimensional parameters for feature interactions often face three issues: expensive computation, difficulty in parameter estimation and lack of structure. Previous work has proposed approaches which can partially resolve the three issues. In particular, models with fac-torized parameters (e.g. Factorization Machines) and sparse learning algorithms (e.g. FTRL-Proximal) can tackle the first two issues but fail to address the third. Regarding to unstructured parameters, constraints or complicated regularization terms are applied such that hierarchical structures can be imposed. However, these methods make the optimization problem more challenging. In this work, we propose Strongly Hierarchical Factorization Machines and ANOVA kernel regression where all the three issues can be addressed without making the optimization problem more difficult. Experimental results show the proposed models significantly outperform the state-of-the-art in two data mining tasks: cold-start user response time prediction and stock volatility prediction.

Original languageEnglish (US)
Pages729-737
Number of pages9
StatePublished - Jan 1 2018
Event2018 SIAM International Conference on Data Mining, SDM 2018 - San Diego, United States
Duration: May 3 2018May 5 2018

Other

Other2018 SIAM International Conference on Data Mining, SDM 2018
CountryUnited States
CitySan Diego
Period5/3/185/5/18

Fingerprint

Analysis of variance (ANOVA)
Factorization
Data mining
Parameter estimation
Learning algorithms

ASJC Scopus subject areas

  • Computer Science Applications
  • Software

Cite this

Guo, R., Alvari, H., & Shakarian, P. (2018). Strongly hierarchical factorization machines and ANOVA kernel regression. 729-737. Paper presented at 2018 SIAM International Conference on Data Mining, SDM 2018, San Diego, United States.

Strongly hierarchical factorization machines and ANOVA kernel regression. / Guo, Ruocheng; Alvari, Hamidreza; Shakarian, Paulo.

2018. 729-737 Paper presented at 2018 SIAM International Conference on Data Mining, SDM 2018, San Diego, United States.

Research output: Contribution to conferencePaper

Guo, R, Alvari, H & Shakarian, P 2018, 'Strongly hierarchical factorization machines and ANOVA kernel regression' Paper presented at 2018 SIAM International Conference on Data Mining, SDM 2018, San Diego, United States, 5/3/18 - 5/5/18, pp. 729-737.
Guo R, Alvari H, Shakarian P. Strongly hierarchical factorization machines and ANOVA kernel regression. 2018. Paper presented at 2018 SIAM International Conference on Data Mining, SDM 2018, San Diego, United States.
Guo, Ruocheng ; Alvari, Hamidreza ; Shakarian, Paulo. / Strongly hierarchical factorization machines and ANOVA kernel regression. Paper presented at 2018 SIAM International Conference on Data Mining, SDM 2018, San Diego, United States.9 p.
@conference{e07bb8ff22fc402f8c3e64e4cfc01e04,
title = "Strongly hierarchical factorization machines and ANOVA kernel regression",
abstract = "High-order parametric models that include terms for feature interactions are applied to various data mining tasks, where ground truth depends on interactions of features. However, with sparse data, the high-dimensional parameters for feature interactions often face three issues: expensive computation, difficulty in parameter estimation and lack of structure. Previous work has proposed approaches which can partially resolve the three issues. In particular, models with fac-torized parameters (e.g. Factorization Machines) and sparse learning algorithms (e.g. FTRL-Proximal) can tackle the first two issues but fail to address the third. Regarding to unstructured parameters, constraints or complicated regularization terms are applied such that hierarchical structures can be imposed. However, these methods make the optimization problem more challenging. In this work, we propose Strongly Hierarchical Factorization Machines and ANOVA kernel regression where all the three issues can be addressed without making the optimization problem more difficult. Experimental results show the proposed models significantly outperform the state-of-the-art in two data mining tasks: cold-start user response time prediction and stock volatility prediction.",
author = "Ruocheng Guo and Hamidreza Alvari and Paulo Shakarian",
year = "2018",
month = "1",
day = "1",
language = "English (US)",
pages = "729--737",
note = "2018 SIAM International Conference on Data Mining, SDM 2018 ; Conference date: 03-05-2018 Through 05-05-2018",

}

TY - CONF

T1 - Strongly hierarchical factorization machines and ANOVA kernel regression

AU - Guo, Ruocheng

AU - Alvari, Hamidreza

AU - Shakarian, Paulo

PY - 2018/1/1

Y1 - 2018/1/1

N2 - High-order parametric models that include terms for feature interactions are applied to various data mining tasks, where ground truth depends on interactions of features. However, with sparse data, the high-dimensional parameters for feature interactions often face three issues: expensive computation, difficulty in parameter estimation and lack of structure. Previous work has proposed approaches which can partially resolve the three issues. In particular, models with fac-torized parameters (e.g. Factorization Machines) and sparse learning algorithms (e.g. FTRL-Proximal) can tackle the first two issues but fail to address the third. Regarding to unstructured parameters, constraints or complicated regularization terms are applied such that hierarchical structures can be imposed. However, these methods make the optimization problem more challenging. In this work, we propose Strongly Hierarchical Factorization Machines and ANOVA kernel regression where all the three issues can be addressed without making the optimization problem more difficult. Experimental results show the proposed models significantly outperform the state-of-the-art in two data mining tasks: cold-start user response time prediction and stock volatility prediction.

AB - High-order parametric models that include terms for feature interactions are applied to various data mining tasks, where ground truth depends on interactions of features. However, with sparse data, the high-dimensional parameters for feature interactions often face three issues: expensive computation, difficulty in parameter estimation and lack of structure. Previous work has proposed approaches which can partially resolve the three issues. In particular, models with fac-torized parameters (e.g. Factorization Machines) and sparse learning algorithms (e.g. FTRL-Proximal) can tackle the first two issues but fail to address the third. Regarding to unstructured parameters, constraints or complicated regularization terms are applied such that hierarchical structures can be imposed. However, these methods make the optimization problem more challenging. In this work, we propose Strongly Hierarchical Factorization Machines and ANOVA kernel regression where all the three issues can be addressed without making the optimization problem more difficult. Experimental results show the proposed models significantly outperform the state-of-the-art in two data mining tasks: cold-start user response time prediction and stock volatility prediction.

UR - http://www.scopus.com/inward/record.url?scp=85048339148&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85048339148&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:85048339148

SP - 729

EP - 737

ER -