Accent identification by combining deep neural networks and recurrent neural networks trained on long and short term features

Yishan Jiao, Ming Tu, Visar Berisha, Julie Liss

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

Automatic identification of foreign accents is valuable for many speech systems, such as speech recognition, speaker identification, voice conversion, etc. The INTERSPEECH 2016 Native Language Sub-Challenge is to identify the native languages of non-native English speakers from eleven countries. Since differences in accent are due to both prosodic and articulation characteristics, a combination of long-term and short-term training is proposed in this paper. Each speech sample is processed into multiple speech segments with equal length. For each segment, deep neural networks (DNNs) are used to train on long-term statistical features, while recurrent neural networks (RNNs) are used to train on short-term acoustic features. The result for each speech sample is calculated by linearly fusing the results from the two sets of networks on all segments. The performance of the proposed system greatly surpasses the provided baseline system. Moreover, by fusing the results with the baseline system, the performance can be further improved.

Original languageEnglish (US)
Pages (from-to)2388-2392
Number of pages5
JournalUnknown Journal
Volume08-12-September-2016
DOIs
StatePublished - 2016

Fingerprint

Sodium Glutamate
Recurrent neural networks
Recurrent Neural Networks
Neural Networks
Term
Baseline
Voice Conversion
Speaker Identification
Language
Speech Recognition
Acoustics
Speech recognition
Linearly
Speech
Deep neural networks
Accent
Native Language
Train

Keywords

  • Accent identification
  • Articulation
  • Deep neural networks
  • Prosody

ASJC Scopus subject areas

  • Language and Linguistics
  • Human-Computer Interaction
  • Signal Processing
  • Software
  • Modeling and Simulation

Cite this

Accent identification by combining deep neural networks and recurrent neural networks trained on long and short term features. / Jiao, Yishan; Tu, Ming; Berisha, Visar; Liss, Julie.

In: Unknown Journal, Vol. 08-12-September-2016, 2016, p. 2388-2392.

Research output: Contribution to journalArticle

@article{9492d3638aae4215a6f3f62a45acdd04,
title = "Accent identification by combining deep neural networks and recurrent neural networks trained on long and short term features",
abstract = "Automatic identification of foreign accents is valuable for many speech systems, such as speech recognition, speaker identification, voice conversion, etc. The INTERSPEECH 2016 Native Language Sub-Challenge is to identify the native languages of non-native English speakers from eleven countries. Since differences in accent are due to both prosodic and articulation characteristics, a combination of long-term and short-term training is proposed in this paper. Each speech sample is processed into multiple speech segments with equal length. For each segment, deep neural networks (DNNs) are used to train on long-term statistical features, while recurrent neural networks (RNNs) are used to train on short-term acoustic features. The result for each speech sample is calculated by linearly fusing the results from the two sets of networks on all segments. The performance of the proposed system greatly surpasses the provided baseline system. Moreover, by fusing the results with the baseline system, the performance can be further improved.",
keywords = "Accent identification, Articulation, Deep neural networks, Prosody",
author = "Yishan Jiao and Ming Tu and Visar Berisha and Julie Liss",
year = "2016",
doi = "10.21437/Interspeech.2016-1148",
language = "English (US)",
volume = "08-12-September-2016",
pages = "2388--2392",
journal = "Scanning Electron Microscopy",
issn = "0586-5581",
publisher = "Scanning Microscopy International",

}

TY - JOUR

T1 - Accent identification by combining deep neural networks and recurrent neural networks trained on long and short term features

AU - Jiao, Yishan

AU - Tu, Ming

AU - Berisha, Visar

AU - Liss, Julie

PY - 2016

Y1 - 2016

N2 - Automatic identification of foreign accents is valuable for many speech systems, such as speech recognition, speaker identification, voice conversion, etc. The INTERSPEECH 2016 Native Language Sub-Challenge is to identify the native languages of non-native English speakers from eleven countries. Since differences in accent are due to both prosodic and articulation characteristics, a combination of long-term and short-term training is proposed in this paper. Each speech sample is processed into multiple speech segments with equal length. For each segment, deep neural networks (DNNs) are used to train on long-term statistical features, while recurrent neural networks (RNNs) are used to train on short-term acoustic features. The result for each speech sample is calculated by linearly fusing the results from the two sets of networks on all segments. The performance of the proposed system greatly surpasses the provided baseline system. Moreover, by fusing the results with the baseline system, the performance can be further improved.

AB - Automatic identification of foreign accents is valuable for many speech systems, such as speech recognition, speaker identification, voice conversion, etc. The INTERSPEECH 2016 Native Language Sub-Challenge is to identify the native languages of non-native English speakers from eleven countries. Since differences in accent are due to both prosodic and articulation characteristics, a combination of long-term and short-term training is proposed in this paper. Each speech sample is processed into multiple speech segments with equal length. For each segment, deep neural networks (DNNs) are used to train on long-term statistical features, while recurrent neural networks (RNNs) are used to train on short-term acoustic features. The result for each speech sample is calculated by linearly fusing the results from the two sets of networks on all segments. The performance of the proposed system greatly surpasses the provided baseline system. Moreover, by fusing the results with the baseline system, the performance can be further improved.

KW - Accent identification

KW - Articulation

KW - Deep neural networks

KW - Prosody

UR - http://www.scopus.com/inward/record.url?scp=84994246161&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84994246161&partnerID=8YFLogxK

U2 - 10.21437/Interspeech.2016-1148

DO - 10.21437/Interspeech.2016-1148

M3 - Article

AN - SCOPUS:84994246161

VL - 08-12-September-2016

SP - 2388

EP - 2392

JO - Scanning Electron Microscopy

JF - Scanning Electron Microscopy

SN - 0586-5581

ER -