A connectionist approach to generating oblique decision trees

Rudy Setiono, Huan Liu

Research output: Contribution to journalArticle

28 Scopus citations

Abstract

Neural networks and decision tree methods are two common approaches to pattern classification. While neural networks can achieve high predictive accuracy rates, the decision boundaries they form are highly nonlinear and generally difficult to comprehend. Decision trees, on the other hand, can be readily translated into a set of rules. In this paper, we present a novel algorithm for generating oblique decision trees that capitalizes on the strength of both approaches. Oblique decision trees classify the patterns by testing on linear combinations of the input attributes. As a result, an oblique decision tree is usually much smaller than the univariate tree generated for the same domain. Our algorithm consists of two components: connectionist and symbolic. A three-layer feedforward neural network is constructed and pruned, a decision tree is then built from the hidden unit activation values of the pruned network. An oblique decision tree is obtained by expressing the activation values using the original input attributes. We test our algorithm on a wide range of problems. The oblique decision trees generated by the algorithm preserve the high accuracy of the neural networks, while keeping the explicitness of decision trees. Moreover, they outperform univariate decision trees generated by the symbolic approach and oblique decision trees built by other approaches in accuracy and tree size.

Original languageEnglish (US)
Pages (from-to)440-444
Number of pages5
JournalIEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Volume29
Issue number3
DOIs
StatePublished - Dec 1 1999
Externally publishedYes

    Fingerprint

Keywords

  • Decision trees
  • Network pruning
  • Neural networks
  • Oblique decision rules

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Software
  • Information Systems
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering

Cite this