Abstract
The large number of parameters in deep neural networks (DNNs) often makes them prohibitive for low-power devices, such as field-programmable gate arrays (FPGA). In this paper, we propose a method to determine the relative importance of all network parameters by measuring the amount of information that the network output carries about each of the parameters - the Fisher Information. Based on the importance ranking, we design a complexity reduction scheme that discards unimportant parameters and assigns more quantization bits to more important parameters. For evaluation, we construct a deep autoencoder and learn a non-linear dimensionality reduction scheme for accelerometer data measuring the gait of individuals with Parkinson's disease. Experimental results confirm that the proposed ranking method can help reduce the complexity of the network with minimal impact on performance.
| Original language | English (US) |
|---|---|
| Title of host publication | 2016 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016 - Proceedings |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| Pages | 2647-2651 |
| Number of pages | 5 |
| Volume | 2016-May |
| ISBN (Electronic) | 9781479999880 |
| DOIs | |
| State | Published - May 18 2016 |
| Event | 41st IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016 - Shanghai, China Duration: Mar 20 2016 → Mar 25 2016 |
Other
| Other | 41st IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016 |
|---|---|
| Country | China |
| City | Shanghai |
| Period | 3/20/16 → 3/25/16 |
Fingerprint
Keywords
- complexity reduction
- deep neural networks
- Fisher Information
- FPGA
- pruning
- quantization
ASJC Scopus subject areas
- Signal Processing
- Software
- Electrical and Electronic Engineering
Cite this
Ranking the parameters of deep neural networks using the fisher information. / Tu, Ming; Berisha, Visar; Woolf, Martin; Seo, Jae-sun; Cao, Yu.
2016 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016 - Proceedings. Vol. 2016-May Institute of Electrical and Electronics Engineers Inc., 2016. p. 2647-2651 7472157.Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - Ranking the parameters of deep neural networks using the fisher information
AU - Tu, Ming
AU - Berisha, Visar
AU - Woolf, Martin
AU - Seo, Jae-sun
AU - Cao, Yu
PY - 2016/5/18
Y1 - 2016/5/18
N2 - The large number of parameters in deep neural networks (DNNs) often makes them prohibitive for low-power devices, such as field-programmable gate arrays (FPGA). In this paper, we propose a method to determine the relative importance of all network parameters by measuring the amount of information that the network output carries about each of the parameters - the Fisher Information. Based on the importance ranking, we design a complexity reduction scheme that discards unimportant parameters and assigns more quantization bits to more important parameters. For evaluation, we construct a deep autoencoder and learn a non-linear dimensionality reduction scheme for accelerometer data measuring the gait of individuals with Parkinson's disease. Experimental results confirm that the proposed ranking method can help reduce the complexity of the network with minimal impact on performance.
AB - The large number of parameters in deep neural networks (DNNs) often makes them prohibitive for low-power devices, such as field-programmable gate arrays (FPGA). In this paper, we propose a method to determine the relative importance of all network parameters by measuring the amount of information that the network output carries about each of the parameters - the Fisher Information. Based on the importance ranking, we design a complexity reduction scheme that discards unimportant parameters and assigns more quantization bits to more important parameters. For evaluation, we construct a deep autoencoder and learn a non-linear dimensionality reduction scheme for accelerometer data measuring the gait of individuals with Parkinson's disease. Experimental results confirm that the proposed ranking method can help reduce the complexity of the network with minimal impact on performance.
KW - complexity reduction
KW - deep neural networks
KW - Fisher Information
KW - FPGA
KW - pruning
KW - quantization
UR - http://www.scopus.com/inward/record.url?scp=84973402931&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84973402931&partnerID=8YFLogxK
U2 - 10.1109/ICASSP.2016.7472157
DO - 10.1109/ICASSP.2016.7472157
M3 - Conference contribution
AN - SCOPUS:84973402931
VL - 2016-May
SP - 2647
EP - 2651
BT - 2016 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
ER -