TY - GEN
T1 - Binarized depthwise separable neural network for object tracking in FPGA
AU - Yang, Li
AU - He, Zhezhi
AU - Fan, Deliang
N1 - Funding Information:
ACKNOWLEDGEMENTS This work is supported in part by the National Science Foundation under Grant No. 1740126 and Semiconductor Research Corporation nCORE. REFERENCES
Publisher Copyright:
© 2019 ACM.
PY - 2019/5/13
Y1 - 2019/5/13
N2 - Object tracking has achieved great advances in the past few years and has been widely applied in vision-based application. Nowadays, deep convolutional neural network has taken an important role in object tracking tasks. However, its enormous model size and massive computation cost have became the main obstacle for deployment of such powerful algorithm in low power and resource limited embedded system, such as FPGA. Due to the popularization of the power-sensitive mobile platform, low power real-time tracking solution is strongly required. In order to address these challenges, we propose a low power and energy-efficient object tracking FPGA implementation based on a newly proposed binarized depthwise separable deep convolutional neural network. It can significantly reduce the model size and computation complexity simultaneously utilizing binarized (i.e., +1 and-1) depthwise separable convolution kernel and our proposed trainable threshold group binarization activation function. It can completely converts the dot product and accumulation based convolution operations into bit-wise XNOR and bit-count operations, while achieving state-of-the-art accuracy. Our proposed binarized depthwise separable model achieves ∼57% Intersection over Union (IOU) on DJI object tracking dataset with only ∼143.9Kb model parameter size. We then deploy our proposed model into the Xilinx PYNQ Z1 board with only 4.9Mb on-chip RAM. The experiment results show that our FPGA implementation achieves 11.1 frames per second for object tracking with only 2.61W.
AB - Object tracking has achieved great advances in the past few years and has been widely applied in vision-based application. Nowadays, deep convolutional neural network has taken an important role in object tracking tasks. However, its enormous model size and massive computation cost have became the main obstacle for deployment of such powerful algorithm in low power and resource limited embedded system, such as FPGA. Due to the popularization of the power-sensitive mobile platform, low power real-time tracking solution is strongly required. In order to address these challenges, we propose a low power and energy-efficient object tracking FPGA implementation based on a newly proposed binarized depthwise separable deep convolutional neural network. It can significantly reduce the model size and computation complexity simultaneously utilizing binarized (i.e., +1 and-1) depthwise separable convolution kernel and our proposed trainable threshold group binarization activation function. It can completely converts the dot product and accumulation based convolution operations into bit-wise XNOR and bit-count operations, while achieving state-of-the-art accuracy. Our proposed binarized depthwise separable model achieves ∼57% Intersection over Union (IOU) on DJI object tracking dataset with only ∼143.9Kb model parameter size. We then deploy our proposed model into the Xilinx PYNQ Z1 board with only 4.9Mb on-chip RAM. The experiment results show that our FPGA implementation achieves 11.1 frames per second for object tracking with only 2.61W.
KW - Binarized convolutional neural network (bnn)
KW - Field-programmable gate array (FPGA)
KW - Object tracking
UR - http://www.scopus.com/inward/record.url?scp=85083252365&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85083252365&partnerID=8YFLogxK
U2 - 10.1145/3299874.3318034
DO - 10.1145/3299874.3318034
M3 - Conference contribution
AN - SCOPUS:85083252365
T3 - Proceedings of the ACM Great Lakes Symposium on VLSI, GLSVLSI
SP - 347
EP - 350
BT - GLSVLSI 2019 - Proceedings of the 2019 Great Lakes Symposium on VLSI
PB - Association for Computing Machinery
T2 - 29th Great Lakes Symposium on VLSI, GLSVLSI 2019
Y2 - 9 May 2019 through 11 May 2019
ER -