Gray-box adversarial testing for control systems with machine learning components

Shakiba Yaghoubi, Georgios Fainekos

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Neural Networks (NN) have been proposed in the past as an effective means for both modeling and control of systems with very complex dynamics. However, despite the extensive research, NN-based controllers have not been adopted by the industry for safety critical systems. The primary reason is that systems with learning based controllers are notoriously hard to test and verify. Even harder is the analysis of such systems against system-level specifications. In this paper, we provide a gradient based method for searching the input space of a closed-loop control system in order to find adversarial samples against some system-level requirements. Our experimental results show that combined with randomized search, our method outperforms Simulated Annealing optimization.

Original languageEnglish (US)
Title of host publicationHSCC 2019 - Proceedings of the 2019 22nd ACM International Conference on Hybrid Systems
Subtitle of host publicationComputation and Control
PublisherAssociation for Computing Machinery, Inc
Pages179-184
Number of pages6
ISBN (Electronic)9781450362825
DOIs
StatePublished - Apr 16 2019
Event22nd ACM International Conference on Hybrid Systems: Computation and Control, HSCC 2019 - Montreal, Canada
Duration: Apr 16 2019Apr 18 2019

Publication series

NameHSCC 2019 - Proceedings of the 2019 22nd ACM International Conference on Hybrid Systems: Computation and Control

Conference

Conference22nd ACM International Conference on Hybrid Systems: Computation and Control, HSCC 2019
CountryCanada
CityMontreal
Period4/16/194/18/19

Fingerprint

Learning systems
Neural networks
Control systems
Closed loop control systems
Controllers
Testing
Simulated annealing
Specifications
Industry

Keywords

  • Neural network
  • Optimization
  • Testing and verification

ASJC Scopus subject areas

  • Computer Science Applications
  • Computer Networks and Communications
  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Cite this

Yaghoubi, S., & Fainekos, G. (2019). Gray-box adversarial testing for control systems with machine learning components. In HSCC 2019 - Proceedings of the 2019 22nd ACM International Conference on Hybrid Systems: Computation and Control (pp. 179-184). (HSCC 2019 - Proceedings of the 2019 22nd ACM International Conference on Hybrid Systems: Computation and Control). Association for Computing Machinery, Inc. https://doi.org/10.1145/3302504.3311814

Gray-box adversarial testing for control systems with machine learning components. / Yaghoubi, Shakiba; Fainekos, Georgios.

HSCC 2019 - Proceedings of the 2019 22nd ACM International Conference on Hybrid Systems: Computation and Control. Association for Computing Machinery, Inc, 2019. p. 179-184 (HSCC 2019 - Proceedings of the 2019 22nd ACM International Conference on Hybrid Systems: Computation and Control).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yaghoubi, S & Fainekos, G 2019, Gray-box adversarial testing for control systems with machine learning components. in HSCC 2019 - Proceedings of the 2019 22nd ACM International Conference on Hybrid Systems: Computation and Control. HSCC 2019 - Proceedings of the 2019 22nd ACM International Conference on Hybrid Systems: Computation and Control, Association for Computing Machinery, Inc, pp. 179-184, 22nd ACM International Conference on Hybrid Systems: Computation and Control, HSCC 2019, Montreal, Canada, 4/16/19. https://doi.org/10.1145/3302504.3311814
Yaghoubi S, Fainekos G. Gray-box adversarial testing for control systems with machine learning components. In HSCC 2019 - Proceedings of the 2019 22nd ACM International Conference on Hybrid Systems: Computation and Control. Association for Computing Machinery, Inc. 2019. p. 179-184. (HSCC 2019 - Proceedings of the 2019 22nd ACM International Conference on Hybrid Systems: Computation and Control). https://doi.org/10.1145/3302504.3311814
Yaghoubi, Shakiba ; Fainekos, Georgios. / Gray-box adversarial testing for control systems with machine learning components. HSCC 2019 - Proceedings of the 2019 22nd ACM International Conference on Hybrid Systems: Computation and Control. Association for Computing Machinery, Inc, 2019. pp. 179-184 (HSCC 2019 - Proceedings of the 2019 22nd ACM International Conference on Hybrid Systems: Computation and Control).
@inproceedings{b48e30636d34420faf978e0115e4373a,
title = "Gray-box adversarial testing for control systems with machine learning components",
abstract = "Neural Networks (NN) have been proposed in the past as an effective means for both modeling and control of systems with very complex dynamics. However, despite the extensive research, NN-based controllers have not been adopted by the industry for safety critical systems. The primary reason is that systems with learning based controllers are notoriously hard to test and verify. Even harder is the analysis of such systems against system-level specifications. In this paper, we provide a gradient based method for searching the input space of a closed-loop control system in order to find adversarial samples against some system-level requirements. Our experimental results show that combined with randomized search, our method outperforms Simulated Annealing optimization.",
keywords = "Neural network, Optimization, Testing and verification",
author = "Shakiba Yaghoubi and Georgios Fainekos",
year = "2019",
month = "4",
day = "16",
doi = "10.1145/3302504.3311814",
language = "English (US)",
series = "HSCC 2019 - Proceedings of the 2019 22nd ACM International Conference on Hybrid Systems: Computation and Control",
publisher = "Association for Computing Machinery, Inc",
pages = "179--184",
booktitle = "HSCC 2019 - Proceedings of the 2019 22nd ACM International Conference on Hybrid Systems",

}

TY - GEN

T1 - Gray-box adversarial testing for control systems with machine learning components

AU - Yaghoubi, Shakiba

AU - Fainekos, Georgios

PY - 2019/4/16

Y1 - 2019/4/16

N2 - Neural Networks (NN) have been proposed in the past as an effective means for both modeling and control of systems with very complex dynamics. However, despite the extensive research, NN-based controllers have not been adopted by the industry for safety critical systems. The primary reason is that systems with learning based controllers are notoriously hard to test and verify. Even harder is the analysis of such systems against system-level specifications. In this paper, we provide a gradient based method for searching the input space of a closed-loop control system in order to find adversarial samples against some system-level requirements. Our experimental results show that combined with randomized search, our method outperforms Simulated Annealing optimization.

AB - Neural Networks (NN) have been proposed in the past as an effective means for both modeling and control of systems with very complex dynamics. However, despite the extensive research, NN-based controllers have not been adopted by the industry for safety critical systems. The primary reason is that systems with learning based controllers are notoriously hard to test and verify. Even harder is the analysis of such systems against system-level specifications. In this paper, we provide a gradient based method for searching the input space of a closed-loop control system in order to find adversarial samples against some system-level requirements. Our experimental results show that combined with randomized search, our method outperforms Simulated Annealing optimization.

KW - Neural network

KW - Optimization

KW - Testing and verification

UR - http://www.scopus.com/inward/record.url?scp=85064967114&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85064967114&partnerID=8YFLogxK

U2 - 10.1145/3302504.3311814

DO - 10.1145/3302504.3311814

M3 - Conference contribution

T3 - HSCC 2019 - Proceedings of the 2019 22nd ACM International Conference on Hybrid Systems: Computation and Control

SP - 179

EP - 184

BT - HSCC 2019 - Proceedings of the 2019 22nd ACM International Conference on Hybrid Systems

PB - Association for Computing Machinery, Inc

ER -