Autonomous security analysis and penetration testing

Ankur Chowdhary, Dijiang Huang, Jayasurya Sevalur Mahendran, Daniel Romo, Yuli Deng, Abdulhakim Sabur

Research output: Chapter in Book/Report/Conference proceedingConference contribution

31 Scopus citations

Abstract

Security Assessment of large networks is a challenging task. Penetration testing (pentesting) is a method of analyzing the attack surface of a network to find security vulnerabilities. Current network pentesting techniques involve a combination of automated scanning tools and manual exploitation of security issues to identify possible threats in a network. The solution scales poorly on a large network. We propose an autonomous security analysis and penetration testing framework (ASAP) that creates a map of security threats and possible attack paths in the network using attack graphs. Our framework utilizes: (i) state of the art reinforcement learning algorithm based on Deep-Q Network (DQN) to identify optimal policy for performing pentesting testing, and (ii) incorporates domain-specific transition matrix and reward modeling to capture the importance of security vulnerabilities and difficulty inherent in exploiting them. ASAP framework generates autonomous attack plans and validates them against real-world networks. The attack plans are generalizable to complex enterprise network, and the framework scales well on a large network. Our empirical evaluation shows that ASAP identifies non-intuitive attack plans on an enterprise network. The DQN planning algorithm employed scales well on a large network ~ 60 -70(s) for generating an attack plan for network with 300 hosts.

Original languageEnglish (US)
Title of host publicationProceedings - 2020 16th International Conference on Mobility, Sensing and Networking, MSN 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages508-515
Number of pages8
ISBN (Electronic)9781728199160
DOIs
StatePublished - Dec 2020
Event16th International Conference on Mobility, Sensing and Networking, MSN 2020 - Tokyo, Japan
Duration: Dec 17 2020Dec 19 2020

Publication series

NameProceedings - 2020 16th International Conference on Mobility, Sensing and Networking, MSN 2020

Conference

Conference16th International Conference on Mobility, Sensing and Networking, MSN 2020
Country/TerritoryJapan
CityTokyo
Period12/17/2012/19/20

Keywords

  • Attack Graphs
  • Cloud Network
  • Deep-Q Network (DQN)
  • Internet of Things (IoT)
  • Penetration Testing
  • Reinforcement Learning

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems and Management
  • Safety, Risk, Reliability and Quality
  • Instrumentation

Fingerprint

Dive into the research topics of 'Autonomous security analysis and penetration testing'. Together they form a unique fingerprint.

Cite this