TY - GEN
T1 - The world (of CTF) is not enough data
T2 - 5th IEEE International Conference on Collaboration and Internet Computing, CIC 2019
AU - Ferguson-Walter, Kimberly
AU - Major, Maxine
AU - Van Bruggen, Dirk
AU - Fugate, Sunny
AU - Gutzwiller, Robert
N1 - Publisher Copyright:
© 2019 IEEE.
PY - 2019/12
Y1 - 2019/12
N2 - The human side of cyber is fundamentally important to understanding and improving cyber operations. With the exception of Capture the Flag (CTF) exercises, cyber testing and experimentation tends to ignore the human attacker. While traditional CTF events include a deeply rooted human component, they rarely aim to measure human performance, cognition, or psychology. We argue that CTF is not sufficient for measuring these aspects of the human; instead, we examine the value in performing red team behavioral and cognitive testing in a large-scale, controlled human-subject experiment. In this paper we describe the pros and cons of performing this type of experimentation and provide detailed exposition of the data collection and experimental controls used during a recent cyber deception experiment - the Tularosa Study. Finally, we will discuss lessons learned and how our experiences can inform best practices in future cyber operations studies of human behavior and cognition.
AB - The human side of cyber is fundamentally important to understanding and improving cyber operations. With the exception of Capture the Flag (CTF) exercises, cyber testing and experimentation tends to ignore the human attacker. While traditional CTF events include a deeply rooted human component, they rarely aim to measure human performance, cognition, or psychology. We argue that CTF is not sufficient for measuring these aspects of the human; instead, we examine the value in performing red team behavioral and cognitive testing in a large-scale, controlled human-subject experiment. In this paper we describe the pros and cons of performing this type of experimentation and provide detailed exposition of the data collection and experimental controls used during a recent cyber deception experiment - the Tularosa Study. Finally, we will discuss lessons learned and how our experiences can inform best practices in future cyber operations studies of human behavior and cognition.
KW - Computer-security-computer-networks-computer-hacking-cyber-defense-research-and-development-humans-testing-deception
UR - http://www.scopus.com/inward/record.url?scp=85080864846&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85080864846&partnerID=8YFLogxK
U2 - 10.1109/CIC48465.2019.00048
DO - 10.1109/CIC48465.2019.00048
M3 - Conference contribution
AN - SCOPUS:85080864846
T3 - Proceedings - 2019 IEEE 5th International Conference on Collaboration and Internet Computing, CIC 2019
SP - 346
EP - 353
BT - Proceedings - 2019 IEEE 5th International Conference on Collaboration and Internet Computing, CIC 2019
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 12 December 2019 through 14 December 2019
ER -