Do you feel lucky? A large-scale analysis of risk-rewards trade-offs in cyber security

A crucial part of a cyber-criminal's job is to balance the risks and rewards of his every action. For example, an expert spammer will tune a bot's email-sending rate to achieve a good throughput with an acceptable risk of being detected. Then, such a cyber-criminal has to choose how to launder the money he made with spamming, and he will have to consider many options (money mules, Bitcoin, etc.) that will offer different returns and risks. Although understanding these trade-offs and coming as close as possible to their optimum is what discriminates winners and losers in the cyber-crime world, there has been little study on this matter, as setting up a large-scale study to study how cyber-criminals deal with these risk-reward trade-offs is challenging. Computer security competitions provide a great opportunity both to educate students and to study realistic cyber-security scenarios in a controlled environment. Looking to study the risk-reward trade-offs seen in real cyber-security incidents, we designed and hosted a novel format for a Capture the Flag cyber-security contest, involving 89 teams comprising over 1,000 students across the globe. In this paper, we describe the intuition, intent, and design of the contest. Additionally, we present an analysis of the data set collected, evaluate its effectiveness in modeling risk-reward behavior, examine the strategies of the competing teams, and estimate the effectiveness of such strategies.