Using instruction sequence abstraction for shellcode detection and attribution

Ziming Zhao, Gail-Joon Ahn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Scopus citations

Abstract

Although several research teams have focused on binary code injection, it is still an unsolved problem. Misuse-based detection lacks the flexibility to tackle unseen malicious code samples and anomaly-based detection on byte patterns is highly vulnerable to byte cramming and blending attacks. In addition, it is desperately needed to correlate newly-detected code injection instances with known samples for better understanding the attack events and tactically mitigating future threats. In this paper, we propose a technique for modeling shellcode detection and attribution through a novel feature extraction method, called instruction sequence abstraction, that extracts coarse-grained features from an instruction sequence. Our technique facilitates a Markov-chain-based model for shellcode detection and support vector machines for encoded shellcode attribution. We also describe our experimental results on shellcode samples to demonstrate the effectiveness of our approach.

Original languageEnglish (US)
Title of host publication2013 IEEE Conference on Communications and Network Security, CNS 2013
PublisherIEEE Computer Society
Pages323-331
Number of pages9
ISBN (Print)9781479908950
DOIs
StatePublished - Jan 1 2013
Event1st IEEE International Conference on Communications and Network Security, CNS 2013 - Washington, DC, United States
Duration: Oct 14 2013Oct 16 2013

Publication series

Name2013 IEEE Conference on Communications and Network Security, CNS 2013

Other

Other1st IEEE International Conference on Communications and Network Security, CNS 2013
CountryUnited States
CityWashington, DC
Period10/14/1310/16/13

ASJC Scopus subject areas

  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Using instruction sequence abstraction for shellcode detection and attribution'. Together they form a unique fingerprint.

  • Cite this

    Zhao, Z., & Ahn, G-J. (2013). Using instruction sequence abstraction for shellcode detection and attribution. In 2013 IEEE Conference on Communications and Network Security, CNS 2013 (pp. 323-331). [6682722] (2013 IEEE Conference on Communications and Network Security, CNS 2013). IEEE Computer Society. https://doi.org/10.1109/CNS.2013.6682722