Using instruction sequence abstraction for shellcode detection and attribution

Ziming Zhao, Gail-Joon Ahn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Citations (Scopus)

Abstract

Although several research teams have focused on binary code injection, it is still an unsolved problem. Misuse-based detection lacks the flexibility to tackle unseen malicious code samples and anomaly-based detection on byte patterns is highly vulnerable to byte cramming and blending attacks. In addition, it is desperately needed to correlate newly-detected code injection instances with known samples for better understanding the attack events and tactically mitigating future threats. In this paper, we propose a technique for modeling shellcode detection and attribution through a novel feature extraction method, called instruction sequence abstraction, that extracts coarse-grained features from an instruction sequence. Our technique facilitates a Markov-chain-based model for shellcode detection and support vector machines for encoded shellcode attribution. We also describe our experimental results on shellcode samples to demonstrate the effectiveness of our approach.

Original languageEnglish (US)
Title of host publication2013 IEEE Conference on Communications and Network Security, CNS 2013
PublisherIEEE Computer Society
Pages323-331
Number of pages9
ISBN (Print)9781479908950
DOIs
StatePublished - 2013
Event1st IEEE International Conference on Communications and Network Security, CNS 2013 - Washington, DC, United States
Duration: Oct 14 2013Oct 16 2013

Other

Other1st IEEE International Conference on Communications and Network Security, CNS 2013
CountryUnited States
CityWashington, DC
Period10/14/1310/16/13

Fingerprint

Binary codes
Markov processes
Support vector machines
Feature extraction

ASJC Scopus subject areas

  • Computer Networks and Communications

Cite this

Zhao, Z., & Ahn, G-J. (2013). Using instruction sequence abstraction for shellcode detection and attribution. In 2013 IEEE Conference on Communications and Network Security, CNS 2013 (pp. 323-331). [6682722] IEEE Computer Society. https://doi.org/10.1109/CNS.2013.6682722

Using instruction sequence abstraction for shellcode detection and attribution. / Zhao, Ziming; Ahn, Gail-Joon.

2013 IEEE Conference on Communications and Network Security, CNS 2013. IEEE Computer Society, 2013. p. 323-331 6682722.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zhao, Z & Ahn, G-J 2013, Using instruction sequence abstraction for shellcode detection and attribution. in 2013 IEEE Conference on Communications and Network Security, CNS 2013., 6682722, IEEE Computer Society, pp. 323-331, 1st IEEE International Conference on Communications and Network Security, CNS 2013, Washington, DC, United States, 10/14/13. https://doi.org/10.1109/CNS.2013.6682722
Zhao Z, Ahn G-J. Using instruction sequence abstraction for shellcode detection and attribution. In 2013 IEEE Conference on Communications and Network Security, CNS 2013. IEEE Computer Society. 2013. p. 323-331. 6682722 https://doi.org/10.1109/CNS.2013.6682722
Zhao, Ziming ; Ahn, Gail-Joon. / Using instruction sequence abstraction for shellcode detection and attribution. 2013 IEEE Conference on Communications and Network Security, CNS 2013. IEEE Computer Society, 2013. pp. 323-331
@inproceedings{27c8817e67ae45628aa54bbb5b9b01ca,
title = "Using instruction sequence abstraction for shellcode detection and attribution",
abstract = "Although several research teams have focused on binary code injection, it is still an unsolved problem. Misuse-based detection lacks the flexibility to tackle unseen malicious code samples and anomaly-based detection on byte patterns is highly vulnerable to byte cramming and blending attacks. In addition, it is desperately needed to correlate newly-detected code injection instances with known samples for better understanding the attack events and tactically mitigating future threats. In this paper, we propose a technique for modeling shellcode detection and attribution through a novel feature extraction method, called instruction sequence abstraction, that extracts coarse-grained features from an instruction sequence. Our technique facilitates a Markov-chain-based model for shellcode detection and support vector machines for encoded shellcode attribution. We also describe our experimental results on shellcode samples to demonstrate the effectiveness of our approach.",
author = "Ziming Zhao and Gail-Joon Ahn",
year = "2013",
doi = "10.1109/CNS.2013.6682722",
language = "English (US)",
isbn = "9781479908950",
pages = "323--331",
booktitle = "2013 IEEE Conference on Communications and Network Security, CNS 2013",
publisher = "IEEE Computer Society",

}

TY - GEN

T1 - Using instruction sequence abstraction for shellcode detection and attribution

AU - Zhao, Ziming

AU - Ahn, Gail-Joon

PY - 2013

Y1 - 2013

N2 - Although several research teams have focused on binary code injection, it is still an unsolved problem. Misuse-based detection lacks the flexibility to tackle unseen malicious code samples and anomaly-based detection on byte patterns is highly vulnerable to byte cramming and blending attacks. In addition, it is desperately needed to correlate newly-detected code injection instances with known samples for better understanding the attack events and tactically mitigating future threats. In this paper, we propose a technique for modeling shellcode detection and attribution through a novel feature extraction method, called instruction sequence abstraction, that extracts coarse-grained features from an instruction sequence. Our technique facilitates a Markov-chain-based model for shellcode detection and support vector machines for encoded shellcode attribution. We also describe our experimental results on shellcode samples to demonstrate the effectiveness of our approach.

AB - Although several research teams have focused on binary code injection, it is still an unsolved problem. Misuse-based detection lacks the flexibility to tackle unseen malicious code samples and anomaly-based detection on byte patterns is highly vulnerable to byte cramming and blending attacks. In addition, it is desperately needed to correlate newly-detected code injection instances with known samples for better understanding the attack events and tactically mitigating future threats. In this paper, we propose a technique for modeling shellcode detection and attribution through a novel feature extraction method, called instruction sequence abstraction, that extracts coarse-grained features from an instruction sequence. Our technique facilitates a Markov-chain-based model for shellcode detection and support vector machines for encoded shellcode attribution. We also describe our experimental results on shellcode samples to demonstrate the effectiveness of our approach.

UR - http://www.scopus.com/inward/record.url?scp=84893589593&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84893589593&partnerID=8YFLogxK

U2 - 10.1109/CNS.2013.6682722

DO - 10.1109/CNS.2013.6682722

M3 - Conference contribution

AN - SCOPUS:84893589593

SN - 9781479908950

SP - 323

EP - 331

BT - 2013 IEEE Conference on Communications and Network Security, CNS 2013

PB - IEEE Computer Society

ER -