Abstract
Social bots are an effective tool in the arsenal of malicious actors who manipulate discussions on social media. Bots help spread misinformation, promote political propaganda, and inflate the popularity of users and content. Hence, it is necessary to differentiate bot accounts and human users. There are several bot detection methods that approach this problem. Conventional methods either focus on precision regardless of the overall performance or optimize overall performance, say F1, without monitoring its effect on precision or recall. Focusing on precision means that those users marked as bots are more likely than not bots but a large portion of the bots could remain undetected. From a user’s perspective, however, it is more desirable to have less interaction with bots, even if it would incur a loss in precision. This can be achieved by a detection method with higher recall. A trivial, but useless, solution for high recall is to classify every account (human or bot) as bot, hence, resulting in poor overall performance. In this work, we investigate if it is feasible for a method to focus on recall without considerable loss in overall performance. Extensive experiments with recall and precision trade-off suggest that high recall can be achieved without much overall performance deterioration. This research leads to a recall-focused approach to bot detection, REFOCUS, with some lessons learned and future directions.
| Original language | English (US) |
|---|---|
| Title of host publication | Social, Cultural, and Behavioral Modeling - 12th International Conference, SBP-BRiMS 2019, Proceedings |
| Editors | Robert Thomson, Christopher Dancy, Ayaz Hyder, Halil Bisgin |
| Publisher | Springer Verlag |
| Pages | 39-49 |
| Number of pages | 11 |
| ISBN (Print) | 9783030217402 |
| DOIs | |
| State | Published - Jan 1 2019 |
| Event | 12th International Conference on Social Computing, Behavioral-Cultural Modeling, and Prediction and Behavior Representation in Modeling and Simulation, SBP-BRiMS 2019 - Washington D.C., United States Duration: Jul 9 2019 → Jul 12 2019 |
Publication series
| Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
|---|---|
| Volume | 11549 LNCS |
| ISSN (Print) | 0302-9743 |
| ISSN (Electronic) | 1611-3349 |
Conference
| Conference | 12th International Conference on Social Computing, Behavioral-Cultural Modeling, and Prediction and Behavior Representation in Modeling and Simulation, SBP-BRiMS 2019 |
|---|---|
| Country | United States |
| City | Washington D.C. |
| Period | 7/9/19 → 7/12/19 |
Fingerprint
Keywords
- Bot detection
- Recall
- Social bots
- Social media
ASJC Scopus subject areas
- Theoretical Computer Science
- Computer Science(all)
Cite this
Bot detection : Will focusing on recall cause overall performance deterioration? / H. Nazer, Tahora; Davis, Matthew; Karami, Mansooreh; Akoglu, Leman; Koelle, David; Liu, Huan.
Social, Cultural, and Behavioral Modeling - 12th International Conference, SBP-BRiMS 2019, Proceedings. ed. / Robert Thomson; Christopher Dancy; Ayaz Hyder; Halil Bisgin. Springer Verlag, 2019. p. 39-49 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11549 LNCS).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - Bot detection
T2 - Will focusing on recall cause overall performance deterioration?
AU - H. Nazer, Tahora
AU - Davis, Matthew
AU - Karami, Mansooreh
AU - Akoglu, Leman
AU - Koelle, David
AU - Liu, Huan
PY - 2019/1/1
Y1 - 2019/1/1
N2 - Social bots are an effective tool in the arsenal of malicious actors who manipulate discussions on social media. Bots help spread misinformation, promote political propaganda, and inflate the popularity of users and content. Hence, it is necessary to differentiate bot accounts and human users. There are several bot detection methods that approach this problem. Conventional methods either focus on precision regardless of the overall performance or optimize overall performance, say F1, without monitoring its effect on precision or recall. Focusing on precision means that those users marked as bots are more likely than not bots but a large portion of the bots could remain undetected. From a user’s perspective, however, it is more desirable to have less interaction with bots, even if it would incur a loss in precision. This can be achieved by a detection method with higher recall. A trivial, but useless, solution for high recall is to classify every account (human or bot) as bot, hence, resulting in poor overall performance. In this work, we investigate if it is feasible for a method to focus on recall without considerable loss in overall performance. Extensive experiments with recall and precision trade-off suggest that high recall can be achieved without much overall performance deterioration. This research leads to a recall-focused approach to bot detection, REFOCUS, with some lessons learned and future directions.
AB - Social bots are an effective tool in the arsenal of malicious actors who manipulate discussions on social media. Bots help spread misinformation, promote political propaganda, and inflate the popularity of users and content. Hence, it is necessary to differentiate bot accounts and human users. There are several bot detection methods that approach this problem. Conventional methods either focus on precision regardless of the overall performance or optimize overall performance, say F1, without monitoring its effect on precision or recall. Focusing on precision means that those users marked as bots are more likely than not bots but a large portion of the bots could remain undetected. From a user’s perspective, however, it is more desirable to have less interaction with bots, even if it would incur a loss in precision. This can be achieved by a detection method with higher recall. A trivial, but useless, solution for high recall is to classify every account (human or bot) as bot, hence, resulting in poor overall performance. In this work, we investigate if it is feasible for a method to focus on recall without considerable loss in overall performance. Extensive experiments with recall and precision trade-off suggest that high recall can be achieved without much overall performance deterioration. This research leads to a recall-focused approach to bot detection, REFOCUS, with some lessons learned and future directions.
KW - Bot detection
KW - Recall
KW - Social bots
KW - Social media
KW - Twitter
UR - http://www.scopus.com/inward/record.url?scp=85068154080&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85068154080&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-21741-9_5
DO - 10.1007/978-3-030-21741-9_5
M3 - Conference contribution
SN - 9783030217402
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 39
EP - 49
BT - Social, Cultural, and Behavioral Modeling - 12th International Conference, SBP-BRiMS 2019, Proceedings
A2 - Thomson, Robert
A2 - Dancy, Christopher
A2 - Hyder, Ayaz
A2 - Bisgin, Halil
PB - Springer Verlag
ER -