Building a large-scale microscopic road network traffic simulator in apache spark

Zishan Fu, Jia Yu, Mohamed Sarwat

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Scopus citations


Road network traffic data has been widely studied by researchers and practitioners in different areas such as urban planning, traffic prediction, and spatial-Temporal databases. For instance, researchers use such data to evaluate the impact of road network changes. Unfortunately, collecting large-scale high-quality urban traffic data requires tremendous efforts because participating vehicles must install GPS receivers and administrators must continuously monitor these devices. There has been a number of urban traffic simulators trying to generate such data with different features. However, they suffer from two critical issues (1) scalability: most of them only offer single-machine solution which is not adequate to produce large-scale data. Some simulators can generate traffic in parallel but do not well balance the load among machines in a cluster. (2) granularity: many simulators do not consider microscopic traffic situations including traffic lights, lane changing, car following. In the paper, we propose GeoSparkSim, a scalable traffic simulator which extends Apache Spark to generate large-scale road network traffic datasets with microscopic traffic simulation. The proposed system seamlessly integrates with a Spark-based spatial data management system, GeoSpark, to deliver a holistic approach that allows data scientists to simulate, analyze and visualize largescale urban traffic data. To implement microscopic traffic models, GeoSparkSim employs a simulation-Aware vehicle partitioning method to partition vehicles among different machines such that each machine has a balanced workload. The experimental analysis shows that GeoSparkSim can simulate the movements of 200 thousand vehicles over a very large road network (250 thousand road junctions and 300 thousand road segments).

Original languageEnglish (US)
Title of host publicationProceedings - 2019 20th International Conference on Mobile Data Management, MDM 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages9
ISBN (Electronic)9781728133638
StatePublished - Jun 2019
Event20th International Conference on Mobile Data Management, MDM 2019 - Hong Kong, Hong Kong
Duration: Jun 10 2019Jun 13 2019

Publication series

NameProceedings - IEEE International Conference on Mobile Data Management
ISSN (Print)1551-6245


Conference20th International Conference on Mobile Data Management, MDM 2019
Country/TerritoryHong Kong
CityHong Kong


  • Apache Spark
  • Microscopic traffic simulation
  • Spatio-Temporal Data
  • Traffic model

ASJC Scopus subject areas

  • Engineering(all)


Dive into the research topics of 'Building a large-scale microscopic road network traffic simulator in apache spark'. Together they form a unique fingerprint.

Cite this