PlinyCompute: A platform for high-performance, distributed, data-intensive tool development

Jia Zou, R. Matthew Barnett, Tania Lorido-Botran, Shangyu Luo, Carlos Monroy, Sourav Sikdar, Kia Teymourian, Binhang Yuan, Chris Jermaine

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

This paper describes PlinyCompute, a system for development of high-performance, data-intensive, distributed computing tools and libraries. In the large, PlinyCompute presents the programmer with a very high-level, declarative interface, relying on automatic, relational-database style optimization to figure out how to stage distributed computations. However, in the small, PlinyCompute presents the capable systems programmer with a persistent object data model and API (the "PC object model") and associated memory management system that has been designed from the ground-up for high performance, distributed, data-intensive computing. This contrasts with most other Big Data systems, which are constructed on top of the Java Virtual Machine (JVM), and hence must at least partially cede performance-critical concerns such as memory management (including layout and de/allocation) and virtual method/-function dispatch to the JVM. This hybrid approach-declarative in the large, trusting the programmer's ability to utilize PC object model efficiently in the small-results in a system that is ideal for the development of reusable, data-intensive tools and libraries.

Original languageEnglish (US)
Title of host publicationSIGMOD 2018 - Proceedings of the 2018 International Conference on Management of Data
EditorsGautam Das, Christopher Jermaine, Ahmed Eldawy, Philip Bernstein
PublisherAssociation for Computing Machinery
Pages1189-1204
Number of pages16
ISBN (Electronic)9781450317436
DOIs
StatePublished - May 27 2018
Event44th ACM SIGMOD International Conference on Management of Data, SIGMOD 2018 - Houston, United States
Duration: Jun 10 2018Jun 15 2018

Publication series

NameProceedings of the ACM SIGMOD International Conference on Management of Data
ISSN (Print)0730-8078

Conference

Conference44th ACM SIGMOD International Conference on Management of Data, SIGMOD 2018
CountryUnited States
CityHouston
Period6/10/186/15/18

Keywords

  • Distributed computing
  • Object model
  • Query compilation

ASJC Scopus subject areas

  • Software
  • Information Systems

Fingerprint Dive into the research topics of 'PlinyCompute: A platform for high-performance, distributed, data-intensive tool development'. Together they form a unique fingerprint.

  • Cite this

    Zou, J., Barnett, R. M., Lorido-Botran, T., Luo, S., Monroy, C., Sikdar, S., Teymourian, K., Yuan, B., & Jermaine, C. (2018). PlinyCompute: A platform for high-performance, distributed, data-intensive tool development. In G. Das, C. Jermaine, A. Eldawy, & P. Bernstein (Eds.), SIGMOD 2018 - Proceedings of the 2018 International Conference on Management of Data (pp. 1189-1204). (Proceedings of the ACM SIGMOD International Conference on Management of Data). Association for Computing Machinery. https://doi.org/10.1145/3183713.3196933