BitTorious volunteer: Server-side extensions for centrally-managed volunteer storage in BitTorrent swarms

Preston V. Lee, Valentin Dinu

Research output: Contribution to journalArticle

Abstract

Background: Our publication of the BitTorious portal [1] demonstrated the ability to create a privatized distributed data warehouse of sufficient magnitude for real-world bioinformatics studies using minimal changes to the standard BitTorrent tracker protocol. In this second phase, we release a new server-side specification to accept anonymous philantropic storage donations by the general public, wherein a small portion of each user's local disk may be used for archival of scientific data. We have implementated the server-side announcement and control portions of this BitTorrent extension into v3.0.0 of the BitTorious portal, upon which compatible clients may be built. Results: Automated test cases for the BitTorious Volunteer extensions have been added to the portal's v3.0.0 release, supporting validation of the "peer affinity" concept and announcement protocol introduced by this specification. Additionally, a separate reference implementation of affinity calculation has been provided in C++ for informaticians wishing to integrate into libtorrent-based projects. Conclusions: The BitTorrent "affinity" extensions as provided in the BitTorious portal reference implementation allow data publishers to crowdsource the extreme storage prerequisites for research in "big data" fields. With sufficient awareness and adoption of BitTorious Volunteer-based clients by the general public, the BitTorious portal may be able to provide peta-scale storage resources to the scientific community at relatively insignificant financial cost.

Original languageEnglish (US)
Article number364
JournalBMC Bioinformatics
Volume16
Issue number1
DOIs
StatePublished - Nov 4 2015

Fingerprint

BitTorrent
Swarm
Crowdsourcing
Volunteers
Servers
Server
Affine transformation
Computational Biology
Costs and Cost Analysis
Specification
Sufficient
Specifications
Network protocols
Research
Data warehouses
Data Warehouse
Bioinformatics
C++
Extremes
Integrate

Keywords

  • Big data
  • Bioinformatics
  • BitTorious
  • BitTorrent
  • Citizen scientist
  • Data transfer
  • Open source
  • Software

ASJC Scopus subject areas

  • Applied Mathematics
  • Structural Biology
  • Biochemistry
  • Molecular Biology
  • Computer Science Applications

Cite this

BitTorious volunteer : Server-side extensions for centrally-managed volunteer storage in BitTorrent swarms. / Lee, Preston V.; Dinu, Valentin.

In: BMC Bioinformatics, Vol. 16, No. 1, 364, 04.11.2015.

Research output: Contribution to journalArticle

@article{c9064993c10d45fabeedfd7fdcb30b10,
title = "BitTorious volunteer: Server-side extensions for centrally-managed volunteer storage in BitTorrent swarms",
abstract = "Background: Our publication of the BitTorious portal [1] demonstrated the ability to create a privatized distributed data warehouse of sufficient magnitude for real-world bioinformatics studies using minimal changes to the standard BitTorrent tracker protocol. In this second phase, we release a new server-side specification to accept anonymous philantropic storage donations by the general public, wherein a small portion of each user's local disk may be used for archival of scientific data. We have implementated the server-side announcement and control portions of this BitTorrent extension into v3.0.0 of the BitTorious portal, upon which compatible clients may be built. Results: Automated test cases for the BitTorious Volunteer extensions have been added to the portal's v3.0.0 release, supporting validation of the {"}peer affinity{"} concept and announcement protocol introduced by this specification. Additionally, a separate reference implementation of affinity calculation has been provided in C++ for informaticians wishing to integrate into libtorrent-based projects. Conclusions: The BitTorrent {"}affinity{"} extensions as provided in the BitTorious portal reference implementation allow data publishers to crowdsource the extreme storage prerequisites for research in {"}big data{"} fields. With sufficient awareness and adoption of BitTorious Volunteer-based clients by the general public, the BitTorious portal may be able to provide peta-scale storage resources to the scientific community at relatively insignificant financial cost.",
keywords = "Big data, Bioinformatics, BitTorious, BitTorrent, Citizen scientist, Data transfer, Open source, Software",
author = "Lee, {Preston V.} and Valentin Dinu",
year = "2015",
month = "11",
day = "4",
doi = "10.1186/s12859-015-0779-6",
language = "English (US)",
volume = "16",
journal = "BMC Bioinformatics",
issn = "1471-2105",
publisher = "BioMed Central",
number = "1",

}

TY - JOUR

T1 - BitTorious volunteer

T2 - Server-side extensions for centrally-managed volunteer storage in BitTorrent swarms

AU - Lee, Preston V.

AU - Dinu, Valentin

PY - 2015/11/4

Y1 - 2015/11/4

N2 - Background: Our publication of the BitTorious portal [1] demonstrated the ability to create a privatized distributed data warehouse of sufficient magnitude for real-world bioinformatics studies using minimal changes to the standard BitTorrent tracker protocol. In this second phase, we release a new server-side specification to accept anonymous philantropic storage donations by the general public, wherein a small portion of each user's local disk may be used for archival of scientific data. We have implementated the server-side announcement and control portions of this BitTorrent extension into v3.0.0 of the BitTorious portal, upon which compatible clients may be built. Results: Automated test cases for the BitTorious Volunteer extensions have been added to the portal's v3.0.0 release, supporting validation of the "peer affinity" concept and announcement protocol introduced by this specification. Additionally, a separate reference implementation of affinity calculation has been provided in C++ for informaticians wishing to integrate into libtorrent-based projects. Conclusions: The BitTorrent "affinity" extensions as provided in the BitTorious portal reference implementation allow data publishers to crowdsource the extreme storage prerequisites for research in "big data" fields. With sufficient awareness and adoption of BitTorious Volunteer-based clients by the general public, the BitTorious portal may be able to provide peta-scale storage resources to the scientific community at relatively insignificant financial cost.

AB - Background: Our publication of the BitTorious portal [1] demonstrated the ability to create a privatized distributed data warehouse of sufficient magnitude for real-world bioinformatics studies using minimal changes to the standard BitTorrent tracker protocol. In this second phase, we release a new server-side specification to accept anonymous philantropic storage donations by the general public, wherein a small portion of each user's local disk may be used for archival of scientific data. We have implementated the server-side announcement and control portions of this BitTorrent extension into v3.0.0 of the BitTorious portal, upon which compatible clients may be built. Results: Automated test cases for the BitTorious Volunteer extensions have been added to the portal's v3.0.0 release, supporting validation of the "peer affinity" concept and announcement protocol introduced by this specification. Additionally, a separate reference implementation of affinity calculation has been provided in C++ for informaticians wishing to integrate into libtorrent-based projects. Conclusions: The BitTorrent "affinity" extensions as provided in the BitTorious portal reference implementation allow data publishers to crowdsource the extreme storage prerequisites for research in "big data" fields. With sufficient awareness and adoption of BitTorious Volunteer-based clients by the general public, the BitTorious portal may be able to provide peta-scale storage resources to the scientific community at relatively insignificant financial cost.

KW - Big data

KW - Bioinformatics

KW - BitTorious

KW - BitTorrent

KW - Citizen scientist

KW - Data transfer

KW - Open source

KW - Software

UR - http://www.scopus.com/inward/record.url?scp=84946402526&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84946402526&partnerID=8YFLogxK

U2 - 10.1186/s12859-015-0779-6

DO - 10.1186/s12859-015-0779-6

M3 - Article

C2 - 26537733

AN - SCOPUS:84946402526

VL - 16

JO - BMC Bioinformatics

JF - BMC Bioinformatics

SN - 1471-2105

IS - 1

M1 - 364

ER -