Optimizing sparse RFI prediction using deep learning

Joshua Kerrigan, Paul la Plante, Saul Kohn, Jonathan C. Pober, James Aguirre, Zara Abdurashidova, Paul Alexander, Zaki S. Ali, Yanga Balfour, Adam P. Beardsley, Gianni Bernardi, Judd D. Bowman, Richard F. Bradley, Jacob Burba, Chris L. Carilli, Carina Cheng, David R. DeBoer, Matt Dexter, Eloy de Lera Acedo, Joshua S. DillonJulia Estrada, Aaron Ewall-Wice, Nicolas Fagnoni, Randall Fritz, Steve R. Furlanetto, Brian Glendenning, Bradley Greig, Jasper Grobbelaar, Deepthi Gorthi, Ziyaad Halday, Bryna J. Hazelton, Jack Hickish, Daniel C. Jacobs, Austin Julius, Nicholas S. Kern, Piyanat Kittiwisit, Matthew Kolopanis, Adam Lanman, Telalo Lekalake, Adrian Liu, David MacMahon, Lourence Malan, Cresshim Malgas, Matthys Maree, Zachary E. Martinot, Eunice Matsetela, Andrei Mesinger, Mathakane Molewa, Miguel F. Morales, Tshegofalang Mosiane, Abraham R. Neben, Aaron R. Parsons, Nipanjana Patra, Samantha Pieterse, Nima Razavi-Ghods, Jon Ringuette, James Robnett, Kathryn Rosie, Peter Sims, Craig Smith, Angelo Syce, Nithyanandan Thyagarajan, Peter K.G. Williams, Haoxuan Zheng

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Radio frequency interference (RFI) is an ever-present limiting factor among radio telescopes even in the most remote observing locations. When looking to retain the maximum amount of sensitivity and reduce contamination for Epoch of Reionization studies, the identification and removal of RFI is especially important. In addition to improved RFI identification, we must also take into account computational efficiency of the RFI-Identification algorithm as radio interferometer arrays such as the Hydrogen Epoch of Reionization Array (HERA) grow larger in number of receivers. To address this, we present a deep fully convolutional neural network (DFCN) that is comprehensive in its use of interferometric data, where both amplitude and phase information are used jointly for identifying RFI. We train the network using simulated HERA visibilities containing mock RFI, yielding a known ‘ground truth’ data set for evaluating the accuracy of various RFI algorithms. Evaluation of the DFCN model is performed on observations from the 67 dish build-out, HERA-67, and achieves a data throughput of 1.6 × 105 HERA time-ordered 1024 channelled visibilities per hour per GPU. We determine that relative to an amplitude only network including visibility phase adds important adjacent time–frequency context which increases discrimination between RFI and non-RFI. The inclusion of phase when predicting achieves a recall of 0.81, precision of 0.58, and F2 score of 0.75 as applied to our HERA-67 observations.

Original languageEnglish (US)
Pages (from-to)2605-2615
Number of pages11
JournalMonthly Notices of the Royal Astronomical Society
Volume488
Issue number2
DOIs
StatePublished - Jan 1 2019

Fingerprint

radio frequency interference
learning
radio
time measurement
prediction
predictions
visibility
hydrogen
radio interferometers
ground truth
parabolic reflectors
radio telescopes
discrimination
contamination
interferometer
receivers
limiting factor
train
inclusions
interference

Keywords

  • Methods: data analysis – Techniques: interferometric

ASJC Scopus subject areas

  • Astronomy and Astrophysics
  • Space and Planetary Science

Cite this

Kerrigan, J., la Plante, P., Kohn, S., Pober, J. C., Aguirre, J., Abdurashidova, Z., ... Zheng, H. (2019). Optimizing sparse RFI prediction using deep learning. Monthly Notices of the Royal Astronomical Society, 488(2), 2605-2615. https://doi.org/10.1093/mnras/stz1865

Optimizing sparse RFI prediction using deep learning. / Kerrigan, Joshua; la Plante, Paul; Kohn, Saul; Pober, Jonathan C.; Aguirre, James; Abdurashidova, Zara; Alexander, Paul; Ali, Zaki S.; Balfour, Yanga; Beardsley, Adam P.; Bernardi, Gianni; Bowman, Judd D.; Bradley, Richard F.; Burba, Jacob; Carilli, Chris L.; Cheng, Carina; DeBoer, David R.; Dexter, Matt; de Lera Acedo, Eloy; Dillon, Joshua S.; Estrada, Julia; Ewall-Wice, Aaron; Fagnoni, Nicolas; Fritz, Randall; Furlanetto, Steve R.; Glendenning, Brian; Greig, Bradley; Grobbelaar, Jasper; Gorthi, Deepthi; Halday, Ziyaad; Hazelton, Bryna J.; Hickish, Jack; Jacobs, Daniel C.; Julius, Austin; Kern, Nicholas S.; Kittiwisit, Piyanat; Kolopanis, Matthew; Lanman, Adam; Lekalake, Telalo; Liu, Adrian; MacMahon, David; Malan, Lourence; Malgas, Cresshim; Maree, Matthys; Martinot, Zachary E.; Matsetela, Eunice; Mesinger, Andrei; Molewa, Mathakane; Morales, Miguel F.; Mosiane, Tshegofalang; Neben, Abraham R.; Parsons, Aaron R.; Patra, Nipanjana; Pieterse, Samantha; Razavi-Ghods, Nima; Ringuette, Jon; Robnett, James; Rosie, Kathryn; Sims, Peter; Smith, Craig; Syce, Angelo; Thyagarajan, Nithyanandan; Williams, Peter K.G.; Zheng, Haoxuan.

In: Monthly Notices of the Royal Astronomical Society, Vol. 488, No. 2, 01.01.2019, p. 2605-2615.

Research output: Contribution to journalArticle

Kerrigan, J, la Plante, P, Kohn, S, Pober, JC, Aguirre, J, Abdurashidova, Z, Alexander, P, Ali, ZS, Balfour, Y, Beardsley, AP, Bernardi, G, Bowman, JD, Bradley, RF, Burba, J, Carilli, CL, Cheng, C, DeBoer, DR, Dexter, M, de Lera Acedo, E, Dillon, JS, Estrada, J, Ewall-Wice, A, Fagnoni, N, Fritz, R, Furlanetto, SR, Glendenning, B, Greig, B, Grobbelaar, J, Gorthi, D, Halday, Z, Hazelton, BJ, Hickish, J, Jacobs, DC, Julius, A, Kern, NS, Kittiwisit, P, Kolopanis, M, Lanman, A, Lekalake, T, Liu, A, MacMahon, D, Malan, L, Malgas, C, Maree, M, Martinot, ZE, Matsetela, E, Mesinger, A, Molewa, M, Morales, MF, Mosiane, T, Neben, AR, Parsons, AR, Patra, N, Pieterse, S, Razavi-Ghods, N, Ringuette, J, Robnett, J, Rosie, K, Sims, P, Smith, C, Syce, A, Thyagarajan, N, Williams, PKG & Zheng, H 2019, 'Optimizing sparse RFI prediction using deep learning', Monthly Notices of the Royal Astronomical Society, vol. 488, no. 2, pp. 2605-2615. https://doi.org/10.1093/mnras/stz1865
Kerrigan J, la Plante P, Kohn S, Pober JC, Aguirre J, Abdurashidova Z et al. Optimizing sparse RFI prediction using deep learning. Monthly Notices of the Royal Astronomical Society. 2019 Jan 1;488(2):2605-2615. https://doi.org/10.1093/mnras/stz1865
Kerrigan, Joshua ; la Plante, Paul ; Kohn, Saul ; Pober, Jonathan C. ; Aguirre, James ; Abdurashidova, Zara ; Alexander, Paul ; Ali, Zaki S. ; Balfour, Yanga ; Beardsley, Adam P. ; Bernardi, Gianni ; Bowman, Judd D. ; Bradley, Richard F. ; Burba, Jacob ; Carilli, Chris L. ; Cheng, Carina ; DeBoer, David R. ; Dexter, Matt ; de Lera Acedo, Eloy ; Dillon, Joshua S. ; Estrada, Julia ; Ewall-Wice, Aaron ; Fagnoni, Nicolas ; Fritz, Randall ; Furlanetto, Steve R. ; Glendenning, Brian ; Greig, Bradley ; Grobbelaar, Jasper ; Gorthi, Deepthi ; Halday, Ziyaad ; Hazelton, Bryna J. ; Hickish, Jack ; Jacobs, Daniel C. ; Julius, Austin ; Kern, Nicholas S. ; Kittiwisit, Piyanat ; Kolopanis, Matthew ; Lanman, Adam ; Lekalake, Telalo ; Liu, Adrian ; MacMahon, David ; Malan, Lourence ; Malgas, Cresshim ; Maree, Matthys ; Martinot, Zachary E. ; Matsetela, Eunice ; Mesinger, Andrei ; Molewa, Mathakane ; Morales, Miguel F. ; Mosiane, Tshegofalang ; Neben, Abraham R. ; Parsons, Aaron R. ; Patra, Nipanjana ; Pieterse, Samantha ; Razavi-Ghods, Nima ; Ringuette, Jon ; Robnett, James ; Rosie, Kathryn ; Sims, Peter ; Smith, Craig ; Syce, Angelo ; Thyagarajan, Nithyanandan ; Williams, Peter K.G. ; Zheng, Haoxuan. / Optimizing sparse RFI prediction using deep learning. In: Monthly Notices of the Royal Astronomical Society. 2019 ; Vol. 488, No. 2. pp. 2605-2615.
@article{c84a7c5756194afa9570a86b74d71c82,
title = "Optimizing sparse RFI prediction using deep learning",
abstract = "Radio frequency interference (RFI) is an ever-present limiting factor among radio telescopes even in the most remote observing locations. When looking to retain the maximum amount of sensitivity and reduce contamination for Epoch of Reionization studies, the identification and removal of RFI is especially important. In addition to improved RFI identification, we must also take into account computational efficiency of the RFI-Identification algorithm as radio interferometer arrays such as the Hydrogen Epoch of Reionization Array (HERA) grow larger in number of receivers. To address this, we present a deep fully convolutional neural network (DFCN) that is comprehensive in its use of interferometric data, where both amplitude and phase information are used jointly for identifying RFI. We train the network using simulated HERA visibilities containing mock RFI, yielding a known ‘ground truth’ data set for evaluating the accuracy of various RFI algorithms. Evaluation of the DFCN model is performed on observations from the 67 dish build-out, HERA-67, and achieves a data throughput of 1.6 × 105 HERA time-ordered 1024 channelled visibilities per hour per GPU. We determine that relative to an amplitude only network including visibility phase adds important adjacent time–frequency context which increases discrimination between RFI and non-RFI. The inclusion of phase when predicting achieves a recall of 0.81, precision of 0.58, and F2 score of 0.75 as applied to our HERA-67 observations.",
keywords = "Methods: data analysis – Techniques: interferometric",
author = "Joshua Kerrigan and {la Plante}, Paul and Saul Kohn and Pober, {Jonathan C.} and James Aguirre and Zara Abdurashidova and Paul Alexander and Ali, {Zaki S.} and Yanga Balfour and Beardsley, {Adam P.} and Gianni Bernardi and Bowman, {Judd D.} and Bradley, {Richard F.} and Jacob Burba and Carilli, {Chris L.} and Carina Cheng and DeBoer, {David R.} and Matt Dexter and {de Lera Acedo}, Eloy and Dillon, {Joshua S.} and Julia Estrada and Aaron Ewall-Wice and Nicolas Fagnoni and Randall Fritz and Furlanetto, {Steve R.} and Brian Glendenning and Bradley Greig and Jasper Grobbelaar and Deepthi Gorthi and Ziyaad Halday and Hazelton, {Bryna J.} and Jack Hickish and Jacobs, {Daniel C.} and Austin Julius and Kern, {Nicholas S.} and Piyanat Kittiwisit and Matthew Kolopanis and Adam Lanman and Telalo Lekalake and Adrian Liu and David MacMahon and Lourence Malan and Cresshim Malgas and Matthys Maree and Martinot, {Zachary E.} and Eunice Matsetela and Andrei Mesinger and Mathakane Molewa and Morales, {Miguel F.} and Tshegofalang Mosiane and Neben, {Abraham R.} and Parsons, {Aaron R.} and Nipanjana Patra and Samantha Pieterse and Nima Razavi-Ghods and Jon Ringuette and James Robnett and Kathryn Rosie and Peter Sims and Craig Smith and Angelo Syce and Nithyanandan Thyagarajan and Williams, {Peter K.G.} and Haoxuan Zheng",
year = "2019",
month = "1",
day = "1",
doi = "10.1093/mnras/stz1865",
language = "English (US)",
volume = "488",
pages = "2605--2615",
journal = "Monthly Notices of the Royal Astronomical Society: Letters",
issn = "1745-3933",
publisher = "Oxford University Press",
number = "2",

}

TY - JOUR

T1 - Optimizing sparse RFI prediction using deep learning

AU - Kerrigan, Joshua

AU - la Plante, Paul

AU - Kohn, Saul

AU - Pober, Jonathan C.

AU - Aguirre, James

AU - Abdurashidova, Zara

AU - Alexander, Paul

AU - Ali, Zaki S.

AU - Balfour, Yanga

AU - Beardsley, Adam P.

AU - Bernardi, Gianni

AU - Bowman, Judd D.

AU - Bradley, Richard F.

AU - Burba, Jacob

AU - Carilli, Chris L.

AU - Cheng, Carina

AU - DeBoer, David R.

AU - Dexter, Matt

AU - de Lera Acedo, Eloy

AU - Dillon, Joshua S.

AU - Estrada, Julia

AU - Ewall-Wice, Aaron

AU - Fagnoni, Nicolas

AU - Fritz, Randall

AU - Furlanetto, Steve R.

AU - Glendenning, Brian

AU - Greig, Bradley

AU - Grobbelaar, Jasper

AU - Gorthi, Deepthi

AU - Halday, Ziyaad

AU - Hazelton, Bryna J.

AU - Hickish, Jack

AU - Jacobs, Daniel C.

AU - Julius, Austin

AU - Kern, Nicholas S.

AU - Kittiwisit, Piyanat

AU - Kolopanis, Matthew

AU - Lanman, Adam

AU - Lekalake, Telalo

AU - Liu, Adrian

AU - MacMahon, David

AU - Malan, Lourence

AU - Malgas, Cresshim

AU - Maree, Matthys

AU - Martinot, Zachary E.

AU - Matsetela, Eunice

AU - Mesinger, Andrei

AU - Molewa, Mathakane

AU - Morales, Miguel F.

AU - Mosiane, Tshegofalang

AU - Neben, Abraham R.

AU - Parsons, Aaron R.

AU - Patra, Nipanjana

AU - Pieterse, Samantha

AU - Razavi-Ghods, Nima

AU - Ringuette, Jon

AU - Robnett, James

AU - Rosie, Kathryn

AU - Sims, Peter

AU - Smith, Craig

AU - Syce, Angelo

AU - Thyagarajan, Nithyanandan

AU - Williams, Peter K.G.

AU - Zheng, Haoxuan

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Radio frequency interference (RFI) is an ever-present limiting factor among radio telescopes even in the most remote observing locations. When looking to retain the maximum amount of sensitivity and reduce contamination for Epoch of Reionization studies, the identification and removal of RFI is especially important. In addition to improved RFI identification, we must also take into account computational efficiency of the RFI-Identification algorithm as radio interferometer arrays such as the Hydrogen Epoch of Reionization Array (HERA) grow larger in number of receivers. To address this, we present a deep fully convolutional neural network (DFCN) that is comprehensive in its use of interferometric data, where both amplitude and phase information are used jointly for identifying RFI. We train the network using simulated HERA visibilities containing mock RFI, yielding a known ‘ground truth’ data set for evaluating the accuracy of various RFI algorithms. Evaluation of the DFCN model is performed on observations from the 67 dish build-out, HERA-67, and achieves a data throughput of 1.6 × 105 HERA time-ordered 1024 channelled visibilities per hour per GPU. We determine that relative to an amplitude only network including visibility phase adds important adjacent time–frequency context which increases discrimination between RFI and non-RFI. The inclusion of phase when predicting achieves a recall of 0.81, precision of 0.58, and F2 score of 0.75 as applied to our HERA-67 observations.

AB - Radio frequency interference (RFI) is an ever-present limiting factor among radio telescopes even in the most remote observing locations. When looking to retain the maximum amount of sensitivity and reduce contamination for Epoch of Reionization studies, the identification and removal of RFI is especially important. In addition to improved RFI identification, we must also take into account computational efficiency of the RFI-Identification algorithm as radio interferometer arrays such as the Hydrogen Epoch of Reionization Array (HERA) grow larger in number of receivers. To address this, we present a deep fully convolutional neural network (DFCN) that is comprehensive in its use of interferometric data, where both amplitude and phase information are used jointly for identifying RFI. We train the network using simulated HERA visibilities containing mock RFI, yielding a known ‘ground truth’ data set for evaluating the accuracy of various RFI algorithms. Evaluation of the DFCN model is performed on observations from the 67 dish build-out, HERA-67, and achieves a data throughput of 1.6 × 105 HERA time-ordered 1024 channelled visibilities per hour per GPU. We determine that relative to an amplitude only network including visibility phase adds important adjacent time–frequency context which increases discrimination between RFI and non-RFI. The inclusion of phase when predicting achieves a recall of 0.81, precision of 0.58, and F2 score of 0.75 as applied to our HERA-67 observations.

KW - Methods: data analysis – Techniques: interferometric

UR - http://www.scopus.com/inward/record.url?scp=85074563295&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85074563295&partnerID=8YFLogxK

U2 - 10.1093/mnras/stz1865

DO - 10.1093/mnras/stz1865

M3 - Article

AN - SCOPUS:85074563295

VL - 488

SP - 2605

EP - 2615

JO - Monthly Notices of the Royal Astronomical Society: Letters

JF - Monthly Notices of the Royal Astronomical Society: Letters

SN - 1745-3933

IS - 2

ER -