EMPIRE-PIC: A Performance Portable Unstructured Particle-in-Cell Code

EMPIRE-PIC: A Performance Portable Unstructured Particle-in-Cell Code

Year:    2021

Author:    Matthew T. Bettencourt, Dominic A. S. Brown, Keith L. Cartwright, Eric C. Cyr, Christian A. Glusa, Paul T. Lin, Stan G. Moore, Duncan A. O. McGregor, Roger P. Pawlowski, Edward G. Phillips, Nathan V. Roberts, Steven A. Wright, Satheesh Maheswaran, John P. Jones, Stephen A. Jarvis

Communications in Computational Physics, Vol. 30 (2021), Iss. 4 : pp. 1232–1268

Abstract

In this paper we introduce EMPIRE-PIC, a finite element method particle-in-cell (FEM-PIC) application developed at Sandia National Laboratories. The code has been developed in C++ using the Trilinos library and the Kokkos Performance Portability Framework to enable running on multiple modern compute architectures while only requiring maintenance of a single codebase. EMPIRE-PIC is capable of solving both electrostatic and electromagnetic problems in two- and three-dimensions to second-order accuracy in space and time. In this paper we validate the code against three benchmark problems — a simple electron orbit, an electrostatic Langmuir wave, and a transverse electromagnetic wave propagating through a plasma. We demonstrate the performance of EMPIRE-PIC on four different architectures: Intel Haswell CPUs, Intel's Xeon Phi Knights Landing, ARM Thunder-X2 CPUs, and NVIDIA Tesla V100 GPUs attached to IBM POWER9 processors. This analysis demonstrates scalability of the code up to more than two thousand GPUs, and greater than one hundred thousand CPUs.

You do not have full access to this article.

Already a Subscriber? Sign in as an individual or via your institution

Journal Article Details

Publisher Name:    Global Science Press

Language:    English

DOI:    https://doi.org/10.4208/cicp.OA-2020-0261

Communications in Computational Physics, Vol. 30 (2021), Iss. 4 : pp. 1232–1268

Published online:    2021-01

AMS Subject Headings:    Global Science Press

Copyright:    COPYRIGHT: © Global Science Press

Pages:    37

Keywords:    PIC electrostatics electromagnetics HPC performance portability.

Author Details

Matthew T. Bettencourt

Dominic A. S. Brown

Keith L. Cartwright

Eric C. Cyr

Christian A. Glusa

Paul T. Lin

Stan G. Moore

Duncan A. O. McGregor

Roger P. Pawlowski

Edward G. Phillips

Nathan V. Roberts

Steven A. Wright

Satheesh Maheswaran

John P. Jones

Stephen A. Jarvis

  1. Characterizing the HERMES-III Extended Magnetically Insulated Transmission Line

    Powell, Troy C. | Cartwright, Keith L.

    IEEE Transactions on Plasma Science, Vol. 51 (2023), Iss. 6 P.1466

    https://doi.org/10.1109/TPS.2023.3275876 [Citations: 0]
  2. DSMC collisions under special relativity

    McDoniel, William J. | Moore, Christopher H. | Cartwright, Keith L.

    2ND INTERNATIONAL CONFERENCE ON ADVANCED EARTH SCIENCE AND FOUNDATION ENGINEERING (ICASF 2023): Advanced Earth Science and Foundation Engineering, (2024), P.080020

    https://doi.org/10.1063/5.0187894 [Citations: 1]
  3. Verification and benchmarking relativistic electron beam transport through a background gas

    Medina, Brandon M. | Grua, Pierre | Cartwright, Keith L. | Hébert, David | Szalek, Nicolas | Caizergues, Clément | Owens, Israel | Rhoades, Elaine L. | Gardelle, Jacques | Moore, Christopher H.

    Computer Physics Communications, Vol. 288 (2023), Iss. P.108721

    https://doi.org/10.1016/j.cpc.2023.108721 [Citations: 0]
  4. The influence of surface impurities on photoelectric currents driven by intense soft x rays

    Roberds, N. A.

    Physics of Plasmas, Vol. 29 (2022), Iss. 3

    https://doi.org/10.1063/5.0059958 [Citations: 0]
  5. Massively parallel axisymmetric fluid model for streamer discharges

    Fierro, A. | Alibalazadeh, A. | Stephens, J. | Moore, C.

    Computer Physics Communications, Vol. 305 (2024), Iss. P.109345

    https://doi.org/10.1016/j.cpc.2024.109345 [Citations: 0]
  6. Parasitic Modulation of Microwave Signals by a Hypersonic Plasma Layer

    Roberds, Nicholas A. | Young, Matthew W. | Miller, Nathan E. | Logemann, Caleb | Statom, Tony K. | Wagnild, Ross M.

    IEEE Transactions on Plasma Science, Vol. 52 (2024), Iss. 3 P.672

    https://doi.org/10.1109/TPS.2024.3370112 [Citations: 1]
  7. Neural-network based collision operators for the Boltzmann equation

    Miller, Sean T. | Roberts, Nathan V. | Bond, Stephen D. | Cyr, Eric C.

    Journal of Computational Physics, Vol. 470 (2022), Iss. P.111541

    https://doi.org/10.1016/j.jcp.2022.111541 [Citations: 7]
  8. Robust verification of stochastic simulation codes

    Radtke, Gregg A. | Martin, Nevin | Moore, Christopher H. | Huang, Andy | Cartwright, Keith L.

    Journal of Computational Physics, Vol. 451 (2022), Iss. P.110855

    https://doi.org/10.1016/j.jcp.2021.110855 [Citations: 5]