Year: 2021
Author: Matthew T. Bettencourt, Dominic A. S. Brown, Keith L. Cartwright, Eric C. Cyr, Christian A. Glusa, Paul T. Lin, Stan G. Moore, Duncan A. O. McGregor, Roger P. Pawlowski, Edward G. Phillips, Nathan V. Roberts, Steven A. Wright, Satheesh Maheswaran, John P. Jones, Stephen A. Jarvis
Communications in Computational Physics, Vol. 30 (2021), Iss. 4 : pp. 1232–1268
Abstract
In this paper we introduce EMPIRE-PIC, a finite element method particle-in-cell (FEM-PIC) application developed at Sandia National Laboratories. The code has been developed in C++ using the Trilinos library and the Kokkos Performance Portability Framework to enable running on multiple modern compute architectures while only requiring maintenance of a single codebase. EMPIRE-PIC is capable of solving both electrostatic and electromagnetic problems in two- and three-dimensions to second-order accuracy in space and time. In this paper we validate the code against three benchmark problems — a simple electron orbit, an electrostatic Langmuir wave, and a transverse electromagnetic wave propagating through a plasma. We demonstrate the performance of EMPIRE-PIC on four different architectures: Intel Haswell CPUs, Intel's Xeon Phi Knights Landing, ARM Thunder-X2 CPUs, and NVIDIA Tesla V100 GPUs attached to IBM POWER9 processors. This analysis demonstrates scalability of the code up to more than two thousand GPUs, and greater than one hundred thousand CPUs.
You do not have full access to this article.
Already a Subscriber? Sign in as an individual or via your institution
Journal Article Details
Publisher Name: Global Science Press
Language: English
DOI: https://doi.org/10.4208/cicp.OA-2020-0261
Communications in Computational Physics, Vol. 30 (2021), Iss. 4 : pp. 1232–1268
Published online: 2021-01
AMS Subject Headings: Global Science Press
Copyright: COPYRIGHT: © Global Science Press
Pages: 37
Keywords: PIC electrostatics electromagnetics HPC performance portability.
Author Details
-
Theoretical Assessment of the Transition Between Electron Emission Mechanisms for Nonplanar Diodes
Sinelli, Alex G. | Breen, Lorin I. | Sree Harsha, N. R. | Darr, Adam M. | Komrska, Allison M. | Garner, Allen L.IEEE Transactions on Electron Devices, Vol. 72 (2025), Iss. 1 P.397
https://doi.org/10.1109/TED.2024.3494753 [Citations: 1] -
The P3 Explorer: An Open Database of Performance, Portability, and Productivity
Smith, Matthew A. | Wright, Steven A. | Lantra, Zaman | Mudalige, Gihan2025 33rd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (PDP), (2025), P.512
https://doi.org/10.1109/PDP66500.2025.00079 [Citations: 0] -
DSMC collisions under special relativity
McDoniel, William J. | Moore, Christopher H. | Cartwright, Keith L.2ND INTERNATIONAL CONFERENCE ON ADVANCED EARTH SCIENCE AND FOUNDATION ENGINEERING (ICASF 2023): Advanced Earth Science and Foundation Engineering, (2024), P.080020
https://doi.org/10.1063/5.0187894 [Citations: 1] -
Verification and benchmarking relativistic electron beam transport through a background gas
Medina, Brandon M. | Grua, Pierre | Cartwright, Keith L. | Hébert, David | Szalek, Nicolas | Caizergues, Clément | Owens, Israel | Rhoades, Elaine L. | Gardelle, Jacques | Moore, Christopher H.Computer Physics Communications, Vol. 288 (2023), Iss. P.108721
https://doi.org/10.1016/j.cpc.2023.108721 [Citations: 0] -
The influence of surface impurities on photoelectric currents driven by intense soft x rays
Roberds, N. A.
Physics of Plasmas, Vol. 29 (2022), Iss. 3
https://doi.org/10.1063/5.0059958 [Citations: 0] -
A transient near to far field transformation method and verification benchmarking procedure
Roberds, Nicholas A. | McGregor, Duncan A. O.Journal of Electromagnetic Waves and Applications, Vol. (2025), Iss. P.1
https://doi.org/10.1080/09205071.2025.2510991 [Citations: 0] -
Massively parallel axisymmetric fluid model for streamer discharges
Fierro, A. | Alibalazadeh, A. | Stephens, J. | Moore, C.Computer Physics Communications, Vol. 305 (2024), Iss. P.109345
https://doi.org/10.1016/j.cpc.2024.109345 [Citations: 0] -
Parasitic Modulation of Microwave Signals by a Hypersonic Plasma Layer
Roberds, Nicholas A. | Young, Matthew W. | Miller, Nathan E. | Logemann, Caleb | Statom, Tony K. | Wagnild, Ross M.IEEE Transactions on Plasma Science, Vol. 52 (2024), Iss. 3 P.672
https://doi.org/10.1109/TPS.2024.3370112 [Citations: 1] -
Nonlocal, diamagnetic electromagnetic effects in magnetically insulated transmission lines
Evstatiev, E. G. | Hess, M. H. | Hamlin, N. D. | Hutsel, B. T.Physics of Plasmas, Vol. 32 (2025), Iss. 6
https://doi.org/10.1063/5.0254084 [Citations: 0] -
Robust verification of stochastic simulation codes
Radtke, Gregg A. | Martin, Nevin | Moore, Christopher H. | Huang, Andy | Cartwright, Keith L.Journal of Computational Physics, Vol. 451 (2022), Iss. P.110855
https://doi.org/10.1016/j.jcp.2021.110855 [Citations: 7] -
Characterizing the HERMES-III Extended Magnetically Insulated Transmission Line
Powell, Troy C. | Cartwright, Keith L.IEEE Transactions on Plasma Science, Vol. 51 (2023), Iss. 6 P.1466
https://doi.org/10.1109/TPS.2023.3275876 [Citations: 1] -
Theoretical Coupling of Free-Flowing Ions and Magnetically Insulated Electrons
Darr, A. M. | Cartwright, K. L.Physical Review Letters, Vol. 133 (2024), Iss. 24
https://doi.org/10.1103/PhysRevLett.133.245001 [Citations: 0] -
Neural-network based collision operators for the Boltzmann equation
Miller, Sean T. | Roberts, Nathan V. | Bond, Stephen D. | Cyr, Eric C.Journal of Computational Physics, Vol. 470 (2022), Iss. P.111541
https://doi.org/10.1016/j.jcp.2022.111541 [Citations: 10]