MPI-based Parallelization of XSTAR Photoionization Program

MPI_XSTAR is a computer program for parallelizing execution of multiple XSTAR runs using Message Passing Interface (MPI). XSTAR is part of the HEASARC’s standard HEADAS package, and is a computer program used for calculating the physical conditions and emission spectra of photoionized gases (Kallman & Bautista 2001). MPI_XSTAR uses XSTINITABLE from the HEADAS to generate a XSTAR job list, as well as initial FITS table model files. The job list is then used to make directories in ascending order, where each individual XSTAR is spawned on each processor and outputs are saved. When each processor spawns the XSTAR, the main thread is waited until the XSTAR execution is completed.

Installation

1: Pre-compile Step:

Set OpenMPI Path (bash):

export PATH="/../openmpi/bin:$PATH"
export LD_LIBRARY_PATH=/../openmpi/lib:$LD_LIBRARY_PATH

Set OpenMPI Path (cshrc): setenv PATH “/../openmpi/bin:$PATH” setenv LD_LIBRARY_PATH /../openmpi/lib:$LD_LIBRARY_PATH

2: How to compile and link:

make

How to clean:

make  clean

3: Set HEADAS Paths: Initialize the HEADAS (bash):

export HEADAS=/../heasoft/x86_64-unknown-linux-gnu-libc2.12
source $HEADAS/headas-init.sh

4: How to use mpirun for running mpixstar, for example:

mpirun --mca mpi_warn_on_fork 0 -np 8 ./mpixstar cfrac=0.4 \
temperature=100. lcpres=0 pressure=0.03 spectrum='pow' \
spectun=0 trad=-1. density=1.0e+12 densitytyp=0 \
columnsof=1.0e+20 columntyp=2 column=1.e+24 columnnst=9 columnint=1 \
rlrad38=1.e+6 rlogxityp=2 rlogxisof=0 rlogxi=5 rlogxinst=6 rlogxiint=0 \
habund=1 heabund=1 liabund=0 beabund=0 babund=0 cabund=1 \
nabund=1 oabund=1 fabund=0 neabund=1 naabund=0 mgabund=1 \
alabund=1 siabund=1 pabund=0 sabund=1 clabund=0 arabund=1 \
kabund=0 caabund=1 scabund=0 tiabund=0 vabund=0 crabund=1 \
mnabund=0 feabund=1 coabund=0 niabund=1 cuabund=0 znabund=0 \
modelname='xstar_pg1211' nsteps=10 niter=99 \
lwrite=0 lprint=1 lstep=0 emult=0.5 taumax=5. radexp=0. \
xeemin=0.1 critf=1.e-6 vturbi=100. npass=1 ncn2=9999

5: How to run mpirun on clusters and supercomputers:

• Portable Batch System (PBS):

  qsub mpixstar_run.job
  

• Sun Grid Engine (SGE):

  qsub mpixstar_run.job
  

• Simple Linux Utility for Resource Management (SLURM):

  sbatch mpixstar_run.sh
  

Acknowledgement

• A. Danehkar, M. A. Nowak, J. C. Lee, & R. K. Smith (2018) PASP 130:024501