Spack Installation
Prep of Compiler and MPI with Spack
Latest gcc installation
- The default gcc on CentOS7 is too old. so if you use this gcc, you must run into many problems.
- To avoid this problem, First of all let's install the latest gcc on Spack
spack install gcc
$ spack find gcc
==> 1 installed package
-- linux-centos7-x86_64_v3 / gcc@4.8.5 --------------------------
gcc@12.1.0
spack load gcc
$ gcc --version
gcc (Spack GCC) 12.1.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Register the installed gcc to Spack
spack compiler find
This case, you can see gcc@12.1.0
$ spack compilers
==> Available compilers
-- gcc centos7-x86_64 -------------------------------------------
gcc@12.1.0 gcc@4.8.5
Intel OneAPI Installation
$ spack list filter intel-oneapi
==> 27 packages.
intel-oneapi-advisor intel-oneapi-dpct intel-oneapi-mpi libnetfilter-cttimeout r-genefilter
intel-oneapi-ccl intel-oneapi-dpl intel-oneapi-tbb libnetfilter-queue r-rhdf5filters
intel-oneapi-compilers intel-oneapi-inspector intel-oneapi-vpl perl-xml-filter-buffertext ufo-filters
intel-oneapi-compilers-classic intel-oneapi-ipp intel-oneapi-vtune py-filterpy
intel-oneapi-dal intel-oneapi-ippcp libnetfilter-conntrack py-pandocfilters
intel-oneapi-dnn intel-oneapi-mkl libnetfilter-cthelper r-annotationfilter
spack install intel-oneapi-compilers %gcc@12.1.0
spack load intel-oneapi-compilers
$ icx --version
Intel(R) oneAPI DPC++/C++ Compiler 2022.1.0 (2022.1.0.20220316)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/...(snip)...
Configuration file: /home/...(snip)...
Register Intel OneAPI Compiler to Spack
command
spack compiler find
$ spack compilers
==> Available compilers
-- dpcpp centos7-x86_64 -----------------------------------------
dpcpp@2022.1.0
-- gcc centos7-x86_64 -------------------------------------------
gcc@12.1.0 gcc@4.8.5
-- intel centos7-x86_64 -----------------------------------------
intel@2021.6.0
-- oneapi centos7-x86_64 ----------------------------------------
oneapi@2022.1.0
IntelMPI Installation
command
spack install intel-oneapi-mpi %oneapi@2022.1.0
spack load intel-oneapi-mpi
command
$ mpirun --version
mpirun (Open MPI) 4.1.4
Report bugs to http://www.open-mpi.org/community/help/
OpenFOAM Installation with Spack
- Compiler:
gcc@gcc@12.1.0- If intel-oneapi-compiler, Error was got. So latest gcc is selected.
- MPI:
intel-oneapi-mpi@2021.6.0- As the result of the initial test, IntelMPI can get good performance, compared with OpenMPI
Check the hash of intel-oneapi-mpi
Please get the hash value of the installed oneapi with the following command.
In this case, hash value is 5onbafk
command
spack find -l intel-oneapi-mpi
==> 1 installed packages
-- linux-centos7-skylake_avx512 / oneapi@2022.1.0 ---------------
5onbafk intel-oneapi-mpi@2021.6.0
Define hash-value to a variable
command
MPI_HASH="5onbafk"
OpenFOAM Installation
command
spack install openfoam %gcc@12.1.0 ^/${MPI_HASH}
Load OpenFOAM
command
spack load openfoam
Check the OpenFOAM
$ interFoam -help
Usage: interFoam [OPTIONS]
Options:
-case <dir> Specify case directory to use (instead of cwd)
-decomposeParDict <file>
Use specified file for decomposePar dictionary
-dry-run Check case set-up only using a single time step
-dry-run-write Check case set-up and write only using a single time step
-parallel Run in parallel
-postProcess Execute functionObjects only
-doc Display documentation in browser
-help Display short help and exit
-help-full Display full help and exit
Solver for two incompressible, isothermal immiscible fluids using VOF
phase-fraction based interface capturing.
With optional mesh motion and mesh topology changes including adaptive
re-meshing.
Using: OpenFOAM-v2206 (2206) - visit www.openfoam.com
Build: _76d719d1e6-20220624
Arch: LSB;label=32;scalar=64
Appendix
OpenFOAM information on Spack
you can see library-dependencies and variants for the compile.
$ spack info openfoam
Package: openfoam
Description:
OpenFOAM is a GPL-opensource C++ CFD-toolbox. This offering is supported
by OpenCFD Ltd, producer and distributor of the OpenFOAM software via
www.openfoam.com, and owner of the OPENFOAM trademark. OpenCFD Ltd has
been developing and releasing OpenFOAM since its debut in 2004.
Homepage: https://www.openfoam.com/
Preferred version:
2206 https://sourceforge.net/projects/openfoam/files/v2206/OpenFOAM-v2206.tgz
Safe versions:
develop [git] https://develop.openfoam.com/Development/openfoam.git on branch develop
master [git] https://develop.openfoam.com/Development/openfoam.git on branch master
2206 https://sourceforge.net/projects/openfoam/files/v2206/OpenFOAM-v2206.tgz
2112_220610 https://sourceforge.net/projects/openfoam/files/v2112/OpenFOAM-v2112_220610.tgz
2112 https://sourceforge.net/projects/openfoam/files/v2112/OpenFOAM-v2112.tgz
2106_220610 https://sourceforge.net/projects/openfoam/files/v2106/OpenFOAM-v2106_220610.tgz
2106_211215 https://sourceforge.net/projects/openfoam/files/v2106/OpenFOAM-v2106_211215.tgz
2106 https://sourceforge.net/projects/openfoam/files/v2106/OpenFOAM-v2106.tgz
2012_220610 https://sourceforge.net/projects/openfoam/files/v2012/OpenFOAM-v2012_220610.tgz
2012_210414 https://sourceforge.net/projects/openfoam/files/v2012/OpenFOAM-v2012_210414.tgz
2012 https://sourceforge.net/projects/openfoam/files/v2012/OpenFOAM-v2012.tgz
2006_220610 https://sourceforge.net/projects/openfoam/files/v2006/OpenFOAM-v2006_220610.tgz
2006_201012 https://sourceforge.net/projects/openfoam/files/v2006/OpenFOAM-v2006_201012.tgz
2006 https://sourceforge.net/projects/openfoam/files/v2006/OpenFOAM-v2006.tgz
1912_220610 https://sourceforge.net/projects/openfoam/files/v1912/OpenFOAM-v1912_220610.tgz
1912_200506 https://sourceforge.net/projects/openfoam/files/v1912/OpenFOAM-v1912_200506.tgz
1912_200403 https://sourceforge.net/projects/openfoam/files/v1912/OpenFOAM-v1912_200403.tgz
1912 https://sourceforge.net/projects/openfoam/files/v1912/OpenFOAM-v1912.tgz
1906_200312 https://sourceforge.net/projects/openfoam/files/v1906/OpenFOAM-v1906_200312.tgz
1906_191103 https://sourceforge.net/projects/openfoam/files/v1906/OpenFOAM-v1906_191103.tgz
1906 https://sourceforge.net/projects/openfoam/files/v1906/OpenFOAM-v1906.tgz
1812_200312 https://sourceforge.net/projects/openfoam/files/v1812/OpenFOAM-v1812_200312.tgz
1812_191001 https://sourceforge.net/projects/openfoam/files/v1812/OpenFOAM-v1812_191001.tgz
1812_190531 https://sourceforge.net/projects/openfoam/files/v1812/OpenFOAM-v1812_190531.tgz
1812 https://sourceforge.net/projects/openfoam/files/v1812/OpenFOAM-v1812.tgz
1806 https://sourceforge.net/projects/openfoam/files/v1806/OpenFOAM-v1806.tgz
1712 https://sourceforge.net/projects/openfoam/files/v1712/OpenFOAM-v1712.tgz
1706 https://sourceforge.net/projects/openfoam/files/v1706/OpenFOAM-v1706.tgz
1612 https://sourceforge.net/projects/openfoam/files/v1612+/OpenFOAM-v1612+.tgz
Deprecated versions:
None
Variants:
Name [Default] When Allowed values Description
============== ==== ============== ==================================================
float32 [off] -- on, off Use single-precision
int64 [off] -- on, off With 64-bit labels
kahip [off] -- on, off With kahip decomposition
knl [off] -- on, off Use KNL compiler settings
metis [off] -- on, off With metis decomposition
mgridgen [off] -- on, off With mgridgen support
paraview [off] -- on, off Build paraview plugins and runtime post-processing
scotch [on] -- on, off With scotch/ptscotch decomposition
source [on] -- on, off Install library/application sources and tutorials
spdp [off] -- on, off Use single/double mixed precision
vtk [off] -- on, off With VTK runTimePostProcessing
zoltan [off] -- on, off With zoltan renumbering
Build Dependencies:
adios2 boost cgal cmake fftw-api flex kahip m4 metis mpi paraview parmgridgen scotch vtk zlib zoltan
Link Dependencies:
adios2 boost cgal fftw-api flex kahip metis mpi paraview scotch vtk zlib zoltan
Run Dependencies:
None
How to distinguish applications with the same version and different compile conditions on Spack
How to check the hash of an installed application and the variant-conditions on Spack
- You can see hash for each openfoam with
-loption. - You can also see Variant-Conditions for the installation with
-voption
STEP1
$ spack find -lv openfoam
==> 5 installed packages
-- linux-centos7-skylake_avx512 / gcc@12.1.0 --------------------
izv4we3 openfoam@2206~float32~int64~kahip~knl~metis~mgridgen~paraview+scotch+source~spdp~vtk~zoltan
pgmobng openfoam@2206~float32~int64~kahip~knl~metis~mgridgen~paraview+scotch+source~spdp~vtk~zoltan
ovt7fb4 openfoam@2206~float32~int64~kahip~knl~metis~mgridgen~paraview+scotch+source~spdp~vtk~zoltan
sgalfui openfoam@2206~float32~int64~kahip~knl~metis~mgridgen~paraview+scotch+source~spdp~vtk~zoltan
-- linux-centos7-skylake_avx512 / oneapi@2022.1.0 ---------------
bs6ib3o openfoam@2206~float32~int64~kahip~knl~metis~mgridgen~paraview+scotch+source~spdp~vtk~zoltan
Check the lib-dependencies
- if you want to check Library dependencies for a certain openfoam, please get the hash of the application above. for example
izv4we3 - The hash you want to check is decided, you can see lib-dependencies with the following commands.
- For example, This is useful if you want to switch to openfoam with different MPI-lib
HASH="izv4we3"
$ spack find -ld /${HASH}
==> 1 installed package
-- linux-centos7-skylake_avx512 / gcc@12.1.0 --------------------
izv4we3 openfoam@2206
xxvcgxf adios2@2.8.1
uxeczyl bzip2@1.0.8
26nu5db diffutils@3.8
e2jcaca libiconv@1.16
bbo5g7r c-blosc@1.21.1
ptkahsc cmake@3.23.2
fov7vip ncurses@6.2
wkherf6 pkgconf@1.8.0
jri63p6 openssl@1.1.1p
itda7co ca-certificates-mozilla@2022-03-29
pytjagw perl@5.34.1
6vpregl berkeley-db@18.1.40
uwcdtdl gdbm@1.19
sdbhx7q readline@8.1.2
iffomx3 zlib@1.2.12
oh3fgkz lz4@1.9.3
p3h5hwm snappy@1.1.8
ub4z64d zstd@1.5.2
5onbafk intel-oneapi-mpi@2021.6.0
m6zrw3m libfabric@1.14.1
5fii44x libffi@3.4.2
tktzhgt libpng@1.6.37
ievzj2o sz@2.1.12.2
k33yqv7 zfp@0.5.5
aqltvxd boost@1.79.0
chmrkhm cgal@4.13
otkgdzw eigen@3.4.0
adxc7gc gmp@6.2.1
ps5hqpj autoconf@2.69
s3o4eaq m4@1.4.19
bb74ie4 libsigsegv@2.13
gi7vs7a automake@1.16.5
pcqswzt libtool@2.4.7
dwe6krm mpfr@4.1.0
uunlpwt autoconf-archive@2022.02.11
q5xx7dm texinfo@6.5
kdfbbt2 fftw@3.3.10
xopdau4 flex@2.6.4
jzsrd5k bison@3.8.2
wzibbwh findutils@4.9.0
au3gxwj gettext@0.21
zyqemyn libxml2@2.9.13
ys73qyv xz@5.2.5
xiyb6ww tar@1.34
mdkeqw7 pigz@2.7
3jxn4if help2man@1.47.16
tzdw73i scotch@7.0.1
Sample: Check mpi dependencies at once.
$ spack find -ld openfoam | grep -e openfoam@2206 -e mpi
izv4we3 openfoam@2206
5onbafk intel-oneapi-mpi@2021.6.0
pgmobng openfoam@2206
7dvvkku openmpi@4.1.4
ovt7fb4 openfoam@2206
2aoek3i openmpi@4.1.4
sgalfui openfoam@2206
sm6zwel openmpi@4.1.4
bs6ib3o openfoam@2206
sm6zwel openmpi@4.1.4