GAMESS-USをソースからビルドしたときの個人的メモです。

環境

MacOS 11.4 (Big Sur), M1 chip, 16 GB RAM
Ubuntu 20.04, Ryzen 7 3800X, 32 GB RAM

2021年7月現在のGAMESS最新版（ver. June 30, 2021 R1）でM1 chipのMacに対応したみたいです。

事前準備

GAMESSの公式ウェブサイトからダウンロードページに入り、License Agreementに同意し、Source Code Distributionをリクエストする。
しばらくすると、登録したメールアドレスにソースコードのダウンロードに必要なユーザー名とパスワードが届く。
ユーザー名はsourceで固定だが、パスワードは毎週変更されるようだ。

MacOSでのインストール方法

gfortranのバージョンを確認しておく。

$ gfortran --version
GNU Fortran (Homebrew GCC 11.1.0_1) 11.1.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

ソースコードを適当なディレクトリ（ここでは${HOME}/appsとする）にダウンロードし、展開する。

cd ${HOME}/apps
wget https://www.msg.chem.iastate.edu/GAMESS/download/source/gamess-current.tar.gz \
    --user=source --password=hoge --no-check-certificate
tar zxvf gamess-current.tar.gz

hogeはメールに書かれていたパスワードに変える。
展開されたフォルダに移動し、configを起動する。

cd gamess
./config

色々と質問されるので答えていく。

This script asks a few questions, depending on your computer system,
to set up compiler names, libraries, message passing libraries,
and so forth.
 
You can quit at any time by pressing control-C, and then <ENTER>.
 
Please open a second window by logging into your target machine,
in case this script asks you to 'type' a command to learn something
about your system software situation.  All such extra questions will
use the word 'type' to indicate it is a command for the other window.

After the new window is open, please hit <ENTER> to go on.

<ENTER>を押す。

GAMESS can compile on the following 32 bit or 64 bit machines:
hpe-cray-ex  - HPE Cray's EX architecture (e.g., narwhal, spock)
hpe-cray-cs  - HPE Cray's CS architecture (e.g., tulip)
cray-xt      - Cray's XT massively parallel system, running CNL
cray-xc      - Cray's XC massively parallel system (e.g., DoE Theta, DoD Onyx)
fj-a64fx     - Fujitsu PRIMEHPC with A64FX cpus and Fujitsu Compilers (e.g., FX1000, FX700)
ibm64        - IBM, Power8 chip or newer, running AIX or Linux (e.g., DoE Summit, DoD Hokulea)
linux32      - Linux (any 32 bit distribution), for x86
linux64      - Linux (any 64 bit distribution), for x86_64, ia64, or arm64 chips,
               using gfortran, ifort, or perhaps PGI compilers.
mac64        - Apple Mac, any chip, running OS X 10.5 or newer
win64        - Windows 64-bit (Windows 10)
singularity  - GAMESS Singularity container image
 
type 'uname -a' to partially clarify your computer's flavor.
 
please enter your target machine name:

mac64と入力し、<ENTER>を押す。

Where is the GAMESS software on your system?
A typical response might be /u1/mike/gamess,
most probably the correct answer is /Users/taiyocching/apps/gamess
 
GAMESS directory? [/Users/taiyocching/apps/gamess]

<ENTER>を押す。

Setting up GAMESS compile and link for GMS_TARGET=mac64
GAMESS software is located at GMS_PATH=/Users/taiyocching/apps/gamess
 
Please provide the name of the build location.
This may be the same location as the GAMESS directory.

GAMESS build directory? [/Users/taiyocching/apps/gamess]

<ENTER>を押す。

Please provide a version number for the GAMESS executable.
This will be used as the middle part of the binary's name,
for example: gamess.00.x

Version? [00]

<ENTER>を押す。

Compiling on Apple equipment is straightforward.
 
This target assumes the use of the GNU compilers, gfortran/gcc.
Information about first installing Xcode from Apple, to get gcc,
and then obtaining a precompiled gfortran for your Apple OS X version
can be found in the 'comp' script.
 
Please type 'gfortran -dumpversion' or else 'gfortran -v' to
detect the version number of your gfortran.
This reply should be a string with at least two decimal points,
such as 4.1.2 or 4.6.1, or maybe even 4.4.2-12.
The reply may be labeled as a 'gcc' version,
but it is really your gfortran version.

Please enter only the first decimal place, such as 4.6 or 4.8:

上で確認したバージョン番号11.1を入力し、<ENTER>を押す。

GAMESS-US now supports LibXC.
LibXC is a library of exchange-correlation functionals for density-functional
theory (DFT). It adds many new DFT functionals that are enabled with the
$LIBXC input group. This feature requires modern Fortran 2003 but not all
compilers support it. For gfortran, the minimal tested version is gfortran 5.5.
If your gfortran is older than this, you can try to use the LibXC interface
but it may not work. The IBM XL Fortran compiler is not fully supported!

This option can be manually changed later by modifying install.info

Optional: Build LibXC interface?  (yes/no):

noと入力し、<ENTER>を押す。

Users have the option of compiling the beta version of the
active-space CCSDt and CC(t;3) codes developed at Michigan
State University (CCTYP = CCSD3A and CCT3, respectively).

These builds take a considerable amount of time and memory for
compilation due to the amount of machine generated source code.
We recommend that users interested in installing these codes
compile GAMESS in parallel using the Makefile generated during
the initial configuration ('make -j [number of cores]').

This option can be manually changed later by modifying install.info

Optional: Build Michigan State University CCT3 & CCSD3A methods?  (yes/no):

noと入力し、<ENTER>を押す。これにてconfigは終了。
次にDDI（Distributed Data Interface）なるGAMESS独自の通信ライブラリ（？）をビルドする。

cd ddi
./compddi 2>&1 | tee compddi.log

うまく行っていればDDI compilation ended successfully.と表示される。
生成されたddikick.xを上のディレクトリに移動させ、もとのディレクトリに戻る。

mv ddikick.x ..
cd ..

GAMESS本体をビルド・リンクする。

./compall 2>&1 | tee compall.log
./lked 2>&1 | tee lked.log

The linking of GAMESS to binary gamess.00.x was successful.と表示されたらたぶんOK。
次に適当なテキストエディタでGAMESSの実行スクリプトであるrungmsを開き、63-66行目を以下の様に変更する。

rungms

set TARGET=sockets
set SCR=/tmp
set USERSCR=/tmp
set GMSPATH=${HOME}/apps/gamess

各項目の意味については上の方の行に説明がある。
次に、正しくビルドできているかテストを行う。

./runall 00
tests/standard/checktst

ちなみに1つ目のコマンドの00は、ビルドしたバイナリを指定する引数で、gamess.00.xを使うという意味。
私の環境だとすぐにテストが終了し、48個中47個のテストに失敗した。
失敗したlogファイルの中身を見てみると、共有メモリが足りてない的なことが書いてあった。

exam01.log

DDI Process 0: shmget returned an error.
Error EINVAL: Attempting to create 160525768 bytes of shared memory.
Check system limits on the size of SysV shared memory segments.

共有メモリの設定を確認してみる。

$ sysctl -a | grep kern.sysv.shm     
kern.sysv.shmall: 1024
kern.sysv.shmmax: 4194304
kern.sysv.shmmin: 1
kern.sysv.shmmni: 32
kern.sysv.shmseg: 8

kern.sysv.shmallとkern.sysv.shmmaxの値が小さく、全然足りていないことがわかる（各パラメータの意味についてはこのページが参考になる）。
以下のコマンドで設定を変更する。

$ sudo sysctl kern.sysv.shmall=1048576
kern.sysv.shmall: 1024 -> 1048576
$ sudo sysctl kern.sysv.shmmax=4294967296
kern.sysv.shmmax: 4194304 -> 4294967296

logファイルおよび/tmpフォルダ内の一時ファイルを削除し、再度テストを実行したところ、All 48 test results are correct!と表示されて成功した。
なお、この設定は再起動するとデフォルト値に戻ってしまう。
毎回設定するのが面倒な場合は、/Library/LaunchDaemons/com.startup.sysctl.plistを作成する。

com.startup.sysctl.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.startup.sysctl</string>
    <key>LaunchOnlyOnce</key>
    <true/>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/sbin/sysctl</string>
        <string>kern.sysv.shmall=1048576</string>
        <string>kern.sysv.shmmax=4294967296</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
</dict>
</plist>

次いでcom.startup.sysctl.plistの所有者/所有グループを変更し、再起動すると設定が反映されている。

$ sudo chown root:wheel /Library/LaunchDaemons/com.startup.sysctl.plist
$ sudo reboot
$ sysctl -a | grep kern.sysv.shm
kern.sysv.shmall: 1048576
kern.sysv.shmmax: 4294967296
kern.sysv.shmmin: 1
kern.sysv.shmmni: 32
kern.sysv.shmseg: 8

なお、/etc/sysctl.confに設定を書き込むという方法はBig Surでは廃止されているので注意（下記ページ参照）。

Ubuntuでのインストール方法

基本的にはMacOSと同様だが、BLASを別途インストールしておく必要がある。
今回はOpenBLASをソースからビルドして使うことにした。

$ git clone https://github.com/xianyi/OpenBLAS.git
$ cd OpenBLAS
$ make -j16

### 中略 ###

OpenBLAS build complete. (BLAS CBLAS LAPACK LAPACKE)

 OS               ... Linux             
 Architecture     ... x86_64               
 BINARY           ... 64bit                 
 C compiler       ... GCC  (cmd & version : cc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0)
 Fortran compiler ... GFORTRAN  (cmd & version : GNU Fortran (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0)
 Library Name     ... libopenblas_zenp-r0.3.17.dev.a (Multi-threading; Max num-threads is 16)

$ make install PREFIX=${HOME}/apps/openblas

MacOSの場合と同様にしてインストールを進める。

$ gfortran --version
GNU Fortran (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ cd ${HOME}/apps
$ wget https://www.msg.chem.iastate.edu/GAMESS/download/source/gamess-current.tar.gz \
    --user=source --password=hoge --no-check-certificate
$ tar zxvf gamess-current.tar.gz
$ cd gamess
$ ./config

configの質問内容は、MacOSの場合とは若干異なるので一応記載する。

GAMESS can compile on the following 32 bit or 64 bit machines:
hpe-cray-ex  - HPE Cray's EX architecture (e.g., narwhal, spock)
hpe-cray-cs  - HPE Cray's CS architecture (e.g., tulip)
cray-xt      - Cray's XT massively parallel system, running CNL
cray-xc      - Cray's XC massively parallel system (e.g., DoE Theta, DoD Onyx)
fj-a64fx     - Fujitsu PRIMEHPC with A64FX cpus and Fujitsu Compilers (e.g., FX1000, FX700)
ibm64        - IBM, Power8 chip or newer, running AIX or Linux (e.g., DoE Summit, DoD Hokulea)
linux32      - Linux (any 32 bit distribution), for x86
linux64      - Linux (any 64 bit distribution), for x86_64, ia64, or arm64 chips,
               using gfortran, ifort, or perhaps PGI compilers.
mac64        - Apple Mac, any chip, running OS X 10.5 or newer
win64        - Windows 64-bit (Windows 10)
singularity  - GAMESS Singularity container image
 
type 'uname -a' to partially clarify your computer's flavor.
 
please enter your target machine name:

linux64と入力し、<ENTER>を押す。

Where is the GAMESS software on your system?
A typical response might be /u1/mike/gamess,
most probably the correct answer is /home/taiyocching/apps/gamess
 
GAMESS directory? [/home/taiyocching/apps/gamess]

<ENTER>を押す。

Setting up GAMESS compile and link for GMS_TARGET=linux64
GAMESS software is located at GMS_PATH=/home/taiyocching/apps/gamess
 
Please provide the name of the build location.
This may be the same location as the GAMESS directory.
 
GAMESS build directory? [/home/tai/apps/gamess]

<ENTER>を押す。

Please provide a version number for the GAMESS executable.
This will be used as the middle part of the binary's name,
for example: gamess.00.x

Version? [00]

<ENTER>を押す。

Linux offers many choices for FORTRAN compilers, including the GNU
compiler suite's free compiler 'gfortran', usually included in
any Linux distribution.  If gfortran is not installed, it can be
installed from your distribution media.
 
To check on installed GNU compilers, for RedHat/SUSE style Linux,
   type 'rpm -aq | grep gcc' for both languages,
and for Debian/Ubuntu style Linux, it takes two commands
   type 'dpkg -l | grep gcc'
   type 'dpkg -l | grep gfortran'
 
There are also other compilers (some commercial), namely Intel's 'ifort',
Portland Group's 'pgfortran', AMD's 'AOCC',
and ARM's armflang.
The last four are not common, and aren't as well tested.
 
type 'which gfortran'  to look for GNU's gfortran   (a good choice),
type 'which ifort'     to look for Intel's compiler (a good choice),
type 'which pgfortran' to look for Portland Group's compiler,
type 'which aocc'      to look for AMD's compiler.
type 'which armflang'  to look for ARM compiler.
Please enter your choice of FORTRAN:

gfortranと入力し、<ENTER>を押す。

Please type 'gfortran -dumpversion' or else 'gfortran -v' to
detect the version number of your gfortran.
This reply should be a string with at least two decimal points,
such as 4.9.4 or 6.3.0.
The reply may be labeled as a 'gcc' version,
but it is really your gfortran version.
 
Please enter only the first decimal place, such as 4.9:

9.3と入力し、<ENTER>を押す。

Linux distributions do not include a standard math library.
 
There are several reasonable add-on library choices,
       MKL from Intel           for 32 or 64 bit Linux (very fast)
      ACML from AMD             for 32 or 64 bit Linux (free)
     LibFLAME from AMD          for 64 bit Linux (free)
     ATLAS from www.rpmfind.net for 32 or 64 bit Linux (free)
  PGI BLAS from Portland Group  for 32 or 64 bit Linux
     ArmPL from ARM             for 64 bit Linux
and one very unreasonable option, namely 'none', which will use
some slow FORTRAN routines supplied with GAMESS.  Choosing 'none'
will run MP2 jobs 2x slower, or CCSD(T) jobs 5x slower.
 
Some typical places (but not the only ones) to find math libraries are
Type 'ls /opt/intel/mkl'                 to look for MKL
Type 'ls /opt/intel/Compiler/mkl'        to look for MKL
Type 'ls /opt/intel/composerxe/mkl'      to look for MKL
Type 'echo $MKLROOT'                     to look for MKL
Type 'ls -d /opt/acml*'                  to look for ACML
Type 'ls -d /usr/local/acml*'            to look for ACML
Type 'ls /usr/lib64/atlas'               to look for Atlas
Type 'ls /opt/pgi/linux86-64/*/lib/*     to look for libblas.a from PGI
Type 'ls /opt/pgi/osx86-64/*/lib/*       to look for libblas.a from PGI
Type 'echo $ARMPL_DIR'                   to look for ArmPL
 
Enter your choice of 'mkl' or 'atlas' or 'acml' or 'libflame' or 'openblas' or 'pgiblas' or 'armpl' or 'none':

openblasと入力し、<ENTER>を押す。

Where is your libopenblas.a file located?
 
enter this full pathname:

/home/taiyocching/apps/openblas/libと入力し、<ENTER>を押す。

If you have a slow network, like Gigabit Ethernet (GE), or
if you have so few nodes you won't run extensively in parallel, or
if you have no MPI library installed, or
if you want a fail-safe compile/link and easy execution,
     choose 'sockets'
to use good old reliable standard TCP/IP networking.
 
If you have an expensive but fast network like Infiniband (IB), and
if you have an MPI library correctly installed,
     choose 'mpi'.
 
If you wish to use a combination of TCP/IP networking for small
messages and MPI for large messages in a 'mixed' fashion,
     choose 'mixed'.
 
communication library ('serial','sockets' or 'mpi' or 'mixed')?

socketsと入力し、<ENTER>を押す。

GAMESS-US now supports LibXC.
LibXC is a library of exchange-correlation functionals for density-functional
theory (DFT). It adds many new DFT functionals that are enabled with the
$LIBXC input group. This feature requires modern Fortran 2003 but not all
compilers support it. For gfortran, the minimal tested version is gfortran 5.5.
If your gfortran is older than this, you can try to use the LibXC interface
but it may not work. The IBM XL Fortran compiler is not fully supported!

This option can be manually changed later by modifying install.info

Optional: Build LibXC interface?  (yes/no):

noと入力し、<ENTER>を押す。

Users have the option of compiling the beta version of the
active-space CCSDt and CC(t;3) codes developed at Michigan
State University (CCTYP = CCSD3A and CCT3, respectively).

These builds take a considerable amount of time and memory for
compilation due to the amount of machine generated source code.
We recommend that users interested in installing these codes
compile GAMESS in parallel using the Makefile generated during
the initial configuration ('make -j [number of cores]').

This option can be manually changed later by modifying install.info

Optional: Build Michigan State University CCT3 & CCSD3A methods?  (yes/no):

noと入力し、<ENTER>を押す。

64 bit Linux and IBM builds can attach a special LIBCCHEM code for fast
MP2 and CCSD(T) runs.  The LIBCCHEM code can utilize nVIDIA GPUs,
through the CUDA libraries, if GPUs are available.
Usage of LIBCCHEM requires installation of HDF5 I/O software as well.
GAMESS+LIBCCHEM binaries are unable to run most of GAMESS computations,
and are a bit harder to create due to the additional CUDA/HDF5 software.
Therefore, the first time you run 'config', the best answer is 'no'!
If you decide to try LIBCCHEM later, just run this 'config' again.
 
Do you want to try LIBCCHEM?  (yes/no):

noと入力し、<ENTER>を押すとconfigは終了。
後はMacOSの場合と全く同様にしてインストールできた。
Ubuntuの場合、テストの際に共有メモリに関するエラーは出なかった。

計算実行に関するメモ

下記のaliasを設定しておくと便利。

.zshrc

alias rungms=${HOME}/apps/gamess/rungms

計算を実行するときは、rungms input.inp 00 8 >& output.logなどとする。
なお、00はconfig時に指定したバイナリの名称（すなわちgamess.00.xを使うということ）、8は計算に使用するコア数である。

インプットファイルはAvogadroなどを使って作成するといいと思う。

OpenBLASを使用する場合、OpenBLAS自体のスレッディングをオフにするとGAMESSの実行速度が上がるという検証結果があり、当環境でも再現性が確認できた。

よって、.zshrcに下記の一文を追加しておくとよい。

.zshrc

export OPENBLAS_NUM_THREADS=1