パッケージを仮想環境でテストする。そのための備忘録。
Gerry Jenkins氏のYouTube の説明( https://www.youtube.com/watch?v=3AnlvTgsoYM )を踏襲する。
Donwload
- Ubuntu 20.04 Desktop ( https://ubuntu.com/download/desktop )
- VirtualBox 6.1.18 ( https://www.virtualbox.org )
Ubuntu20.04は、disk image (*.iso) が2.6 GB。
Install VirtualBox
- ダブルクリックして、ダウンロードマネージャ的なものに従う。
Install Ubuntu
- Users/[username]/Application/VirtualBox を開く。
- "New" をクリック
場所などを指定する
- Name: ubuntu20.04
- Machine Folder: /Users/kohei/VirtualBoxVMs
- Type: Linux
- Version: Ubuntu (64 bit)
メモリなど
-
Memory size: 4096 MB (1024 MB recommended)
-
Disk size: 35 GB
何かと不要なものがインストールされがちなので(Officeなど)、ミニマムインストールにする。
Setting for Ubuntu
Guest Addon
-
Copy & Paste / Resize the window interactively.
-
Tutorial YouTube (https://www.youtube.com/watch?v=Kbez-XdXqrw) by ProgrammingKnowledge2
$ sudo apt install linux-headers-$(uname -r) build-essential dkms
VirtualBox > Devices > Insert Guest Addition > Run > [Press Enter to exit terminal after a while]
> Power off (shut down) Ubuntu.
Virtual Box > General > Advanced > Shared Clipboard, Drag and drop 両方とも"bidirectional"に設定。
> Start Ubuntu
Shared folder
- At first I followed
https://www.youtube.com/watch?v=9-teQnZ8LEY
but it did not work. A comment posted to the same YouTube clip by Griffy DZ -- Check here : "google : VirtualBox 6.0.14: shared folder doesn't appear in “/media”" saved my day.
https://askubuntu.com/questions/1181438/virtualbox-6-0-14-shared-folder-doesnt-appear-in-media
Python
Follow the same approach as elsewhere.
$ conda activate py37
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
$ conda init <SHELL_NAME>
Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell
See 'conda init --help' for more information and options.
IMPORTANT: You may need to close and restart your shell after running 'conda init'.
$ conda init bash
shell
- bash is the default SHELL.
So one can activate py37
environment using bash
.
$ echo "$SHELL"
/bin/bash
[Different window]
(base) kohei@kohei-VirtualBox:~$ conda activate py37
(py37) kohei@kohei-VirtualBox:~$ python
Python 3.7.9 (default, Aug 31 2020, 12:42:55)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
However, because NEMO seems to use csh
, we need to use csh
.
$ csh
Command 'csh' not found, but can be installed with:
sudo apt install csh # version 20110502-5, or
sudo apt install tcsh # version 6.21.00-1
(base) kohei@kohei-VirtualBox:~$ sudo apt install csh
(base) kohei@kohei-VirtualBox:~$ csh
%
% python
Python 3.7.0 (default, Jun 28 2018, 13:15:42)
[GCC 7.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
% conda activate py37
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
$ conda init <SHELL_NAME>
Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell
See 'conda init --help' for more information and options.
IMPORTANT: You may need to close and restart your shell after running 'conda init'.
% conda init csh
ArgumentError: Invalid shells:
- csh
Currently available shells are:
- bash
- fish
- powershell
- tcsh
- xonsh
- zsh
... OK, maybe we need tcsh
.
tcsh
(base) kohei@kohei-VirtualBox:~$ tcsh
Command 'tcsh' not found, but can be installed with:
sudo apt install tcsh
(base) kohei@kohei-VirtualBox:~$ sudo apt install tcsh
(base) kohei@kohei-VirtualBox:~$ tcsh
kohei-VirtualBox:~> python
Python 3.7.0 (default, Jun 28 2018, 13:15:42)
[GCC 7.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
kohei-VirtualBox:~> conda init tcsh
no change /home/kohei/.pyenv/versions/anaconda3-5.2.0/condabin/conda
no change /home/kohei/.pyenv/versions/anaconda3-5.2.0/bin/conda
no change /home/kohei/.pyenv/versions/anaconda3-5.2.0/bin/conda-env
no change /home/kohei/.pyenv/versions/anaconda3-5.2.0/bin/activate
no change /home/kohei/.pyenv/versions/anaconda3-5.2.0/bin/deactivate
no change /home/kohei/.pyenv/versions/anaconda3-5.2.0/etc/profile.d/conda.sh
no change /home/kohei/.pyenv/versions/anaconda3-5.2.0/etc/fish/conf.d/conda.fish
no change /home/kohei/.pyenv/versions/anaconda3-5.2.0/shell/condabin/Conda.psm1
no change /home/kohei/.pyenv/versions/anaconda3-5.2.0/shell/condabin/conda-hook.ps1
no change /home/kohei/.pyenv/versions/anaconda3-5.2.0/lib/python3.7/site-packages/xontrib/conda.xsh
no change /home/kohei/.pyenv/versions/anaconda3-5.2.0/etc/profile.d/conda.csh
modified /home/kohei/.tcshrc
==> For changes to take effect, close and re-open your current shell. <==
(base) kohei@kohei-VirtualBox:~$ tcsh
kohei-VirtualBox:~> conda activate py37
(py37) kohei-VirtualBox:~> python
Python 3.7.9 (default, Aug 31 2020, 12:42:55)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
... OK, now I see that py37
environment can be used with bash
and tcsh
. This is reassuring because originally I put all the environmental variables (such as path) in .bash_profile
.
cmake
$ cmake
Command 'cmake' not found, but can be installed with:
sudo snap install cmake # version 3.19.3, or
sudo apt install cmake # version 3.16.3-1ubuntu1
See 'snap info cmake' for additional versions.
$ sudo apt install cmake
pgplot
- gcc に含まれているものの、pgplot を入れるやり方を踏襲するために、あえて gfortran を入れた。
$ sudo apt-get install xorg-dev
$ sudo apt-get install gfortran
$ cd /usr/local/src
$ sudo mv ~/Downloads/pgplot/ .
$ sudo mkdir /usr/local/pgplot
$ cd /usr/local/pgplot
$ sudo cp /usr/local/src/pgplot/drivers.list .
$ sudo nano drivers.list
... # edit contents ...
$ sudo /usr/local/src/pgplot/makemake /usr/local/src/pgplot linux g77_gcc_aout
$ sudo nano makefile
$ sudo make
$ sudo make cpg
$ sudo make clean
""".bash_profile
export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"
alias activate="source $PYENV_ROOT/versions/anaconda3-5.2.0/bin/activate"
# pgplot [ https://guaix.fis.ucm.es/~ncl/howto/howto-pgplot ]
export PGPLOT_DIR=/usr/local/pgplot
export PGPLOT_DEV=/Xserve
"""
$ source ~/.bash_profile
# demo
$ /usr/local/pgplot/pgdemo1
$ /usr/local/pgplot/pgdemo2
$ /usr/local/pgplot/pgdemo3
cfitsio
- Download 3.490 version of CFITSIO from https://heasarc.gsfc.nasa.gov/fitsio/
- Extract it under
my_libs_3
$ cd ***/my_libs_3/cfitsio-3.49
$ ./configure ### [--prefix=/target/installation/path]
$ make ### (or 'make shared')
$ make install ### (this step is optional)
hdf4
sudo apt-get install hdf4-tools libhdf4-dev libhdf4-0 libhdf4-0-alt
hdf5
sudo apt-get install hdf5-tools hdf5-helpers libhdf5-dev libhdf5-doc libhdf5-serial-dev
netcdf
https://cloud-gc.readthedocs.io/en/stable/chapter04_developer-guide/install-basic.html
(netcdfのテスト方法なども記載されている。)
$ sudo apt-get install libnetcdf-dev libnetcdff-dev
swig
$ sudo apt install swig
boost
$ sudo apt-get install libboost-all-dev
(これを走らせると、425MBくらいディスク容量を使う、とのメッセージが来る。)
build-essential
やってみたが、最新らしい。
$ sudo apt install build-essential
Reading package lists... Done
Building dependency tree
Reading state information... Done
build-essential is already the newest version (12.8ubuntu1.1).
The following packages were automatically installed and are no longer required:
libfprint-2-tod1 libllvm10
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 17 not upgraded.
install
(base) kohei@kohei-VirtualBox:~/my_libs_3$ wget https://teuben.github.io/nemo/install_nemo.sh
(base) kohei@kohei-VirtualBox:~/my_libs_3$ ls -l
total 12
drwxr-xr-x 7 kohei kohei 4096 1月 27 02:09 cfitsio-3.49
-rw-rw-r-- 1 kohei kohei 4385 1月 24 14:51 install_nemo.sh
(base) kohei@kohei-VirtualBox:~/my_libs_3$ chmod +x install_nemo.sh
(base) kohei@kohei-VirtualBox:~/my_libs_3$ ls -l
total 12
drwxr-xr-x 7 kohei kohei 4096 1月 27 02:09 cfitsio-3.49
-rwxrwxr-x 1 kohei kohei 4385 1月 24 14:51 install_nemo.sh
(base) kohei@kohei-VirtualBox:~$ source activate py37
(py37) kohei@kohei-VirtualBox:~/my_libs_3$ ./install_nemo.sh
install_nemo.sh: Version 1.1 -- 24-nov-2020
Using a direct git clone. You can can activate local hacking with:
git clone https://github.com/teuben/nemo nemo.git
Using:
nemo=nemo
branch=master
opt=1
mknemos=hdf4,hdf5,cfitsio,fftw,wcslib,gsl,netcdf4
python=0
url=https://github.com/teuben/nemo
falcon=1
yapp=pgplot
check=1
bench=1
make[1]: Leaving directory '/home/kohei/my_libs_3/nemo/src/image/cdf'
### Warning: Use rehash for access to the new programs
Continue with "make check"
(source nemo_start.sh; src/scripts/testsuite)
TESTSUITE: /home/kohei/my_libs_3/nemo/src/image/fits OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/image/io OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/image/misc OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/image/rotcur OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/image/trans OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/image/wcs OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/kernel/fortran OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/kernel/io OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/kernel/misc OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/kernel/tab OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/nbody/cores OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/nbody/evolve/aarseth/nbody0 OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/nbody/evolve/aarseth/tools OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/nbody/evolve/dehnen OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/nbody/evolve/directcode OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/nbody/evolve/flowcode OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/nbody/evolve/hackcode/hackcode1 OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/nbody/evolve/hackcode/treecode1 OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/nbody/evolve/scfm OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/nbody/evolve/sellwood/tools OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/nbody/image OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/nbody/init OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/nbody/io_nemo OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/nbody/io/starlab OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/nbody/io OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/nbody/reduc OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/nbody/trans OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/nbody/xyz OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/orbit/misc OK
TESTSUITE: /home/kohei/my_libs_3/nemo/src/orbit/potential OK
TESTSUITE: /home/kohei/my_libs_3/nemo/usr/wolfire OK
grep: tmp/test2213110/testsuite.log: No such file or directory
Results in /home/kohei/my_libs_3/nemo/tmp/test146848/testsuite.log
BSF regression results that failed:
BSF ccd.out: 41.6353 31.2251 -1 93 115 FAIL
BSF ccd.out: 41.6266 31.2366 -1 93 115 expected
BSF run4c/out.snap: -39.4595 1450.66 -53438.8 14.2075 1356 FAIL
BSF run4c/out.snap: -39.4594 1450.66 -53438.8 14.2074 1356 expected
BSF regressions that are not OK: 0
If you have any unsolved problems, this file should be saved.
TESTSUITE started 2021年 1月 27日 水曜日 12:54:18 JST
TESTSUITE ended 2021年 1月 27日 水曜日 12:54:27 JST
Continue with "make checklog" (if you see any FAILS)
(source nemo_start.sh; /usr/bin/time src/scripts/nemo.bench bsf=0)
NEMOBENCH: (2020-11-22) bench148173 : nbody0=3072 nbody1=10240 nbody3=1000000 size=256 clean=1 bsf=0 tmp=bench148173
kohei-VirtualBox
Linux kohei-VirtualBox 5.8.0-40-generic #45~20.04.1-Ubuntu SMP Fri Jan 15 11:35:04 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
2021年 1月 27日 水曜日 12:54:27 JST
Times: directcode hackcode1 mkplummer gyrfalcON potcode mkspiral ccdmath ccdpot mkorbit orbint
.dhmgpmccmo.
Skipping BSF testing. Use bsf=1 to enable this.
CPU_USAGE ccdpot : 10.62 10.54 0.00 0.00 0.00 1720071830
CPU_USAGE directcode : 3.56 3.47 0.00 0.00 0.00 1720070104
CPU_USAGE gyrfalcON : 2.18 2.15 0.00 0.00 0.00 1720070654
CPU_USAGE hackcode1 : 1.93 1.90 0.01 0.00 0.00 1720070461
CPU_USAGE mkorbit : 0.00 0.00 0.00 0.00 0.00 1720072893
CPU_USAGE mkplummer : 0.00 0.00 0.00 0.00 0.00 1720070654
CPU_USAGE mkspiral : 7.39 4.60 1.99 0.00 0.00 1720071090
CPU_USAGE orbint : 1.09 1.08 0.00 0.00 0.00 1720072893
CPU_USAGE potcode : 2.18 2.15 0.00 0.00 0.00 1720070872
Processor:
Intel(R) Core(TM) i7-7920HQ CPU @ 3.10GHz
Cleaning bench148173 (use clean=0 to prevent this)
25.97user 2.21system 0:29.30elapsed 96%CPU (0avgtext+0avgdata 174236maxresident)k
1080inputs+4413936outputs (10major+510766minor)pagefaults 0swaps
Started: 2021年 1月 27日 水曜日 12:22:26 JST
Ended: 2021年 1月 27日 水曜日 12:54:57 JST
All done.
(ba)sh users: source nemo/nemo_start.sh to activate NEMO in your shell
(t)csh users: source nemo/nemo_start.csh to activate NEMO in your shell
(py37) kohei@kohei-VirtualBox:~/my_libs_3$
なお、デスクトップで同じ作業をしたところ、最後の出力は以下の通り:
### Warning: Use rehash for access to the new programs
Continue with "make check"
(source nemo_start.sh; src/scripts/testsuite)
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/image/fits OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/image/io OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/image/misc OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/image/rotcur OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/image/trans OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/image/wcs OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/kernel/fortran OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/kernel/io OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/kernel/misc OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/kernel/tab OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/nbody/cores OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/nbody/evolve/aarseth/nbody0 OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/nbody/evolve/aarseth/tools OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/nbody/evolve/dehnen OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/nbody/evolve/directcode OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/nbody/evolve/flowcode OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/nbody/evolve/hackcode/hackcode1 OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/nbody/evolve/hackcode/treecode1 OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/nbody/evolve/scfm OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/nbody/evolve/sellwood/tools OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/nbody/image OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/nbody/init OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/nbody/io_nemo OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/nbody/io/starlab OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/nbody/io OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/nbody/reduc OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/nbody/trans OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/nbody/xyz OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/orbit/misc OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/src/orbit/potential OK
TESTSUITE: /home/nemo_kohei/my_libs_3/nemo/usr/wolfire OK
grep: tmp/test2213110/testsuite.log: No such file or directory
Results in /home/nemo_kohei/my_libs_3/nemo/tmp/test155055/testsuite.log
BSF regression results that failed:
BSF ccd.out: 41.6353 31.2251 -1 93 115 FAIL
BSF ccd.out: 41.6266 31.2366 -1 93 115 expected
BSF run4c/out.snap: -39.4595 1450.66 -53438.8 14.2075 1356 FAIL
BSF run4c/out.snap: -39.4594 1450.66 -53438.8 14.2074 1356 expected
BSF regressions that are not OK: 0
If you have any unsolved problems, this file should be saved.
TESTSUITE started Wed 27 Jan 2021 02:53:32 PM JST
TESTSUITE ended Wed 27 Jan 2021 02:53:37 PM JST
Continue with "make checklog" (if you see any FAILS)
(source nemo_start.sh; /usr/bin/time src/scripts/nemo.bench bsf=0)
NEMOBENCH: (2020-11-22) bench156378 : nbody0=3072 nbody1=10240 nbody3=1000000 size=256 clean=1 bsf=0 tmp=bench156378
Ubuntu2004-emt01
Linux Ubuntu2004-emt01 5.8.0-38-generic #43~20.04.1-Ubuntu SMP Tue Jan 12 16:39:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Wed 27 Jan 2021 02:53:37 PM JST
Times: directcode hackcode1 mkplummer gyrfalcON potcode mkspiral ccdmath ccdpot mkorbit orbint
.dhmgpmccmo.
Skipping BSF testing. Use bsf=1 to enable this.
CPU_USAGE ccdpot : 5.93 5.92 0.00 0.00 0.00 1725501595
CPU_USAGE directcode : 2.93 2.92 0.00 0.00 0.00 1725500359
CPU_USAGE gyrfalcON : 1.59 1.59 0.00 0.00 0.00 1725500802
CPU_USAGE hackcode1 : 1.49 1.49 0.00 0.00 0.00 1725500652
CPU_USAGE mkorbit : 0.00 0.00 0.00 0.00 0.00 1725502188
CPU_USAGE mkplummer : 0.00 0.00 0.00 0.00 0.00 1725500801
CPU_USAGE mkspiral : 4.66 3.53 1.12 0.00 0.00 1725501128
CPU_USAGE orbint : 0.74 0.74 0.00 0.00 0.00 1725502188
CPU_USAGE potcode : 1.67 1.66 0.00 0.00 0.00 1725500961
Processor:
AMD Ryzen Threadripper 3990X 64-Core Processor
Cleaning bench156378 (use clean=0 to prevent this)
17.90user 1.31system 0:19.23elapsed 99%CPU (0avgtext+0avgdata 174328maxresident)k
200inputs+4413936outputs (1major+510831minor)pagefaults 0swaps
Started: Wed 27 Jan 2021 02:46:12 PM JST
Ended: Wed 27 Jan 2021 02:53:57 PM JST
All done.
(ba)sh users: source nemo/nemo_start.sh to activate NEMO in your shell
(t)csh users: source nemo/nemo_start.csh to activate NEMO in your shell