#はじめに
OpenShiftのお勉強のログです。
OpenShiftのクラスターをまともに構成しようとするとそれなりにリソースが必要になりますし、Cloud上のマネージドサービスを使用するにも有償になるのでなかなか手が出しにくいです。
ですが、CodeReady Containersというシングルノードで構成可能なパッケージ(minikubeのようなもの)が提供されているようなので、WindowsPC上にCodeReady Containersの環境を構築してOpenShift環境をいじってみようと思います。
ここでは、CodeReady Containers V1.10を扱います(記事執筆時点での最新)。
さて、このCodeReady Containersというのは、Windows, MacOS, Linux版が提供されていますが、それぞれ以下の仮想化技術が使われるようです。
- Windows: Hyper-V
- MacOS: HyperKit
- Linux: Libvirt
私が使用しているWindows環境ではVirtualBoxをフル活用しているので、Hyper-Vは出来るだけ使いたくありません。以前はHyper-VとVirtualBoxの共存はNGでした。最近はどうやらHyper-VもVirtualBoxもNested Virtualizationという機能がサポートされたようで、その辺の制限が緩和されてきたような記事も見られるのですが、無条件に共存可という訳でもなさそうで挙動はかなりあやしそうです。
で、以下の構成を試しました。
シナリオ(1) Windows10のVirtualBoxにCentOSを入れて、そこでLinux版CodeReady(Libvirt)をセットアップ
シナリオ(2) Windows10のHyper-Vを有効化し、Windows版CodeReadyをセットアップ
これまでのVirtualBox資産を有効活用したいので(1)が動けばよかったのですが、結論から言うとNGでした。crc startの途中でCPUが100%張り付いたまま固まります(1時間以上ねばりましたが反応無く断念)。
VirtualBox 6.1でIntel CPU上のNested Virtualizationがサポート
この辺の記事によると仮想化+仮想化なのでオーバーヘッドがかかるようで、CPUもさることながらディスクアクセスが極端に遅くなっている例もあるようです。
以前minikubeをvirtualbox上に構成していたので、同じような感覚で考えていたのですがminikubeとはちょっと仮想化のレベルが違いますね。
しょうがないので(2)を試したら、こちらは一応なんとか動いたようですが、若干挙動があやしいです。
VirtualBoxを使用するにはHyper-Vを無効化する必要があり、そのためにはWindows再起動が必要になるので相当鬱陶しいのですが...。なんとかならんのかねー。
##関連記事
Red Hat OpenShift Study - (1)CodeReady Containers on Windows10
Red Hat OpenShift Study - (2)Windows10上でOpenShift CLI(oc)を使ってみる
#参考情報
Red Hat CodeReady Containers 1.10 - Getting Started Guide
ダウンロード
Red Hat CodeReady Containers (OpenShift4) のインストールのメモ
【OpenShift 4】CodeReady Containers 1.2.0をWindowsで動かそうとしたら大変だった話
Can crc be run in a CentOS box on VirtualBox?
VirtualBox で仮想マシンが入れ子 (Nested Virtualization) できるようになった
VirtualBox 6.1でIntel CPU上のNested Virtualizationがサポート
Hyper-Vネットワークの基本
Hyper-Vのネットワーク設定についての覚え書き
Installing CodeReady Containers (OpenShift4) on Hyper-V (Win 10)
Hyper-Vで物理マシンと仮想マシンの両方をインターネット接続するための仮想スイッチの種類
Windows 10 with Hyper-V で NAT を使用する
#環境情報
今回使用する環境情報です。
VirtualBox V6.1からIntlのNested Virtualizationがサポートされたようなので、VirtualBoxを最新版(6.1.8)に上げました。
H/W
ThinkPad T480
CPU: Intel Core i5-8350U 1.70GHz (4core 8スレッド)
Memory: 32GB
SSD
HostOS
Windows10 Enterprise
VirtualBox V6.1.8
ちなみに、CodeReady Containers V1.10のHardware要件はこんな感じです。
参考: Hardware requirements
4 virtual CPUs (vCPUs)
9 GB of free memory
35 GB of storage space
Note
The OpenShift cluster requires these minimum resources to run in the CodeReady Containers virtual machine. Some workloads may require more resources.
#環境セットアップ: シナリオ(1) / 失敗編
VirtualBox上にCentOS8.1入れて、その上にCodeReady Containersをセットアップするシナリオです。
仮想マシンのリソース割り当て
vCPU: 7core
Memory: 28GB
Disk: 200GB
CentOS V8.1
CentOSインストール
VirtualBox上にCentOS8.1をインストールします。
ISOイメージを以下からダウンロード
Download CentOS
適当にこんな感じで。
ベース環境:
サーバー(GUI使用)
その他のソフトウェア:
デバッグツール
FTPサーバー
ネットワークファイルシステムクライアント
パフォーマンスツール
コンテナ管理
開発ツール
グラフィカル管理ツール
システムツール
前提としてlibvirtとNetworkManagerのパッケージが必要ということなので確認しておく。
# yum list installed | grep NetworkManager
...
NetworkManager.x86_64 1:1.20.0-3.el8 @anaconda
NetworkManager-adsl.x86_64 1:1.20.0-3.el8 @anaconda
NetworkManager-bluetooth.x86_64 1:1.20.0-3.el8 @anaconda
NetworkManager-config-server.noarch 1:1.20.0-3.el8 @anaconda
NetworkManager-libnm.x86_64 1:1.20.0-3.el8 @anaconda
NetworkManager-libreswan.x86_64 1.2.10-4.el8 @AppStream
NetworkManager-team.x86_64 1:1.20.0-3.el8 @anaconda
NetworkManager-tui.x86_64 1:1.20.0-3.el8 @anaconda
NetworkManager-wifi.x86_64 1:1.20.0-3.el8 @anaconda
NetworkManager-wwan.x86_64 1:1.20.0-3.el8 @anaconda
...
[root@test12 ~]# yum list installed | grep libvirt
...
libvirt.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-bash-completion.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-client.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-daemon.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-daemon-config-network.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-daemon-config-nwfilter.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-daemon-driver-interface.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-daemon-driver-network.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-daemon-driver-nodedev.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-daemon-driver-nwfilter.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-daemon-driver-qemu.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-daemon-driver-secret.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-daemon-driver-storage.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-daemon-driver-storage-core.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-daemon-driver-storage-disk.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-daemon-driver-storage-gluster.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-daemon-driver-storage-iscsi.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-daemon-driver-storage-logical.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-daemon-driver-storage-mpath.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-daemon-driver-storage-rbd.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-daemon-driver-storage-scsi.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-daemon-kvm.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
libvirt-gconfig.x86_64 2.0.0-1.el8 @AppStream
libvirt-glib.x86_64 2.0.0-1.el8 @AppStream
libvirt-gobject.x86_64 2.0.0-1.el8 @AppStream
libvirt-libs.x86_64 4.5.0-35.3.module_el8.1.0+297+df420408 @AppStream
...
CodeReady Containersのセットアップ
ダウンロード
上のサイトから、crc-linux-amd64.tar.xzと、pull secretファイル(pull-secret.txt)をダウンロードしておきます。
前提としてroot以外のユーザーを使用し、sudoが実行できるようにしておきます。
tar.xzを解凍
[user01@test12 /Inst_Image/OpenShift]$ tar -Jxvf crc-linux-amd64.tar.xz
crc-linux-1.10.0-amd64/
crc-linux-1.10.0-amd64/LICENSE
crc-linux-1.10.0-amd64/doc.pdf
crc-linux-1.10.0-amd64/crc
解凍したディレクトリに移動してcrc setupコマンド実行
[user01@test12 /Inst_Image/OpenShift/crc-linux-1.10.0-amd64]$ ./crc setup
INFO Checking if oc binary is cached
INFO Caching oc binary
INFO Checking if podman remote binary is cached
INFO Checking if CRC bundle is cached in '$HOME/.crc'
INFO Unpacking bundle from the CRC binary
INFO Checking if running as non-root
INFO Checking if Virtualization is enabled
INFO Setting up virtualization
FATA You need to enable virtualization in BIOS
ふむ。
VirtualBoxの設定を見ると「ネステッドVT-x/AMD-Vを有効化」にチェックが入っていなかったのでチェックして再起動
リトライ
[user01@test12 /Inst_Image/OpenShift/crc-linux-1.10.0-amd64]$ ./crc setup
INFO Checking if oc binary is cached
INFO Checking if podman remote binary is cached
INFO Checking if CRC bundle is cached in '$HOME/.crc'
INFO Checking if running as non-root
INFO Checking if Virtualization is enabled
INFO Checking if KVM is enabled
INFO Checking if libvirt is installed
INFO Installing libvirt service and dependencies
INFO Will use root access: install virtualization related packages
INFO Checking if user is part of libvirt group
INFO Adding user to libvirt group
INFO Will use root access: add user to libvirt group
INFO Checking if libvirt is enabled
INFO Checking if libvirt daemon is running
INFO Checking if a supported libvirt version is installed
INFO Checking if crc-driver-libvirt is installed
INFO Installing crc-driver-libvirt
INFO Checking for obsolete crc-driver-libvirt
INFO Checking if libvirt 'crc' network is available
INFO Setting up libvirt 'crc' network
INFO Checking if libvirt 'crc' network is active
INFO Starting libvirt 'crc' network
INFO Checking if NetworkManager is installed
INFO Checking if NetworkManager service is running
INFO Checking if /etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf exists
INFO Writing Network Manager config for crc
INFO Will use root access: write NetworkManager config in /etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf
INFO Will use root access: Changing permission for /etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf to 420
INFO Will use root access: execute systemctl daemon-reload command
INFO Will use root access: execute systemctl stop/start command
INFO Checking if /etc/NetworkManager/dnsmasq.d/crc.conf exists
INFO Writing dnsmasq config for crc
INFO Will use root access: write dnsmasq configuration in /etc/NetworkManager/dnsmasq.d/crc.conf
INFO Will use root access: Changing permission for /etc/NetworkManager/dnsmasq.d/crc.conf to 420
INFO Will use root access: execute systemctl daemon-reload command
INFO Will use root access: execute systemctl stop/start command
Setup is complete, you can now run 'crc start' to start the OpenShift cluster
セットアップは完了したらしい。
起動 => ハング状態
以下のコマンドで起動を試みると...
[user01@test12 /Inst_Image/OpenShift/crc-linux-1.10.0-amd64]$ ./crc start -p ../pull-secret.txt
INFO Checking if oc binary is cached
INFO Checking if podman remote binary is cached
INFO Checking if running as non-root
INFO Checking if Virtualization is enabled
INFO Checking if KVM is enabled
INFO Checking if libvirt is installed
INFO Checking if user is part of libvirt group
INFO Checking if libvirt is enabled
INFO Checking if libvirt daemon is running
INFO Checking if a supported libvirt version is installed
INFO Checking if crc-driver-libvirt is installed
INFO Checking if libvirt 'crc' network is available
INFO Checking if libvirt 'crc' network is active
INFO Checking if NetworkManager is installed
INFO Checking if NetworkManager service is running
INFO Checking if /etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf exists
INFO Checking if /etc/NetworkManager/dnsmasq.d/crc.conf exists
INFO Loading bundle: crc_libvirt_4.4.3.crcbundle ...
INFO Checking size of the disk image /home/user01/.crc/cache/crc\libvirt_4.4.3/crc.qcow2 ...
INFO Creating CodeReady Containers VM for OpenShift 4.4.3...
上の状態でCPU張り付いた状態になり、GuestOS側の操作が何もできなくなりました。1時間以上粘りましたがそこまで反応無いと使い物にならないので断念しました。
#環境セットアップ: シナリオ(2) / 成功編?
Windows10のHyper-Vを有効化して、Windows版CodeReady Containersをセットアップしてみます。
Hyper-V有効化
VirtualBoxのためにこれまで封印してきたHyper-Vをいよいよ有効化します。
まずはHyper-Vのインストール。以下に従ってHyper-V機能をインストールします。
Windows 10 上に Hyper-V をインストールする
Hyper-V機能をインストールして再起動すると有効化されます。
参考: Hyper-V 有効化/無効化切替え
Hyper-V機能をインストール/アンインストールで切り替えるのはちょっと面倒ですよね。この機能の有効化/無効化はコマンドで制御できるようです。いずれにしても再起動は必要ですが。
Hyper-VのON・OFFを簡単に自動化すっぞ！！
以下のような単純なバッチを作ってPATHに通しておきます(再起動は手動でやりたいのでとりあえずパラメーターの設定変更のみバッチに組み込む)。
hyperv.bat
@echo off
setlocal enabledelayedexpansion
set var=%1
if "%var%" == "on" (
echo set Hyper-V: ON
bcdedit /set hypervisorlaunchtype auto
echo please reboot!
echo.
) else if "%var%" == "off" (
echo set Hyper-V: OFF
bcdedit /set hypervisorlaunchtype off
echo please reboot!
echo.
) else if "%var%" == "status" (
echo check status
bcdedit /enum | find "hypervisorlaunchtype"
echo.
) else (
echo "Usage: hyperv <on|off|status>"
echo.
)
bcdeditコマンドは管理者権限が必要のようなので、コマンドプロンプトを管理者として実行し、そこから上のバッチを実行する必要があります。
実行例(on=>offへの変更時)
C:\>hyperv status
check status
hypervisorlaunchtype Auto
C:\>hyperv off
set Hyper-V: OFF
この操作を正しく終了しました。
please reboot
C:\>hyperv status
check status
hypervisorlaunchtype Off
で、再起動すれば設定変更が反映されます。
CodeReady Containersのセットアップ
ダウンロード
上のサイトから、crc-windows-amd64.zipと、pull secretファイル(pull-secret.txt)をダウンロードしておきます。
zipを展開して、展開したディレクトリに移動して、crc versionコマンドを実行してみます。
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>crc version
crc version: 1.10.0+9025021
OpenShift version: 4.4.3 (embedded in binary)
セットアップコマンド実行
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>crc setup
INFO Checking if oc binary is cached
INFO Caching oc binary
INFO Checking if podman remote binary is cached
INFO Checking if CRC bundle is cached in '$HOME/.crc'
INFO Unpacking bundle from the CRC binary
INFO Checking if running as normal user
INFO Checking Windows 10 release
INFO Checking if Hyper-V is installed and operational
INFO Checking if user is a member of the Hyper-V Administrators group
INFO Adding user to the Hyper-V Administrators group
INFO Will run as admin: add user to hyperv admins group
INFO Checking if Hyper-V service is enabled
INFO Checking if the Hyper-V virtual switch exist
ERRO Unable to perform Hyper-V administrative commands. Please make sure to re-login or reboot your system
FATA Unable to perform Hyper-V administrative commands. Please make sure to re-login or reboot your system
エラーになります。ユーザーのホームディレクトリ下に.crcというディレクトリが作成されていて、crc.logが吐かれているので確認します。
user_home.crc\crc.log
...
time="2020-05-27T08:47:21+09:00" level=info msg="Checking if Hyper-V is installed and operational"
time="2020-05-27T08:47:22+09:00" level=info msg="Checking if user is a member of the Hyper-V Administrators group"
time="2020-05-27T08:47:22+09:00" level=debug msg="User is not a member of the Hyper-V administrators group"
time="2020-05-27T08:47:22+09:00" level=info msg="Adding user to the Hyper-V Administrators group"
time="2020-05-27T08:47:23+09:00" level=info msg="Will run as admin: add user to hyperv admins group"
time="2020-05-27T08:47:25+09:00" level=info msg="Checking if Hyper-V service is enabled"
time="2020-05-27T08:47:25+09:00" level=info msg="Checking if the Hyper-V virtual switch exist"
time="2020-05-27T08:47:28+09:00" level=debug msg="Virtual Switch not found"
time="2020-05-27T08:47:28+09:00" level=error msg="Unable to perform Hyper-V administrative commands. Please make sure to re-login or reboot your system"
time="2020-05-27T08:47:28+09:00" level=fatal msg="Unable to perform Hyper-V administrative commands. Please make sure to re-login or reboot your system"
...
Hyper-V Administratorsグループにユーザーが属していなかったので追加してくれたらしい。親切！。有効化するためにはログインし直しもしくは再起動が必要そうなので、Windows再起動してからリトライします。
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>crc setup
INFO Checking if oc binary is cached
INFO Checking if podman remote binary is cached
INFO Checking if CRC bundle is cached in '$HOME/.crc'
INFO Checking if running as normal user
INFO Checking Windows 10 release
INFO Checking if Hyper-V is installed and operational
INFO Checking if user is a member of the Hyper-V Administrators group
INFO Checking if Hyper-V service is enabled
INFO Checking if the Hyper-V virtual switch exist
INFO Found Virtual Switch to use: Default Switch
Setup is complete, you can now run 'crc start' to start the OpenShift cluster
今度は成功しました！
結論: 恐らく、実行するユーザーを事前にHyper-V Administratorsグループに登録して再起動してから(もしくはログインし直してから?)セットアップコマンドを実行すればよかったのだと思います。
参考: Hyper-V Administratorsグループのプロパティ確認
おまけ(上のPD時のメモ)
以下の記事で似たような事象が紹介されていますが、ちょっと状況は異なるようです。
参考: 【OpenShift 4】CodeReady Containers 1.2.0をWindowsで動かそうとしたら大変だった話
今回インストールしようとしているのはV1.10なのでそれで改善されているかと思いきや、以下のようなIssueも上がっているのに解決されていないままCloseされているようです。
https://github.com/code-ready/crc/issues/876
V1.10のソース見ても「fixHyperVVirtualSwitch」という関数自体消えていています。
Hyper-Vマネージャーを起動し、操作-仮想スイッチマネージャーを選択します。
これを見ると、Default Switchというのが提供されていて、これを使うように構成されたということのようです。
V1.10のコードを少し追ってみると...
仮想スイッチの有無判定はこの辺
https://github.com/code-ready/crc/blob/v1.10.0/pkg/crc/preflight/preflight_checks_windows.go#L150-L161
hyperv.AlternativeNetworkの値はこちら
https://github.com/code-ready/crc/blob/v1.10.0/pkg/crc/machine/hyperv/constants_windows.go#L5
SelectSwitchByNameOrDefault関数
https://github.com/code-ready/crc/blob/v1.10.0/pkg/os/windows/network/vswitch_windows.go#L12-L20
GetDefaultSwitchName関数
https://github.com/code-ready/crc/blob/v1.10.0/pkg/os/windows/network/vswitch_windows.go#L37-L47
つまり、crcという名前の仮想スイッチがあればそれを使う、無ければデフォルトの仮想スイッチ名を取得してそれを使う、どちらも無ければエラー、ということのようです。
CodeReady Containers操作
起動
以下のコマンドで起動します。1回目の起動は13分くらいかかりました。
最初の起動ログ
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>crc start -p ..\pull-secret.txt
INFO Checking if oc binary is cached
INFO Checking if podman remote binary is cached
INFO Checking if running as normal user
INFO Checking Windows 10 release
INFO Checking if Hyper-V is installed and operational
INFO Checking if user is a member of the Hyper-V Administrators group
INFO Checking if Hyper-V service is enabled
INFO Checking if the Hyper-V virtual switch exist
INFO Found Virtual Switch to use: Default Switch
INFO Extracting bundle: crc_hyperv_4.4.3.crcbundle ...
INFO Checking size of the disk image C:\Users\TomohiroTaguchi\.crc\cache\crc_hyperv_4.4.3\crc.vhdx ...
INFO Creating CodeReady Containers VM for OpenShift 4.4.3...
INFO CodeReady Containers VM is running
INFO Verifying validity of the cluster certificates ...
INFO Will run as admin: add dns server address to interface vEthernet (Default Switch)
INFO Check internal and public DNS query ...
WARN Failed public DNS query from the cluster: ssh command error:
command : host -R 3 quay.io
err : Process exited with status 1
output : ;; connection timed out; no servers could be reached
:
INFO Check DNS query from host ...
INFO Generating new SSH key
INFO Copying kubeconfig file to instance dir ...
INFO Starting OpenShift kubelet service
INFO Configuring cluster for first start
INFO Adding user's pull secret ...
INFO Updating cluster ID ...
INFO Starting OpenShift cluster ... [waiting 3m]
INFO
INFO To access the cluster, first set up your environment by following 'crc oc-env' instructions
INFO Then you can access it by running 'oc login -u developer -p developer https://api.crc.testing:6443'
INFO To login as an admin, run 'oc login -u kubeadmin -p jh3kL-Te6cD-BKDG7-3rvSu https://api.crc.testing:6443'
INFO
INFO You can now run 'crc console' and use these credentials to access the OpenShift web console
Started the OpenShift cluster
WARN The cluster might report a degraded or error state. This is expected since several operators have been disabled to lower the resource usage. For more information, please consult the documentation
2回目は約6分くらい
2回目の起動ログ
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>crc start -p ..\pull-secret.txt
INFO Checking if oc binary is cached
INFO Checking if podman remote binary is cached
INFO Checking if running as normal user
INFO Checking Windows 10 release
INFO Checking if Hyper-V is installed and operational
INFO Checking if user is a member of the Hyper-V Administrators group
INFO Checking if Hyper-V service is enabled
INFO Checking if the Hyper-V virtual switch exist
INFO Found Virtual Switch to use: Default Switch
INFO Starting CodeReady Containers VM for OpenShift 4.4.3...
INFO CodeReady Containers VM is running
INFO Verifying validity of the cluster certificates ...
INFO Will run as admin: add dns server address to interface vEthernet (Default Switch)
INFO Check internal and public DNS query ...
INFO Check DNS query from host ...
INFO Starting OpenShift kubelet service
INFO Starting OpenShift cluster ... [waiting 3m]
INFO
INFO To access the cluster, first set up your environment by following 'crc oc-env' instructions
INFO Then you can access it by running 'oc login -u developer -p developer https://api.crc.testing:6443'
INFO To login as an admin, run 'oc login -u kubeadmin -p jh3kL-Te6cD-BKDG7-3rvSu https://api.crc.testing:6443'
INFO
INFO You can now run 'crc console' and use these credentials to access the OpenShift web console
Started the OpenShift cluster
WARN The cluster might report a degraded or error state. This is expected since several operators have been disabled to lower the resource usage. For more information, please consult the documentation
WARNING出てるけどよく分かりません。
一応動いてるっぽい。
ステータス確認。
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>crc status
CRC VM: Running
OpenShift: Running (v4.4.3)
Disk Usage: 11.82GB of 32.72GB (Inside the CRC VM)
Cache Usage: 14.27GB
Cache Directory: C:\Users\TomohiroTaguchi\.crc\cache
Hyper-Vマネージャー見てみると、crcという仮想サーバーが作成されていました。
4vCPU, Memory 9GB が割り当てられているようです。
OpenShift Webコンソール
crc consoleコマンドを実行すると、ブラウザでOpenShift Web consoleが開きます。
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>crc console
Opening the OpenShift Web Console in the default browser...
cuc startのログに、kubeadminのパスワードがあるのでそれで
画面開くんだけど、値がNot Availableだらけなんですけど???
例えばNodeの画面見ようとすると、タイミングによっては表示されますが、以下のようなエラーが出たりします。
Error Loading Nodes: Call to /api/kubernetes/api/v1/nodes?limit=250 timed out after 20000ms.
どうもタイムアウトが多発してうまく画面上に情報が表示できていないような感じです。
ocコマンド
oc-envで必要な環境設定のコマンドを確認します。
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>crc oc-env
SET PATH=C:\Users\TomohiroTaguchi\.crc\bin;%PATH%
REM Run this command to configure your shell:
REM @FOR /f "tokens=*" %i IN ('crc oc-env') DO @call %i
oc-envで指定されたコマンドをそのまま実行します。
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>@FOR /f "tokens=*" %i IN ('crc oc-env') DO @call %i
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>echo %PATH%
C:\Users\TomohiroTaguchi\.crc\bin;...
developerでログイン
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>oc login -u developer -p developer https://api.crc.testing:6443
The server uses a certificate signed by an unknown authority.
You can bypass the certificate check, but any data you send to the server could be intercepted by others.
Use insecure connections? (y/n): y
Login successful.
You don't have any projects. You can try to create a new project, by running
oc new-project <projectname>
Welcome! See 'oc help' to get started.
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>oc get co
Error from server (Forbidden): clusteroperators.config.openshift.io is forbidden: User "developer" cannot list resource "clusteroperators" in API group "config.openshift.io" at the cluster scope
kubeadminでログインし直し
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>oc login -u kubeadmin -p jh3kL-Te6cD-BKDG7-3rvSu https://api.crc.testing:6443
Login successful.
You have access to 57 projects, the list has been suppressed. You can list all projects with 'oc projects'
Using project "default".
いくつかコマンド打ってみます
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>oc get node
NAME STATUS ROLES AGE VERSION
crc-dv9sm-master-0 Ready master,worker 24d v1.17.1
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>oc get all
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 172.30.0.1 <none> 443/TCP 24d
service/openshift ExternalName <none> kubernetes.default.svc.cluster.local <none> 24d
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>oc get clusteroperators
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE
authentication 4.4.3 True False False 24d
cloud-credential 4.4.3 True False False 24d
cluster-autoscaler 4.4.3 True False False 24d
console 4.4.3 True False False 24d
csi-snapshot-controller 4.4.3 True False False 24d
dns 4.4.3 True False False 71m
etcd 4.4.3 True False False 24d
image-registry 4.4.3 True False False 24d
ingress 4.4.3 True False False 24d
insights 4.4.3 True False False 24d
kube-apiserver 4.4.3 True False False 24d
kube-controller-manager 4.4.3 True False False 24d
kube-scheduler 4.4.3 True False False 24d
kube-storage-version-migrator 4.4.3 True False False 24d
machine-api 4.4.3 True False False 24d
machine-config 4.4.3 True False False 24d
marketplace 4.4.3 True False False 50m
monitoring 4.4.3 True False False 24d
network 4.4.3 True False False 24d
node-tuning 4.4.3 True False False 24d
openshift-apiserver 4.4.3 True False False 49m
openshift-controller-manager 4.4.3 True False False 23d
openshift-samples 4.4.3 True False False 36m
operator-lifecycle-manager 4.4.3 True False False 24d
operator-lifecycle-manager-catalog 4.4.3 True False False 24d
operator-lifecycle-manager-packageserver 4.4.3 True False False 50m
service-ca 4.4.3 True False False 24d
service-catalog-apiserver 4.4.3 True False False 24d
service-catalog-controller-manager 4.4.3 True False False 24d
storage 4.4.3 True False False 24d
ocコマンドで操作が可能なことは確認できました。
停止
停止は1～2分くらい。
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>crc stop
Stopping the OpenShift cluster, this may take a few minutes...
Stopped the OpenShift cluster
メモ
これを稼働させていると、OpenshiftのWeb Consoleに限らずWebのアクセスが全般的に重くなる感じがします。CPUとかメモリがネックになっているのかと思ったのですが、どうやらそうではなくてDNSの名前解決で無駄に時間がかかっている可能性がありそうです。 そのせいでWeb Consoleがタイムアウト多発しているような気がする... ocコマンドは効いているのでOpenshiftとしては動いていると思うのですが。 外部アクセスも出来てるのかよく分からない。 ちょっとこれだとテスト環境として使うのは厳しい。Hyper-VだとPDする気が失せる。Windows版でCRCをうまく使える情報が出そろうのを待つか。 せっかくお手軽にOpenshiftテストできる環境作れると思ったのに...仮想環境カスタマイズ
リソース割り当て変更
デフォルトだとcrc startで作成される仮想マシンは4vCPU, Memory 9GBになっていますが、最初のcrc start実行時のオプションで指定できるようです。
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>crc start --help
Start the OpenShift cluster
Usage:
crc start [flags]
Flags:
-b, --bundle string The system bundle used for deployment of the OpenShift cluster (default "C:\\Users\\TomohiroTaguchi\\.crc\\cache\\crc_hyperv_4.4.3.crcbundle")
-c, --cpus int Number of CPU cores to allocate to the OpenShift cluster (default 4)
--disable-update-check Don't check for update
-h, --help help for start
-m, --memory int MiB of memory to allocate to the OpenShift cluster (default 9216)
-n, --nameserver string IPv4 address of nameserver to use for the OpenShift cluster
-p, --pull-secret-file string File path of image pull secret (download from https://cloud.redhat.com/openshift/install/crc/installer-provisioned)
Global Flags:
-f, --force Forcefully perform an action
--log-level string log level (e.g. "debug | info | warn | error") (default "info")
6vCPU, 24576MB(24GB) を割り当てるようにするには、最初のStart時にこんな感じでオプション指定すればOK
crc start -c 6 -m 24576 -p ..\pull-secret.txt
ネットワーク構成変更 (試行錯誤して結局無かったことにしたメモ)
どうも、クラスター環境から外部のネットワークに接続できていなさそう。Hyper-Vのデフォルトの仮想スイッチは"内部"なので、外のネットワークに接続できないのかなーと思って(=>これが実際のところよく分からない)、外部接続用の仮想スイッチを作成してそちらを使用するようにしてみようと思ったのですが、うまくいきませんでした。解決策が見えない状態でネットワーク構成がよく分からない状態になったので一旦破棄して元にもどしました。
一応作業ログは残しておきます。
Hyper-Vはやっぱりやっかいだ...
ネットワーク構成の手順は、以下の記事を参考にしています。
Installing CodeReady Containers (OpenShift4) on Hyper-V (Win 10)
loopback adapterの追加
Windowsのデバイスマネージャーを開き、操作-レガシハードウェアの追加を選択
次へ
「一覧から選択したハードウェアをインストールする」を選択して次へ
「ネットワークアダプター」を選択して次へ
「Microsoft」-「Microsoft KM-TEst Loopback Adapter」を選択して次へ
次へ
完了
ネットワーク接続の画面を開いて作成された設定を確認します。ここでは「イーサネット3」という名前になっていたので分かりやすさのためLoopbackという名前に変えておきます。
外部仮想スイッチの追加
Hyper-Vの仮想スイッチマネージャーにて、"外部"の仮想スイッチを作成します。
名前をcrcにして作成します。使用するNICには上で作成したLoopback Adapterを指定し、「管理オペレーティングシステムにこのネットワークアダプターの共有を許可する」にチェックします。
はい
上の設定をした後、Loopbackのプロパティを見るとHyper-V拡張可能仮想スイッチにチェックが入っています。
外部接続用ネットワーク構成の共有設定
ネットワーク接続の画面にて、外部ネットワーク接続用のNIC(ここではWifi用)のプロパティを開き、共有設定で以下のようにvEthernet(crc)を指定します。
セットアップやり直し
これで一通り外部ネットワーク接続用の下準備はOKです。
crc delete後、crc setupをやり直します。
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>crc setup
INFO Checking if oc binary is cached
INFO Checking if podman remote binary is cached
INFO Checking if CRC bundle is cached in '$HOME/.crc'
INFO Checking if running as normal user
INFO Checking Windows 10 release
INFO Checking if Hyper-V is installed and operational
INFO Checking if user is a member of the Hyper-V Administrators group
INFO Checking if Hyper-V service is enabled
INFO Checking if the Hyper-V virtual switch exist
INFO Found Virtual Switch to use: crc
Setup is complete, you can now run 'crc start' to start the OpenShift cluster
INFO Found Virtual Switch to use: crc このメッセージで"crc"という仮想スイッチが認識されたことが確認できます。
start
c:\y\Inst_Image\OpenShift\crc-windows-1.10.0-amd64>crc start -p ..\pull-secret.txt --log-level debug
DEBU No new version available. The latest version is 1.10.0
INFO Checking if oc binary is cached
DEBU oc binary already cached
INFO Checking if podman remote binary is cached
DEBU Currently podman remote is not supported
INFO Checking if running as normal user
INFO Checking Windows 10 release
INFO Checking if Hyper-V is installed and operational
INFO Checking if user is a member of the Hyper-V Administrators group
INFO Checking if Hyper-V service is enabled
INFO Checking if the Hyper-V virtual switch exist
INFO Found Virtual Switch to use: crc
Checking file: C:\Users\TomohiroTaguchi\.crc\machines\crc\.crc-exist
INFO Loading bundle: crc_hyperv_4.4.3.crcbundle ...
INFO Checking size of the disk image C:\Users\TomohiroTaguchi\.crc\cache\crc_hyperv_4.4.3\crc.vhdx ...
INFO Creating CodeReady Containers VM for OpenShift 4.4.3...
Found binary path at crc.exe
Launching plugin server for driver hyperv
Plugin server listening at address 127.0.0.1:54251
() Calling .GetVersion
Using API Version 1
() Calling .SetConfigRaw
() Calling .GetMachineName
(crc) Calling .GetMachineName
(crc) Calling .DriverName
Running pre-create checks...
(crc) Calling .PreCreateCheck
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive @(Get-Module -ListAvailable hyper-v).Name | Get-Unique
(crc) DBG | [stdout =====>] : Hyper-V
(crc) DBG |
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive @([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole(([System.Security.Principal.SecurityIdentifier]::new("S-1-5-32-578")))
(crc) DBG | [stdout =====>] : True
(crc) DBG |
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive [Console]::OutputEncoding = [Text.Encoding]::UTF8; (Hyper-V\Get-VMSwitch).Name
(crc) DBG | [stdout =====>] : Default Switch (crc) DBG | crc (crc) DBG | (crc) DBG | [stderr =====>] : (crc) Calling .GetConfigRaw Creating machine...
(crc) Calling .Create
(crc) Downloading C:\Users\TomohiroTaguchi\.crc\cache\crc.vhdx from file://C:/Users/TomohiroTaguchi/.crc/cache/crc_hyperv_4.4.3/crc.vhdx...
(crc) Creating VM...
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive [Console]::OutputEncoding = [Text.Encoding]::UTF8; (Hyper-V\Get-VMSwitch).Name
(crc) DBG | [stdout =====>] : Default Switch
(crc) Using switch "crc"
(crc) DBG | crc
(crc) DBG |
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive Hyper-V\New-VM crc -Path 'C:\Users\TomohiroTaguchi\.crc\machines\crc' -SwitchName 'crc' -MemoryStartupBytes 24576MB
(crc) DBG | [stdout =====>] :
(crc) DBG | Name State CPUUsage(%) MemoryAssigned(M) Uptime Status Version
(crc) DBG | ---- ----- ----------- ----------------- ------ ------ -------
(crc) DBG | crc Off 0 0 00:00:00 Operating normally 9.0
(crc) DBG |
(crc) DBG |
(crc) DBG |
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive Hyper-V\Set-VMMemory -VMName crc -DynamicMemoryEnabled $false
(crc) DBG | [stdout =====>] :
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive Hyper-V\Set-VMProcessor crc -Count 6
(crc) DBG | [stdout =====>] :
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive Hyper-V\Add-VMHardDiskDrive -VMName crc -Path 'C:\Users\TomohiroTaguchi\.crc\machines\crc\crc.vhdx'
(crc) DBG | [stdout =====>] :
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive Hyper-V\Start-VM crc
(crc) Starting VM...
(crc) Waiting for host to start...
(crc) DBG | [stdout =====>] :
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive ( Hyper-V\Get-VM crc ).state
(crc) DBG | [stdout =====>] : Running
(crc) DBG |
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive (( Hyper-V\Get-VM crc ).networkadapters[0]).ipaddresses[0]
(crc) DBG | [stdout =====>] :
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive ( Hyper-V\Get-VM crc ).state
(crc) DBG | [stdout =====>] : Running
(crc) DBG |
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive (( Hyper-V\Get-VM crc ).networkadapters[0]).ipaddresses[0]
(crc) DBG | [stdout =====>] :
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive ( Hyper-V\Get-VM crc ).state
(crc) DBG | [stdout =====>] : Running
(crc) DBG |
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive (( Hyper-V\Get-VM crc ).networkadapters[0]).ipaddresses[0]
(crc) DBG | [stdout =====>] :
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive ( Hyper-V\Get-VM crc ).state
(crc) DBG | [stdout =====>] : Running
(crc) DBG |
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive (( Hyper-V\Get-VM crc ).networkadapters[0]).ipaddresses[0]
(crc) DBG | [stdout =====>] :
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive ( Hyper-V\Get-VM crc ).state
(crc) DBG | [stdout =====>] : Running
(crc) DBG |
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive (( Hyper-V\Get-VM crc ).networkadapters[0]).ipaddresses[0]
(crc) DBG | [stdout =====>] :
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive ( Hyper-V\Get-VM crc ).state
(crc) DBG | [stdout =====>] : Running
(crc) DBG |
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive (( Hyper-V\Get-VM crc ).networkadapters[0]).ipaddresses[0]
(crc) DBG | [stdout =====>] :
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive ( Hyper-V\Get-VM crc ).state
(crc) DBG | [stdout =====>] : Running
(crc) DBG |
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive (( Hyper-V\Get-VM crc ).networkadapters[0]).ipaddresses[0]
(crc) DBG | [stdout =====>] :
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive ( Hyper-V\Get-VM crc ).state
(crc) DBG | [stdout =====>] : Running
(crc) DBG |
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive (( Hyper-V\Get-VM crc ).networkadapters[0]).ipaddresses[0]
(crc) DBG | [stdout =====>] : fe80::f707:de6c:504c:a195
(crc) DBG |
(crc) DBG | [stderr =====>] :
(crc) Calling .GetConfigRaw
(crc) Calling .DriverName
(crc) Calling .DriverName
Waiting for machine to be running, this may take a few minutes...
(crc) Calling .GetState
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive ( Hyper-V\Get-VM crc ).state
(crc) DBG | [stdout =====>] : Running
(crc) DBG |
(crc) DBG | [stderr =====>] :
Machine is up and running!
Machine successfully created
Created C:\Users\TomohiroTaguchi\.crc\machines\crc\.crc-exist
(crc) Calling .GetState
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive ( Hyper-V\Get-VM crc ).state
(crc) DBG | [stdout =====>] : Running
(crc) DBG |
(crc) DBG | [stderr =====>] :
Found binary path at crc.exe
Launching plugin server for driver hyperv
Plugin server listening at address 127.0.0.1:54254
() Calling .GetVersion
Using API Version 1
() Calling .SetConfigRaw
() Calling .GetMachineName
DEBU Waiting until ssh is available
(crc) Calling .GetSSHHostname
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive ( Hyper-V\Get-VM crc ).state
(crc) DBG | [stdout =====>] : Running
(crc) DBG |
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive (( Hyper-V\Get-VM crc ).networkadapters[0]).ipaddresses[0]
(crc) DBG | [stdout =====>] : fe80::f707:de6c:504c:a195
(crc) DBG |
(crc) DBG | [stderr =====>] :
(crc) Calling .GetSSHPort
(crc) Calling .GetSSHKeyPath
(crc) Calling .GetSSHKeyPath
(crc) Calling .GetSSHUsername
Using SSH client type: native
&{{{<nil> 0 [] [] []} core [0x795e20] 0x795df0 <nil> [] 0s} fe80::f707:de6c:504c:a195 22 <nil> <nil>}
DEBU About to run SSH command:
exit 0
Error dialing TCP: dial tcp [fe80::f707:de6c:504c:a195]:22: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
Error dialing TCP: dial tcp [fe80::f707:de6c:504c:a195]:22: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
Error dialing TCP: dial tcp [fe80::f707:de6c:504c:a195]:22: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
Error dialing TCP: dial tcp [fe80::f707:de6c:504c:a195]:22: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
Error dialing TCP: dial tcp [fe80::f707:de6c:504c:a195]:22: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
Error dialing TCP: dial tcp [fe80::f707:de6c:504c:a195]:22: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
Error dialing TCP: dial tcp [fe80::f707:de6c:504c:a195]:22: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
DEBU SSH cmd err, output: <nil>:
INFO CodeReady Containers VM is running
INFO Verifying validity of the cluster certificates ...
(crc) Calling .GetSSHHostname
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive ( Hyper-V\Get-VM crc ).state
(crc) DBG | [stdout =====>] : Running
(crc) DBG |
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive (( Hyper-V\Get-VM crc ).networkadapters[0]).ipaddresses[0]
(crc) DBG | [stdout =====>] : fe80::f707:de6c:504c:a195
(crc) DBG |
(crc) DBG | [stderr =====>] :
(crc) Calling .GetSSHPort
(crc) Calling .GetSSHKeyPath
(crc) Calling .GetSSHKeyPath
(crc) Calling .GetSSHUsername
Using SSH client type: native
&{{{<nil> 0 [] [] []} core [0x795e20] 0x795df0 <nil> [] 0s} fe80::f707:de6c:504c:a195 22 <nil> <nil>}
DEBU About to run SSH command:
date --date="$(sudo openssl x509 -in /var/lib/kubelet/pki/kubelet-client-current.pem -noout -enddate | cut -d= -f 2)" --iso-8601=seconds
DEBU SSH cmd err, output: <nil>: 2020-06-01T05:40:49+00:00
(crc) Calling .GetIP
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive ( Hyper-V\Get-VM crc ).state
(crc) DBG | [stdout =====>] : Running
(crc) DBG |
(crc) DBG | [stderr =====>] :
(crc) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive (( Hyper-V\Get-VM crc ).networkadapters[0]).ipaddresses[0]
(crc) DBG | [stdout =====>] : fe80::f707:de6c:504c:a195
(crc) DBG |
(crc) DBG | [stderr =====>] :
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 1
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 2
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 3
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 4
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 5
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 6
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 7
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 8
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 9
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 10
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 11
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 12
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 13
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 14
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 15
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 16
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 17
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 18
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 19
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 20
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 21
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 22
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 23
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 24
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 25
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 26
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 27
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 28
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
DEBU retry loop 29
ERRO unknown error occurred
DEBU Error finding host IP (unknown error occurred) - retrying
DEBU error: Temporary Error: unknown error occurred - sleeping 2s
ERRO Error determining host IP: Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Temporary Error: unknown error occurred
Making call to close driver server
(crc) Calling .Close
Successfully made call to close driver server
Making call to close connection to plugin binary
Making call to close driver server
(crc) Calling .Close
(crc) DBG | Closing plugin on server side
Successfully made call to close driver server
(crc) DBG | Closing plugin on server side
Making call to close connection to plugin binary
どうやら起動はしているのだが、ホストOSからゲストOSへの通信がうまくいっていないような気がする。