Kubernetes ナビ
本シリーズでは kubernetes クラスタを構成する様々なコンポーネント・技術をざっとご紹介しております。
- 【2021年４月版】Kubernetes ナビ その１: ディストリいろいろ
- 【2021年４月版】Kubernetes ナビ その２: ネットワーク・サービス関連
- 【2021年４月版】Kubernetes ナビ その３: ストレージ関連
- 【2021年４月版】Kubernetes ナビ その４: コンテナ関連
- 【2021年４月版】Kubernetes ナビ その５: Serverless
今回はその３となります。
Kubernetes のストレージ関連のコンポーネントについてざっくり
k8s のストレージ関連コンポーネントは以下の２つの分類に分かれます。というか勝手に"ストレージ関連"にまとめてしまいました。すいません。
- コントロールプレーンのストレージ … クラスタ全体の設定値など保持するストレージ
- ストレージプロバイダ・CSI プロバイダ … コンテナにマウントされるボリュームを提供するストレージ
以下、ストレージ関連のコンポーネントについてご紹介いたします。
コントロールプレーンのストレージ
K8s のクラスタそのものを管理するコンポーネントである "コントロールプレーン(Control Plane)" のストレージは K8s クラスタの根幹に関わる部分です。"コンポーネント"とご紹介していますがディストリによって依存があるのでこのコンポーネントは基本的に交換できません。ディストリによっては設定で切り替えられますが、前もって対応しているストレージは決まっています。
Etcd
コンセンサスアルゴリズムにRaftを採用している分散キーバリューストア(KVS)。CNCF の傘下プロジェクトでもあります。
Dqlite
コンセンサスアルゴリズムにRaftを採用している分散組み込みSQLエンジン。SQLite の分散型だそうです。
Kine
Etcd のAPIをラッピングし、dqlite, sqlite, Postgres, MySQL をバックエンドに出来る。k8s 用なのですべての etcd APIはサポートされていない模様。
KineはEtcdラッパなので、実質 Etcd と Dqlite の２択の模様です。
CSI プロバイダ
CSIに対応した永続化プラグインは各社実装されているようですので、以下にリストがございました。
かなり量がありますが・・・一覧を抜粋いたします。
| 名称 | CSI互換バージョン | 概要 | Podを超える永続性 | 提供されるアクセスモード | その他の機能 |
|---|---|---|---|---|---|
| Alicloud Disk | v1.0 | A Container Storage Interface (CSI) Driver for Alicloud Disk | Persistent | Read/Write Single Pod | Raw Block, Snapshot |
| Alicloud NAS | v1.0 | A Container Storage Interface (CSI) Driver for Alicloud Network Attached Storage (NAS) | Persistent | Read/Write Multiple Pods | |
| Alicloud OSS | v1.0 | A Container Storage Interface (CSI) Driver for Alicloud Object Storage Service (OSS) | Persistent | Read/Write Multiple Pods | |
| ArStor CSI | v1.0 | A Container Storage Interface (CSI) Driver for Huayun Storage Service (ArStor) | Persistent and Ephemeral | Read/Write Single Pod | Raw Block, Snapshot, Expansion, Cloning |
| AWS Elastic Block Storage | v0.3, v1.0 | A Container Storage Interface (CSI) Driver for AWS Elastic Block Storage (EBS) | Persistent | Read/Write Single Pod | Raw Block, Snapshot, Expansion |
| AWS Elastic File System | v0.3, v1.0 | A Container Storage Interface (CSI) Driver for AWS Elastic File System (EFS) | Persistent | Read/Write Multiple Pods | |
| AWS FSx for Lustre | v0.3, v1.0 | A Container Storage Interface (CSI) Driver for AWS FSx for Lustre (EBS) | Persistent | Read/Write Multiple Pods | |
| Azure disk | v0.3, v1.0 | A Container Storage Interface (CSI) Driver for Azure disk | Persistent | Read/Write Single Pod | |
| Azure file | v0.3, v1.0 | A Container Storage Interface (CSI) Driver for Azure file | Persistent | Read/Write Multiple Pods | |
| BeeGFS | v1.3 | A Container Storage Interface (CSI) Driver for the BeeGFS Parallel File System | Persistent | Read/Write Multiple Pods | |
| Bigtera VirtualStor (block) | v0.3, v1.0.0, v1.1.0 | A Container Storage Interface (CSI) Driver for Bigtera VirtualStor block storage | Persistent | Read/Write Single Pod | Raw Block, Snapshot, Expansion |
| Bigtera VirtualStor (filesystem) | v0.3, v1.0.0, v1.1.0 | A Container Storage Interface (CSI) Driver for Bigtera VirtualStor filesystem | Persistent | Read/Write Multiple Pods | Expansion |
| CephFS | v0.3, v1.0.0, v1.1.0, v1.2.0 | A Container Storage Interface (CSI) Driver for CephFS | Persistent | Read/Write Multiple Pods | Expansion, Snapshot, Clone |
| Ceph RBD | v0.3, v1.0.0, v1.1.0, v1.2.0 | A Container Storage Interface (CSI) Driver for Ceph RBD | Persistent | Read/Write Single Pod | Raw Block, Snapshot, Expansion, Topology, Cloning |
| ChubaoFS | v1.0.0 | A Container Storage Interface (CSI) Driver for ChubaoFS Storage | Persistent | Read/Write Multiple Pods | |
| Cinder | v0.3, v1.0, v1.1 | A Container Storage Interface (CSI) Driver for OpenStack Cinder | Persistent and Ephemeral | Depends on the storage backend used | Raw Block, Snapshot, Expansion |
| cloudscale.ch | v1.0 | A Container Storage Interface (CSI) Driver for the cloudscale.ch IaaS platform | Persistent | Read/Write Single Pod | Snapshot |
| Datatom-InfinityCSI | v0.3, v1.0.0, v1.1.0 | A Container Storage Interface (CSI) Driver for DATATOM Infinity storage | Persistent | Read/Write Single Pod | Raw Block, Snapshot, Expansion, Topology |
| Datatom-InfinityCSI (filesystem) | v0.3, v1.0.0, v1.1.0 | A Container Storage Interface (CSI) Driver for DATATOM Infinity filesystem storage | Persistent | Read/Write Multiple Pods | Expansion |
| Datera | v1.0 | A Container Storage Interface (CSI) Driver for Datera Data Services Platform (DSP) | Persistent | Read/Write Single Pod | Snapshot |
| DDN EXAScaler | v1.0, v1.1 | A Container Storage Interface (CSI) Driver for DDN EXAScaler filesystems | Persistent | Read/Write Multiple Pods | Expansion |
| Dell EMC PowerMax | v1.1 | A Container Storage Interface (CSI) Driver for Dell EMC PowerMax | Persistent | Read/Write Single Pod | Raw Block, Snapshot, Expansion, Cloning, Topology |
| Dell EMC PowerScale | v1.1 | A Container Storage Interface (CSI) Driver for Dell EMC PowerScale | Persistent and Ephemeral | Read/Write Multiple Pods | Snapshot, Expansion, Cloning, Topology |
| Dell EMC PowerStore | v1.1 | A Container Storage Interface (CSI) Driver for Dell EMC PowerStore | Persistent and Ephemeral | Read/Write Single Pod | Raw Block, Snapshot, Expansion, Cloning, Topology |
| Dell EMC Unity | v1.1 | A Container Storage Interface (CSI) Driver for Dell EMC Unity | Persistent and Ephemeral | Read/Write Single Pod | Raw Block, Snapshot, Expansion, Cloning, Topology |
| Dell EMC VxFlexOS | v1.1 | A Container Storage Interface (CSI) Driver for Dell EMC VxFlexOS | Persistent | Read/Write Single Pod | Raw Block, Snapshot, Expansion, Cloning, Topology |
| democratic-csi | v1.0,v1.1,v1.2 | Generic CSI plugin supporting zfs based solutions (FreeNAS / TrueNAS and ZoL solutions such as Ubuntu) | Persistent and Ephemeral | Read/Write Single Pod (Block Volume) Read/Write Multiple Pods (File Volume) | Raw Block, Snapshot, Expansion, Cloning |
| Diamanti-CSI | v1.0 | A Container Storage Interface (CSI) Driver for Diamanti DCX Platform | Persistent | Read/Write Single Pod | Raw Block, Snapshot, Expansion |
| DigitalOcean Block Storage | v0.3, v1.0 | A Container Storage Interface (CSI) Driver for DigitalOcean Block Storage | Persistent | Read/Write Single Pod | Raw Block, Snapshot, Expansion |
| Dothill-CSI | v1.3 | Generic CSI plugin supporting Seagate AssuredSan appliances such as HPE MSA, Dell EMC PowerVault ME4 and others ... | Persistent | Read/Write Single Node Expansion | |
| DriveScale | v1.0 | A Container Storage Interface (CSI) Driver for DriveScale software composable infrastructure solution | Persistent | Read/Write Single Pod | |
| Ember CSI | v0.2, v0.3, v1.0 | Multi-vendor CSI plugin supporting over 80 Drivers to provide block and mount storage to Container Orchestration systems. | Persistent | Read/Write Single Pod | Raw Block, Snapshot |
| Excelero NVMesh | v1.0, v1.1 | A Container Storage Interface (CSI) Driver for Excelero NVMesh | Persistent | Read/Write Multiple Pods | Raw Block, Expansion |
| GCE Persistent Disk | v0.3, v1.0 | A Container Storage Interface (CSI) Driver for Google Compute Engine Persistent Disk (GCE PD) | Persistent | Read/Write Single Pod | Raw Block, Snapshot, Expansion, Topology |
| Google Cloud Filestore | v0.3 | A Container Storage Interface (CSI) Driver for Google Cloud Filestore | Persistent | Read/Write Multiple Pods | |
| Google Cloud Storage | v1.0 | A Container Storage Interface (CSI) Driver for Google Cloud Storage | Persistent and Ephemeral | Read/Write Multiple Pods | Expansion |
| GlusterFS | v0.3, v1.0 | A Container Storage Interface (CSI) Driver for GlusterFS | Persistent | Read/Write Multiple Pods | Snapshot |
| Gluster VirtBlock | v0.3, v1.0 | A Container Storage Interface (CSI) Driver for Gluster Virtual Block volumes | Persistent | Read/Write Single Pod | |
| Hammerspace CSI | v0.3, v1.0 | A Container Storage Interface (CSI) Driver for Hammerspace Storage | Persistent | Read/Write Multiple Pods | Raw Block, Snapshot |
| Hedvig | v1.0 | A Container Storage Interface (CSI) Driver for Hedvig | Persistent | Read/Write Multiple Pods | Raw Block, Snapshot, Expansion |
| Hetzner Cloud Volumes CSI | v0.3, v1.0 | A Container Storage Interface (CSI) Driver for Hetzner Cloud Volumes | Persistent | Read/Write Single Pod | Raw Block, Expansion |
| Hitachi Vantara | v1.2 | A Container Storage Interface (CSI) Driver for VSP series Storage | Persistent | Read/Write Single Pod | Raw Block, Snapshot, Expansion, Cloning |
| HPE | v1.0, v1.1, v1.2 | A multi-platform Container Storage Interface (CSI) driver. Supports HPE Nimble Storage, HPE Primera and HPE 3PAR | Persistent and Ephemeral | Read/Write Multiple Pods | Raw Block, Snapshot, Expansion, Cloning |
| Huawei Storage CSI | v1.0 | A Container Storage Interface (CSI) Driver for FusionStorage, OceanStor 100D, OceanStor Dorado V3, OceanStor Dorado V6, OceanStor V3, OceanStor V5 | Persistent | Read/Write Multiple Pod | Snapshot, Expansion, Cloning |
| HyperV CSI | v1.0, v1.1 | A Container Storage Interface (CSI) driver to manage hyperv hosts | Persistent | Read/Write Multiple Pods | |
| IBM Block Storage | v1.0, v1.1, v1.2 | A Container Storage Interface (CSI) Driver for IBM Spectrum Virtualize Family, IBM FlashSystem A9000 and A9000R, IBM DS8880 and DS8900. | Persistent | Read/Write Single Pod | Raw Block, Snapshot, Expansion, Cloning |
| IBM Spectrum Scale | v1.0, v1.1 | A Container Storage Interface (CSI) Driver for the IBM Spectrum Scale File System | Persistent | Read/Write Multiple Pod | |
| IBM Cloud Block Storage VPC CSI Driver | v1.0 | A Container Storage Interface (CSI) Driver for IBM Cloud Kubernetes Service and Red Hat OpenShift on IBM Cloud | Persistent | Read/Write Single Pod | Raw Block |
| Infinidat | v1.0, v1.1 | A Container Storage Interface (CSI) Driver for Infinidat InfiniBox | Persistent | Read/Write Multiple Pods | Raw Block, Snapshot, Expansion, Cloning |
| Inspur InStorage CSI | v1.0 | A Container Storage Interface (CSI) Driver for inspur AS/HF/CS/CF Series Primary Storage, inspur AS13000 Series SDS Storage | Persistent | Read/Write Single Pod | Raw Block, Snapshot, Expansion, Cloning |
| Intel PMEM-CSI | v1.0 | A Container Storage Interface (CSI) driver for PMEM from Intel | Persistent and Ephemeral | Read/Write Single Pod | Raw Block |
| Intelliflash File Storage | v1.0, v1.1, v1.2 | A Container Storage Interface (CSI) Driver for Intelliflash File Storage | Persistent | Read/Write Multiple Pods | Snapshot, Expansion, Cloning, Topology |
| JuiceFS | v0.3, v1.0 | A Container Storage Interface (CSI) Driver for JuiceFS File System | Persistent | Read/Write Multiple Pod | |
| kaDalu | v0.3 | A CSI Driver (and operator) for GlusterFS | Persistent | Read/Write Multiple Pods | |
| KumoScale Block Storage | v1.0 | A Container Storage Interface (CSI) Driver for KumoScale Block Storage | Persistent | Read/Write Single Pod | Raw Block, Snapshot, Expansion, Topology |
| Linode Block Storage | v1.0 | A Container Storage Interface (CSI) Driver for Linode Block Storage | Persistent | Read/Write Single Pod | |
| LINSTOR | v1.2 | A Container Storage Interface (CSI) Driver for LINSTOR volumes | Persistent | Read/Write Single Pod | Raw Block, Snapshot, Expansion, Cloning, Topology |
| Longhorn | v1.1 | A Container Storage Interface (CSI) Driver for Longhorn volumes | Persistent | Read/Write Single Node | Raw Block |
| MacroSAN | v1.0 | A Container Storage Interface (CSI) Driver for MacroSAN Block Storage | Persistent | Read/Write Single Pod | |
| Manila | v1.1, v1.2 | A Container Storage Interface (CSI) Driver for OpenStack Shared File System Service (Manila) | Persistent | Read/Write Multiple Pods | Snapshot, Topology |
| MapR | v1.0 | A Container Storage Interface (CSI) Driver for MapR Data Platform | Persistent | Read/Write Multiple Pods | Snapshot |
| MooseFS | v1.0 | A Container Storage Interface (CSI) Driver for MooseFS clusters. | Persistent | Read/Write Multiple Pods | |
| NetApp | v1.0, v1.1, v1.2 | A Container Storage Interface (CSI) Driver for NetApp's Trident container storage orchestrator | Persistent | Read/Write Multiple Pods | Raw Block, Snapshot, Expansion, Cloning, Topology |
| NexentaStor File Storage | v1.0, v1.1, v1.2 | A Container Storage Interface (CSI) Driver for NexentaStor File Storage | Persistent | Read/Write Multiple Pods | Snapshot, Expansion, Cloning, Topology |
| NexentaStor Block Storage | v1.0, v1.1, v1.2 | A Container Storage Interface (CSI) Driver for NexentaStor over iSCSI protocol | Persistent | Read/Write Multiple Pods | Snapshot, Expansion, Cloning, Topology, Raw block |
| Nutanix | v0.3, v1.0, v1.2 | A Container Storage Interface (CSI) Driver for Nutanix | Persistent | "Read/Write Single Pod" with Nutanix Volumes and "Read/Write Multiple Pods" with Nutanix Files | Raw Block, Snapshot, Expansion, Cloning |
| OpenEBS | v1.0 | A Container Storage Interface (CSI) Driver for OpenEBS | Persistent | Read/Write Single Pod | Expansion, Snapshot, Cloning |
| OpenSDS | v1.0 | A Container Storage Interface (CSI) Driver for OpenSDS | Persistent | Read/Write Single Pod | Raw Block, Snapshot |
| Open-E | v1.0 | A Container Storage Interface (CSI) Driver for Open-E JovianDSS Storage | Persistent | Read/Write Single Pod | Snapshot, Cloning |
| Oracle Cloud Infrastructure(OCI) Block Storage | v1.1 | A Container Storage Interface (CSI) Driver for Oracle Cloud Infrastructure (OCI) Block Storage | Persistent | Read/Write Single Pod | Topology |
| oVirt | v1.0 | A Container Storage Interface (CSI) Driver for oVirt | Persistent | Read/Write Single Pod | Block, File Storage |
| Portworx | v0.3, v1.1 | A Container Storage Interface (CSI) Driver for Portworx | Persistent | Read/Write Multiple Pods | Snapshot, Expansion |
| Pure Storage CSI | v1.0, v1.1, v1.2, v1.3 | A Container Storage Interface (CSI) Driver for Pure Storage's Pure Service Orchestrator | Persistent and Ephemeral | Read/Write Multiple Pods | Snapshot, Cloning, Raw Block, Topology, Expansion |
| QingCloud CSI | v1.1 | A Container Storage Interface (CSI) Driver for QingCloud Block Storage | Persistent | Read/Write Single Pod | Raw Block, Snapshot, Expansion, Cloning |
| QingStor CSI | v0.3, v1.1 | A Container Storage Interface (CSI) Driver for NeonSAN storage system | Persistent | Read/Write Multiple Pods | Raw Block, Snapshot, Expansion, Cloning |
| Quobyte | v0.2 | A Container Storage Interface (CSI) Driver for Quobyte | Persistent | Read/Write Multiple Pods | |
| ROBIN | v0.3, v1.0 | A Container Storage Interface (CSI) Driver for ROBIN | Persistent | Read/Write Multiple Pods | Raw Block, Snapshot, Expansion, Cloning |
| SandStone | v1.0 | A Container Storage Interface (CSI) Driver for SandStone USP | Persistent | Read/Write Multiple Pods | Raw Block, Snapshot, Expansion, Cloning |
| Sangfor-EDS-File-Storage | v1.0 | A Container Storage Interface (CSI) Driver for Sangfor Distributed File Storage(EDS) | Persistent | Read/Write Multiple Pods | |
| Sangfor-EDS-Block-Storage | v1.0 | A Container Storage Interface (CSI) Driver for Sangfor Block Storage(EDS) | Persistent | Read/Write Single Pod | |
| SeaweedFS | v1.0 | A Container Storage Interface (CSI Driver for SeaweedFS) | Persistent | Read/Write Multiple Pods | |
| Secrets Store CSI Driver | v0.0.10 | A Container Storage Interface (CSI) Driver for mounting secrets, keys, and certs stored in enterprise-grade external secrets stores as volumes. | Ephemeral | N/A | |
| SmartX | v1.0 | A Container Storage Interface (CSI) Driver for SmartX ZBS Storage | Persistent | Read/Write Multiple Pods | Snapshot, Expansion |
| SPDK-CSI | v1.1 | A Container Storage Interface (CSI) Driver for SPDK | Persistent and Ephemeral | Read/Write Single Pod | |
| StorageOS | v0.3, v1.0 | A Container Storage Interface (CSI) Driver for StorageOS | Persistent | Read/Write Multiple Pods | |
| Storidge | v0.3, v1.0 | A Container Storage Interface (CSI) Driver for Storidge CIO | Persistent | Read/Write Multiple Pods | Snapshot, Expansion |
| StorPool | v1.0 | A Container Storage Interface (CSI) Driver for StorPool | Persistent and Ephemeral | Read/Write Multiple Pods | Expansion |
| Tencent Cloud Block Storage | v1.0 | A Container Storage Interface (CSI) Driver for Tencent Cloud Block Storage | Persistent | Read/Write Single Pod | Snapshot |
| Tencent Cloud File Storage | v1.0 | A Container Storage Interface (CSI) Driver for Tencent Cloud File Storage | Persistent | Read/Write Multiple Pods | |
| Tencent Cloud Object Storage | v1.0 | A Container Storage Interface (CSI) Driver for Tencent Cloud Object Storage | Persistent | Read/Write Multiple Pods | |
| TopoLVM | v1.1 | A Container Storage Interface (CSI) Driver for LVM | Persistent and Ephemeral | Read/Write Single Pod | Raw Block, Expansion, Topology Aware |
| VAST Data | v1.0 | A Container Storage Interface (CSI) Driver for VAST Data | Persistent | Read/Write Multiple Pods | |
| XSKY-EBS | v1.0 | A Container Storage Interface (CSI) Driver for XSKY Distributed Block Storage (X-EBS) | Persistent | Read/Write Single Pod | Raw Block, Snapshot, Expansion, Cloning |
| XSKY-EUS | v1.0 | A Container Storage Interface (CSI) Driver for XSKY Distributed File Storage (X-EUS) | Persistent | Read/Write Multiple Pods | |
| Vault | v1.0 | A Container Storage Interface (CSI) Driver for mounting HashiCorp Vault secrets as volumes. | Ephemeral | N/A | |
| Veritas InfoScale Volumes | v1.2 | A Container Storage Interface (CSI) Driver for Veritas InfoScale volumes | Persistent | Read/Write Multiple Pods | Snapshot, Expansion, Cloning |
| vSphere | v1.0 | A Container Storage Interface (CSI) Driver for VMware vSphere | Persistent | Read/Write Single Pod (Block Volume), Read/Write Multiple Pods (File Volume) | Raw Block,Expansion (Block Volume), Topology Aware (Block Volume) |
| Vultr Block Storage | v1.2 | A Container Storage Interface (CSI) Driver for Vultr Block Storage | Persistent | Read/Write Single Pod | |
| WekaIO | v1.0 | A Container Storage Interface (CSI) Driver for mounting WekaIO WekaFS filesystem as volumes | Persistent | Read/Write Multiple Pods | |
| Yandex.Cloud | v1.2 | A Container Storage Interface (CSI) plugin for Yandex.Cloud Compute Disks | Persistent | Read/Write Single Pod | |
| YanRongYun | v1.0 | A Container Storage Interface (CSI) Driver for YanRong YRCloudFile Storage | Persistent | Read/Write Multiple Pods | |
| Zadara-CSI | v1.0, v1.1 | A Container Storage Interface (CSI) plugin for Zadara VPSA Storage Array & VPSA All-Flash | Persistent | Read/Write Multiple Pods | Raw Block, Snapshot, Expansion, Cloning |
かなり多いですが、だいたいクラウドサービス用のCSIドライバです。LB/Ingressと同様、"物理層"へのつなぎ込みには物理層依存の箇所が出てきますね。クラウドサービスのk8sを使用する場合には間違いなくCSIストレージが出てきます。そしてそれはクラウドサービスごとに異なりますので個別に調査と検証が必要となります。
ところでBeremetalなK8sで使えるCSIドライバはどれなんだろう・・・？と思って調べたら、Dell の Bare-metal CSI Plugin が見つかりました。名前そのまんまだし、alpha版なので微妙ですが…
調べればいろいろ出てくるかもです。
それでは今回はここまでとします。