Kubernetes ナビ
本シリーズでは kubernetes クラスタを構成する様々なコンポーネント・技術をざっとご紹介しております。
- 【2021年４月版】Kubernetes ナビ その１: ディストリいろいろ
- 【2021年４月版】Kubernetes ナビ その２: ネットワーク・サービス関連
- 【2021年４月版】Kubernetes ナビ その３: ストレージ関連
- 【2021年４月版】Kubernetes ナビ その４: コンテナ関連
- 【2021年４月版】Kubernetes ナビ その５: Serverless
今回はその5となります。
Kubernetes で使える Serverless フレームワークいろいろ
以下で k8s にデプロイできる Serverless フレームワークをご紹介いたします。
1. Knative
元々は Google 社で開発されていたフレームワークをオープンにしたもの。Istio ベースで HTTPサービスもサポートしている。
- 公式サイト: https://knative.dev/
- リポジトリ: https://github.com/knative
サポートされる Event ソース
以下は公式サイトに上げられている代表的な Event コンポーネントです。
| Name | API Version | Maintainer | Description |
|---|---|---|---|
| APIServerSource | v1 | Knative | Brings Kubernetes API server events into Knative. The APIServerSource fires a new event each time a Kubernetes resource is created, updated or deleted. |
| AWS SQS | v1alpha1 | Knative | Brings AWS Simple Queue Service messages into Knative. The AwsSqsSource fires a new event each time an event is published on an AWS SQS topic. |
| Apache Camel | v1alpha1 | Knative | Enables use of Apache Camel components for pushing events into Knative. A CamelSource is an event source that can represent any existing Apache Camel component, that provides a consumer side, and enables publishing events to an addressable endpoint. Each Camel endpoint has the form of a URI where the scheme is the ID of the component to use. CamelSource requires Camel-K to be installed into the current namespace. See the CamelSource example. |
| Apache CouchDB | v1alpha1 | Knative | Brings Apache CouchDB messages into Knative. |
| Apache Kafka | v1beta1 | Knative | Brings Apache Kafka messages into Knative. The KafkaSource reads events from an Apache Kafka Cluster, and passes these events to a sink so that they can be consumed. See the Kafka Source example for more details. |
| Container Source | v1 | Knative | The ContainerSource will instantiate container image(s) that can generate events until the ContainerSource is deleted. This may be used, for example, to poll an FTP server for new files or generate events at a set time interval. Given a spec.template with at least a container image specified, ContainerSource will keep a Pod running with the specified image(s). K_SINK (destination address) and KE_CE_OVERRIDES (JSON CloudEvents attributes) environment variables are injected into the running image(s). It is used by multiple other Sources as underlying infrastructure. Refer to the Container Source example for more details. |
| GitHub | v1alpha1 | Knative | Registers for events of the specified types on the specified GitHub organization or repository, and brings those events into Knative. The GitHubSource fires a new event for selected GitHub event types. See the GitHub Source example for more details. |
| GitLab | v1alpha1 | Knative | Registers for events of the specified types on the specified GitLab repository, and brings those events into Knative. The GitLabSource creates a webhooks for specified event types, listens for incoming events, and passes them to a consumer. See the GitLab Source example for more details. |
| Heartbeats | N/A | Knative | Uses an in-memory timer to produce events at the specified interval. |
| PingSource | v1beta2 | Knative | Produces events with a fixed payload on a specified Cron schedule. See the Ping Source example for more details. |
| RabbitMQ | Active | development | None |
| SinkBinding | v1 | Knative | The SinkBinding can be used to author new event sources using any of the familiar compute abstractions that Kubernetes makes available (e.g. Deployment, Job, DaemonSet, StatefulSet), or Knative abstractions (e.g. Service, Configuration). SinkBinding provides a framework for injecting K_SINK (destination address) and K_CE_OVERRIDES (JSON cloudevents attributes) environment variables into any Kubernetes resource which has a spec.template that looks like a Pod (aka PodSpecable). See the SinkBinding example for more details. |
| WebSocket | N/A | Knative | Opens a WebSocket to the specified source and packages each received message as a Knative event. |
| Auto Container Source | Proof of Concept | None | AutoContainerSource is a controller that allows the Source CRDs without needing a controller. It notices CRDs with a specific label and starts controlling resources of that type. It utilizes Container Source as underlying infrastructure. |
| Amazon CloudWatch | Supported | TriggerMesh | Collects metrics from Amazon CloudWatch. |
| Amazon CloudWatch Logs | Supported | TriggerMesh | Subscribes to log events from an Amazon CloudWatch Logs stream. |
| Amazon CodeCommit | Supported | TriggerMesh | Registers for events emitted by an Amazon CodeCommit source code repository. |
| Amazon Cognito Identity | Supported | TriggerMesh | Registers for events from Amazon Cognito identity pools. |
| Amazon Cognito User | Supported | TriggerMesh | Registers for events from Amazon Cognito user pools. |
| Amazon DynamoDB | Supported | TriggerMesh | Reads records from an Amazon DynamoDB stream. |
| Amazon Kinesis | Supported | TriggerMesh | Reads records from an Amazon Kinesis stream. |
| Amazon SNS | Supported | TriggerMesh | Subscribes to messages from an Amazon SNS topic. |
| Amazon SQS | Supported | TriggerMesh | Consumes messages from an Amazon SQS queue. |
| BitBucket | Proof of Concept | None | Registers for events of the specified types on the specified BitBucket organization/repository. Brings those events into Knative. |
| CloudAuditLogsSource | v1 | Registers for events of the specified types on the specified Google Cloud Audit Logs. Brings those events into Knative. Refer to the CloudAuditLogsSource example for more details. | |
| CloudPubSubSource | v1 | Brings Cloud Pub/Sub messages into Knative. The CloudPubSubSource fires a new event each time a message is published on a Google Cloud Platform PubSub topic. See the CloudPubSubSource example for more details. | |
| CloudSchedulerSource | v1 | Create, update, and delete Google Cloud Scheduler Jobs. When those jobs are triggered, receive the event inside Knative. See the CloudSchedulerSource example for further details. | |
| CloudStorageSource | v1 | Registers for events of the specified types on the specified Google Cloud Storage bucket and optional object prefix. Brings those events into Knative. See the CloudStorageSource example. | |
| DockerHubSource | v1alpha1 | None | Retrieves events from Docker Hub Webhooks and transforms them into CloudEvents for consumption in Knative. |
| FTP / SFTP | Proof of concept | None | Watches for files being uploaded into a FTP/SFTP and generates events for those. |
| Heartbeat | Proof of Concept | None | Uses an in-memory timer to produce events as the specified interval. Uses AutoContainerSource for underlying infrastructure. |
| Konnek | Active Development | None | Retrieves events from cloud platforms (like AWS and GCP) and transforms them into CloudEvents for consumption in Knative. |
| K8s | Proof of Concept | None | Brings Kubernetes cluster events into Knative. Uses AutoContainerSource for underlying infrastructure. |
| RedisSource | v1alpha1 | None | Brings Redis Stream into Knative. |
| Slack | v1alpha1 | TriggerMesh | Subscribes to events from Slack. |
| VMware | Active Development | None | Brings vSphere events into Knative. |
| Zendesk | v1alpha1 | TriggerMesh | Subscribes to events from Zendesk. |
MaintainerがKnativeとなっているEventソースが公式のサポートするものです。
さすがサポートされるイベントソースが多いですね。
2. KEDA
CNCF 傘下の Event-driven Auto-Scaling プロジェクト。KEDA ではイベントソースを "Scaler" としています。システムそのものを Metrix に振り切った感じがいいですね。
- 公式サイト: https://keda.sh/
- リポジトリ: https://github.com/kedacore/keda
サポートされる Scalers
以下は公式サイトに上げられているScalersです。
| Name | Availability | Maintainter | Description |
|---|---|---|---|
| ActiveMQ Artemis | v1.5+ | Community | Scale applications based on ActiveMQ Artemis queues |
| Apache Kafka | v1.0+ | Microsoft | Scale applications based on an Apache Kafka topic or other services that support Kafka protocol. |
| AWS CloudWatch | v1.0+ | Community | Scale applications based on AWS CloudWatch. |
| AWS Kinesis Stream | v1.1+ | Community | Scale applications based on AWS Kinesis Stream. |
| AWS SQS Queue | v1.0+ | Community | Scale applications based on AWS SQS Queue. |
| Azure Blob Storage | v1.1+ | Community | Scale applications based on the count of blobs in a given Azure Blob Storage container. Notioce:As of now, this Azure Blob Storage scaler scales based on the count of the blobs in a container as opposed to the Azure Functions behavior where code is only triggered on new blobs. |
| Azure Event Hubs | v1.0+ | Community | Scale applications based on Azure Event Hubs. Notice:As of now, the Event Hub scaler only supports reading checkpoints from Blob Storage, as well as scaling only Event Hub applications written in C#, Java, Python or created with Azure Functions. |
| Azure Log Analytics | v2.0+ | Community | Scale applications based on Azure Log Analytics query result |
| Azure Monitor | v1.3+ | Community | Scale applications based on Azure Monitor metrics. |
| Azure Service Bus | v1.0+ | Microsoft | Scale applications based on Azure Service Bus Queues or Topics. |
| Azure Storage Queue | v1.0+ | Microsoft | Scale applications based on Azure Storage Queues. |
| CPU | v2.0+ | Community | Scale applications based on cpu metrics. |
| Cron | v1.5+ | Community | Scale applications based on a cron schedule. |
| External | v1.0+ | Microsoft | Scale applications based on an external scaler. |
| External Push | v2.0+ | Microsoft | Scale applications based on an external push scaler. |
| Google Cloud Platform Pub/Sub | v1.0+ | Community | Scale applications based on Google Cloud Platform Pub/Sub. |
| Huawei Cloudeye | v1.1+ | Community | Scale applications based on a Huawei Cloudeye. |
| IBM MQ | v2.0+ | Community | Scale applications based on IBM MQ Queue |
| InfluxDB | v2.1+ | Community | Scale applications based on InfluxDB queries |
| Liiklus Topic | v1.0+ | Community | Scale applications based on Liiklus Topic. |
| Memory | v2.0+ | Community | Scale applications based on memory metrics. Notice: This scaler will never scale to 0 and even when user defines multiple scaler types (eg. Kafka + cpu/memory, or Prometheus + cpu/memory), the deployment will never scale to 0.This scaler only applies to ScaledObject, not to Scaling Jobs. |
| Metrics API | v2.0+ | Community | Scale applications based on a metric provided by an API |
| MongoDB | v2.1+ | Community | Scale applications based on MongoDB queries. |
| MSSQL | v2.2+ | Microsoft | Scale applications based on Microsoft SQL Server (MSSQL) query results. |
| MySQL | v1.2+ | Community | Scale applications based on MySQL query result. |
| NATS Streaming | v1.0+ | Community | Scale applications based on NATS Streaming. |
| OpenStack Swift | v2.1+ | Community | Scale applications based on the count of objects in a given OpenStack Swift container. |
| PostgreSQL | v1.2+ | Community | Scale applications based on a PostgreSQL query. |
| Prometheus | v1.0+ | Community | Scale applications based on Prometheus. |
| RabbitMQ Queue Latest | v1.0+ | Microsoft | Scale applications based on RabbitMQ Queue. |
| Redis Lists | v1.0+ | Community | Scale applications based on Redis Lists. |
| Redis Lists (supports Redis Cluster) | v2.1+ | Community | Redis Lists scaler with support for Redis Cluster topology |
| Redis Streams | v1.5+ | Community | Scale applications based on Redis Streams. |
| Redis Streams (supports Redis Cluster) | v2.1+ | Community | Redis Streams scaler with support for Redis Cluster topology |
こちらの対応する Scaler もなかなかの層の厚さです。
Microsoft が意外と協力してますね。。。
3. Kubeless
Bitnamiがお届けする Serveless フレームワークです。上記のプロジェクトに比べるとまだまだ小ぢんまりしてますが、WEB UIなども提供されるので OSS版の AWS Lambda といった感じでしょうか。
- 公式サイト: https://kubeless.io/
- リポジトリ: https://github.com/kubeless
サポートされるイベントトリガー
今のところサポートされるイベントトリガーは以下の通りです。
- HTTP Trigger
- CronJob Trigger
- PubSub Triggers
- Kafka Trigger
正直、こんなもんでいいのではないでしょうか・・・
4. fission
こちらもミニマルさがウリ？の fission でございます。コンポーネントにビルドツールなどがありますので、イベントドリブンに書いたコードがサクッとデプロイできる、というのは使いやすいかもしれません。
- 公式サイト: https://fission.io/
- リポジトリ: https://github.com/fission/fission
サポートされるイベントトリガー
サポートされるイベントトリガーは以下の通りです。
- HTTP
- NATS Streaming
- Kafka
- AWS SQS (KEDA based)
- AWS Kinesis (KEDA based)
- NATS Streaming (KEDA based)
- Timer Triggers
次期バージョン？では KEDA連携が予定されている模様です。。。組み合わせてくるかー。。。
5. OpenWhisk
Apache 傘下の Serverless プロジェクトです。k8s 専用のプロジェクトではないのでdocker-composeやmesos、ansible、Vagrant デプロイできるらしいです。
対応言語(環境)は一通り…
- nodejs
- Docker
- Python
- go
- Swift
- PHP
- Java
- Ruby
と、対応するコンポーネントは
- alarm
- cloudant
- kafka
- pushnotification
- rss
- jira
などあります。
これらに対する "トリガー" と "ルール" と "アクション" を定義して対象環境にデプロイ、といった感じです。
さすが Apache、裾野が広い。。。
6. OpenFaaS
シンプルさがウリ？の Serverless フレームワーク。K8s だけではなく OpenShift にも対応している模様です。
- 公式サイト: https://www.openfaas.com/
- リポジトリ: https://github.com/openfaas
将来的には アクションのシェアなどしたい模様がうかがえます。Serverless を突き詰めて IFTTT 的なコミュニティベースの イベント／アクション Hub みたいな感じにしたいのでしょうか。まだまだ途上のプロジェクトの模様です。
番外 Tekton
Knativeからビルドコンポーネントだけスピンアウトした CI/CD プロジェクトです。
- 公式サイト: https://tekton.dev/
- リポジトリ: https://github.com/tektoncd
とても分かりやすい説明がありましたので詳しくは以下のページをご覧ください。
CI/CDやるにはそれなりの"装備"が必要なので、CI/CD 自体の維持にかかるコストもバカになりません。それらもまとめて提供してくれるのはありがたいです。
雑感／サーバレスが変えるパラダイム
これまで Servelessフレームワーク を使って開発は行っていたので "Severless" という概念や仕組みは理解していましたが、単に AWS や GCP の上のサービスメニューの一つとして lambda や Functions があってそれらをどう組み合わせて使うか、という単なるオプションの一つであって、大手のクラウドプロバイダを使わないのであれば関係ない、と思っておりましたが…
k8s でガチの Serverless が、Event-Driven + MicroService のアーキテクチャが、どのようなインフラでも動くとなると話は変わってきます。
k3s や k0s であれば手元の Raspberry Pi でも Serverless は動きます。というか、Event-Driven + MicroService は IoT のエッジ側にこそふさわしい技術なんじゃないか とまで思えてきます。
エッジも Serverless でサーバーサイドも Serverless となり、これまでのWebやアプリ、クライアントを起点とする "Pull" 型のモデルから、Push型メインのモデルへと本格的にパラダイムシフトしてしまった感があります。
KEDAのSQLベースのトリガーなんてヤベーよ。。
ということで本日はここまでとしたいと思います。