Kubernetes ナビ

本シリーズでは kubernetes クラスタを構成する様々なコンポーネント・技術をざっとご紹介しております。

今回はその5となります。

Kubernetes で使える Serverless フレームワークいろいろ

以下で k8s にデプロイできる Serverless フレームワークをご紹介いたします。

1. Knative

元々は Google 社で開発されていたフレームワークをオープンにしたもの。Istio ベースで HTTPサービスもサポートしている。

公式サイト: https://knative.dev/
リポジトリ: https://github.com/knative

サポートされる Event ソース

以下は公式サイトに上げられている代表的な Event コンポーネントです。

Name	API Version	Maintainer	Description
APIServerSource	v1	Knative	Brings Kubernetes API server events into Knative. The APIServerSource fires a new event each time a Kubernetes resource is created, updated or deleted.
AWS SQS	v1alpha1	Knative	Brings AWS Simple Queue Service messages into Knative. The AwsSqsSource fires a new event each time an event is published on an AWS SQS topic.
Apache Camel	v1alpha1	Knative	Enables use of Apache Camel components for pushing events into Knative. A CamelSource is an event source that can represent any existing Apache Camel component, that provides a consumer side, and enables publishing events to an addressable endpoint. Each Camel endpoint has the form of a URI where the scheme is the ID of the component to use. CamelSource requires Camel-K to be installed into the current namespace. See the CamelSource example.
Apache CouchDB	v1alpha1	Knative	Brings Apache CouchDB messages into Knative.
Apache Kafka	v1beta1	Knative	Brings Apache Kafka messages into Knative. The KafkaSource reads events from an Apache Kafka Cluster, and passes these events to a sink so that they can be consumed. See the Kafka Source example for more details.
Container Source	v1	Knative	The ContainerSource will instantiate container image(s) that can generate events until the ContainerSource is deleted. This may be used, for example, to poll an FTP server for new files or generate events at a set time interval. Given a spec.template with at least a container image specified, ContainerSource will keep a Pod running with the specified image(s). K_SINK (destination address) and KE_CE_OVERRIDES (JSON CloudEvents attributes) environment variables are injected into the running image(s). It is used by multiple other Sources as underlying infrastructure. Refer to the Container Source example for more details.
GitHub	v1alpha1	Knative	Registers for events of the specified types on the specified GitHub organization or repository, and brings those events into Knative. The GitHubSource fires a new event for selected GitHub event types. See the GitHub Source example for more details.
GitLab	v1alpha1	Knative	Registers for events of the specified types on the specified GitLab repository, and brings those events into Knative. The GitLabSource creates a webhooks for specified event types, listens for incoming events, and passes them to a consumer. See the GitLab Source example for more details.
Heartbeats	N/A	Knative	Uses an in-memory timer to produce events at the specified interval.
PingSource	v1beta2	Knative	Produces events with a fixed payload on a specified Cron schedule. See the Ping Source example for more details.
RabbitMQ	Active	development	None
SinkBinding	v1	Knative	The SinkBinding can be used to author new event sources using any of the familiar compute abstractions that Kubernetes makes available (e.g. Deployment, Job, DaemonSet, StatefulSet), or Knative abstractions (e.g. Service, Configuration). SinkBinding provides a framework for injecting K_SINK (destination address) and K_CE_OVERRIDES (JSON cloudevents attributes) environment variables into any Kubernetes resource which has a spec.template that looks like a Pod (aka PodSpecable). See the SinkBinding example for more details.
WebSocket	N/A	Knative	Opens a WebSocket to the specified source and packages each received message as a Knative event.
Auto Container Source	Proof of Concept	None	AutoContainerSource is a controller that allows the Source CRDs without needing a controller. It notices CRDs with a specific label and starts controlling resources of that type. It utilizes Container Source as underlying infrastructure.
Amazon CloudWatch	Supported	TriggerMesh	Collects metrics from Amazon CloudWatch.
Amazon CloudWatch Logs	Supported	TriggerMesh	Subscribes to log events from an Amazon CloudWatch Logs stream.
Amazon CodeCommit	Supported	TriggerMesh	Registers for events emitted by an Amazon CodeCommit source code repository.
Amazon Cognito Identity	Supported	TriggerMesh	Registers for events from Amazon Cognito identity pools.
Amazon Cognito User	Supported	TriggerMesh	Registers for events from Amazon Cognito user pools.
Amazon DynamoDB	Supported	TriggerMesh	Reads records from an Amazon DynamoDB stream.
Amazon Kinesis	Supported	TriggerMesh	Reads records from an Amazon Kinesis stream.
Amazon SNS	Supported	TriggerMesh	Subscribes to messages from an Amazon SNS topic.
Amazon SQS	Supported	TriggerMesh	Consumes messages from an Amazon SQS queue.
BitBucket	Proof of Concept	None	Registers for events of the specified types on the specified BitBucket organization/repository. Brings those events into Knative.
CloudAuditLogsSource	v1	Google	Registers for events of the specified types on the specified Google Cloud Audit Logs. Brings those events into Knative. Refer to the CloudAuditLogsSource example for more details.
CloudPubSubSource	v1	Google	Brings Cloud Pub/Sub messages into Knative. The CloudPubSubSource fires a new event each time a message is published on a Google Cloud Platform PubSub topic. See the CloudPubSubSource example for more details.
CloudSchedulerSource	v1	Google	Create, update, and delete Google Cloud Scheduler Jobs. When those jobs are triggered, receive the event inside Knative. See the CloudSchedulerSource example for further details.
CloudStorageSource	v1	Google	Registers for events of the specified types on the specified Google Cloud Storage bucket and optional object prefix. Brings those events into Knative. See the CloudStorageSource example.
DockerHubSource	v1alpha1	None	Retrieves events from Docker Hub Webhooks and transforms them into CloudEvents for consumption in Knative.
FTP / SFTP	Proof of concept	None	Watches for files being uploaded into a FTP/SFTP and generates events for those.
Heartbeat	Proof of Concept	None	Uses an in-memory timer to produce events as the specified interval. Uses AutoContainerSource for underlying infrastructure.
Konnek	Active Development	None	Retrieves events from cloud platforms (like AWS and GCP) and transforms them into CloudEvents for consumption in Knative.
K8s	Proof of Concept	None	Brings Kubernetes cluster events into Knative. Uses AutoContainerSource for underlying infrastructure.
RedisSource	v1alpha1	None	Brings Redis Stream into Knative.
Slack	v1alpha1	TriggerMesh	Subscribes to events from Slack.
VMware	Active Development	None	Brings vSphere events into Knative.
Zendesk	v1alpha1	TriggerMesh	Subscribes to events from Zendesk.

MaintainerがKnativeとなっているEventソースが公式のサポートするものです。
さすがサポートされるイベントソースが多いですね。

2. KEDA

CNCF 傘下の Event-driven Auto-Scaling プロジェクト。KEDA ではイベントソースを "Scaler" としています。システムそのものを Metrix に振り切った感じがいいですね。

公式サイト: https://keda.sh/
リポジトリ: https://github.com/kedacore/keda

サポートされる Scalers

以下は公式サイトに上げられているScalersです。

Name	Availability	Maintainter	Description
ActiveMQ Artemis	v1.5+	Community	Scale applications based on ActiveMQ Artemis queues
Apache Kafka	v1.0+	Microsoft	Scale applications based on an Apache Kafka topic or other services that support Kafka protocol.
AWS CloudWatch	v1.0+	Community	Scale applications based on AWS CloudWatch.
AWS Kinesis Stream	v1.1+	Community	Scale applications based on AWS Kinesis Stream.
AWS SQS Queue	v1.0+	Community	Scale applications based on AWS SQS Queue.
Azure Blob Storage	v1.1+	Community	Scale applications based on the count of blobs in a given Azure Blob Storage container. Notioce:As of now, this Azure Blob Storage scaler scales based on the count of the blobs in a container as opposed to the Azure Functions behavior where code is only triggered on new blobs.
Azure Event Hubs	v1.0+	Community	Scale applications based on Azure Event Hubs. Notice:As of now, the Event Hub scaler only supports reading checkpoints from Blob Storage, as well as scaling only Event Hub applications written in C#, Java, Python or created with Azure Functions.
Azure Log Analytics	v2.0+	Community	Scale applications based on Azure Log Analytics query result
Azure Monitor	v1.3+	Community	Scale applications based on Azure Monitor metrics.
Azure Service Bus	v1.0+	Microsoft	Scale applications based on Azure Service Bus Queues or Topics.
Azure Storage Queue	v1.0+	Microsoft	Scale applications based on Azure Storage Queues.
CPU	v2.0+	Community	Scale applications based on cpu metrics.
Cron	v1.5+	Community	Scale applications based on a cron schedule.
External	v1.0+	Microsoft	Scale applications based on an external scaler.
External Push	v2.0+	Microsoft	Scale applications based on an external push scaler.
Google Cloud Platform‎ Pub/Sub	v1.0+	Community	Scale applications based on Google Cloud Platform‎ Pub/Sub.
Huawei Cloudeye	v1.1+	Community	Scale applications based on a Huawei Cloudeye.
IBM MQ	v2.0+	Community	Scale applications based on IBM MQ Queue
InfluxDB	v2.1+	Community	Scale applications based on InfluxDB queries
Liiklus Topic	v1.0+	Community	Scale applications based on Liiklus Topic.
Memory	v2.0+	Community	Scale applications based on memory metrics. Notice: This scaler will never scale to 0 and even when user defines multiple scaler types (eg. Kafka + cpu/memory, or Prometheus + cpu/memory), the deployment will never scale to 0.This scaler only applies to ScaledObject, not to Scaling Jobs.
Metrics API	v2.0+	Community	Scale applications based on a metric provided by an API
MongoDB	v2.1+	Community	Scale applications based on MongoDB queries.
MSSQL	v2.2+	Microsoft	Scale applications based on Microsoft SQL Server (MSSQL) query results.
MySQL	v1.2+	Community	Scale applications based on MySQL query result.
NATS Streaming	v1.0+	Community	Scale applications based on NATS Streaming.
OpenStack Swift	v2.1+	Community	Scale applications based on the count of objects in a given OpenStack Swift container.
PostgreSQL	v1.2+	Community	Scale applications based on a PostgreSQL query.
Prometheus	v1.0+	Community	Scale applications based on Prometheus.
RabbitMQ Queue Latest	v1.0+	Microsoft	Scale applications based on RabbitMQ Queue.
Redis Lists	v1.0+	Community	Scale applications based on Redis Lists.
Redis Lists (supports Redis Cluster)	v2.1+	Community	Redis Lists scaler with support for Redis Cluster topology
Redis Streams	v1.5+	Community	Scale applications based on Redis Streams.
Redis Streams (supports Redis Cluster)	v2.1+	Community	Redis Streams scaler with support for Redis Cluster topology

こちらの対応する Scaler もなかなかの層の厚さです。
Microsoft が意外と協力してますね。。。

3. Kubeless

Bitnamiがお届けする Serveless フレームワークです。上記のプロジェクトに比べるとまだまだ小ぢんまりしてますが、WEB UIなども提供されるので OSS版の AWS Lambda といった感じでしょうか。

公式サイト: https://kubeless.io/
リポジトリ: https://github.com/kubeless

サポートされるイベントトリガー

今のところサポートされるイベントトリガーは以下の通りです。

HTTP Trigger
CronJob Trigger
PubSub Triggers
Kafka Trigger

正直、こんなもんでいいのではないでしょうか・・・

4. fission

こちらもミニマルさがウリ？の fission でございます。コンポーネントにビルドツールなどがありますので、イベントドリブンに書いたコードがサクッとデプロイできる、というのは使いやすいかもしれません。

公式サイト: https://fission.io/
リポジトリ: https://github.com/fission/fission

サポートされるイベントトリガー

サポートされるイベントトリガーは以下の通りです。

HTTP
NATS Streaming
Kafka
AWS SQS (KEDA based)
AWS Kinesis (KEDA based)
NATS Streaming (KEDA based)
Timer Triggers

次期バージョン？では KEDA連携が予定されている模様です。。。組み合わせてくるかー。。。

5. OpenWhisk

Apache 傘下の Serverless プロジェクトです。k8s 専用のプロジェクトではないのでdocker-composeやmesos、ansible、Vagrant デプロイできるらしいです。

公式サイト: https://openwhisk.apache.org/
リポジトリ: https://github.com/apache?q=openwhisk

対応言語(環境)は一通り…

nodejs
Docker
Python
go
Swift
PHP
Java
Ruby

と、対応するコンポーネントは

alarm
cloudant
kafka
pushnotification
rss
jira

などあります。

これらに対する "トリガー" と "ルール" と "アクション" を定義して対象環境にデプロイ、といった感じです。
さすが Apache、裾野が広い。。。

6. OpenFaaS

シンプルさがウリ？の Serverless フレームワーク。K8s だけではなく OpenShift にも対応している模様です。

公式サイト: https://www.openfaas.com/
リポジトリ: https://github.com/openfaas

将来的にはアクションのシェアなどしたい模様がうかがえます。Serverless を突き詰めて IFTTT 的なコミュニティベースのイベント／アクション Hub みたいな感じにしたいのでしょうか。まだまだ途上のプロジェクトの模様です。

番外 Tekton

Knativeからビルドコンポーネントだけスピンアウトした CI/CD プロジェクトです。

公式サイト: https://tekton.dev/
リポジトリ: https://github.com/tektoncd

とても分かりやすい説明がありましたので詳しくは以下のページをご覧ください。

CI/CDやるにはそれなりの"装備"が必要なので、CI/CD 自体の維持にかかるコストもバカになりません。それらもまとめて提供してくれるのはありがたいです。

雑感／サーバレスが変えるパラダイム

これまで Servelessフレームワークを使って開発は行っていたので "Severless" という概念や仕組みは理解していましたが、単に AWS や GCP の上のサービスメニューの一つとして lambda や Functions があってそれらをどう組み合わせて使うか、という単なるオプションの一つであって、大手のクラウドプロバイダを使わないのであれば関係ない、と思っておりましたが…
k8s でガチの Serverless が、Event-Driven + MicroService のアーキテクチャが、どのようなインフラでも動くとなると話は変わってきます。
k3s や k0s であれば手元の Raspberry Pi でも Serverless は動きます。というか、Event-Driven + MicroService は IoT のエッジ側にこそふさわしい技術なんじゃないか とまで思えてきます。
エッジも Serverless でサーバーサイドも Serverless となり、これまでのWebやアプリ、クライアントを起点とする "Pull" 型のモデルから、Push型メインのモデルへと本格的にパラダイムシフトしてしまった感があります。

KEDAのSQLベースのトリガーなんてヤベーよ。。

ということで本日はここまでとしたいと思います。

参考にしたページ