0
Help us understand the problem. What are the problem?

posted at

Ray ClusterをKubernetes上で試す

Rayとは

Pythonを簡単にスケールできるようにするオープンソースプロジェクト

  • Deep Learning
  • Reinforcement Learning
  • Hyperparameter Tuning
  • General Python apps
  • Model Serving
  • Data Processing

MLがメインのようにみえるが、様々な用途に使える。

Ray-operator

RayをKubernetes上で動かすためのOperatorがある

Custom Resource: RayCluster

ステップ

インストール

kubectl apply -k "github.com/ray-project/kuberay/ray-operator/config/default"

Rayクラスタ作成

RayClusterの作成

kubectl apply -f https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-cluster.heterogeneous.yaml

ConfigMapの作成

kubectl apply -f https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/config-map-ray-code.yaml

ConfigMapに、Pythonのコードが入っている

apiVersion: v1
kind: ConfigMap
metadata:
  name: ray-code
data: 
  sample_code.py: |
    import ray
    from os import environ
    redis_pass = environ.get("REDIS_PASSWORD") 
    print("trying to connect to Ray!")
    ray.init(address="auto", _redis_password=redis_pass)
    print("now executing some code with Ray!")
    import time
    start = time.time()
    @ray.remote
    def f():
      time.sleep(0.01)
      return ray._private.services.get_node_ip_address()
    values=set(ray.get([f.remote() for _ in range(1000)]))
    print("Ray Nodes: ",str(values))
    file = open("/tmp/ray_nodes.txt","a")
    file.write("available nodes: %s\n" % str(values))
    file.close()
    end = time.time()
    print("Execution time = ",end - start)

確認: head 1つとworkerが4つ動いている

kubectl get pod
NAME                                                 READY   STATUS    RESTARTS   AGE
raycluster-heterogeneous-head-72l7l                  1/1     Running   0          2m31s
raycluster-heterogeneous-worker-medium-group-zcml2   1/1     Running   0          2m31s
raycluster-heterogeneous-worker-small-group-fvxd5    1/1     Running   0          2m31s
raycluster-heterogeneous-worker-small-group-hrlt7    1/1     Running   0          2m31s
raycluster-heterogeneous-worker-small-group-nkqj8    1/1     Running   0          2m31s

参考

Why not register and get more from Qiita?
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away
Sign upLogin
0
Help us understand the problem. What are the problem?