Go to Qiita Advent Calendar 2024 Top
LoginSignup
3
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 1 year has passed since last update.

クラウドAI by ナレコムAdvent Calendar 2022

Day 7
@SatoshiGachiFujimoto(藤本 賢志)in株式会社ナレッジコミュニケーション

AWS FinchでAmazon LinuxのJupyterLab環境を作ってみた。Lambdaレイヤー作成用にも使える！

Last updated at Posted at 2022-12-06

はじめに

FinchでJupyterLab環境作ります。
さらにLambdaレイヤー作成用にも使ってみます。

開発環境

  • MacBookPro 2018

Amazon LinuxのJupyterLab環境を作成

1．Finch-v0.1.0-x86_64.pkgをインストール

2．finchを初期化

$ finch vm init

INFO[0000] Using default values due to missing config file at "/Users/user/.finch/finch.yaml" 
INFO[0000] "/Users/user/.finch" directory doesn't exist, attempting to create it 
INFO[0000] binaries directory doesn't exist             
INFO[0000] Requesting root access to finish network dependency configuration 
Password:
INFO[0004] sudoers file not found: %!w(*fs.PathError=&{open /etc/sudoers.d/finch-lima 2}) 
INFO[0004] Initializing and starting Finch virtual machine... 

INFO[0288] Finch virtual machine started successfully

3．2回目からはstartを使用

$ finch vm start

FATA[0000] the instance "finch" is already running

4．サンプルのhello-finchを実行
$ finch run --rm public.ecr.aws/finch/hello-finch

public.ecr.aws/finch/hello-finch:latest:                                          resolved       |++++++++++++++++++++++++++++++++++++++| 
index-sha256:a71e474da9ffd6ec3f8236dbf4ef807dd54531d6f05047edaeefa758f1b1bb7e:    done           |++++++++++++++++++++++++++++++++++++++| 
manifest-sha256:2f848edb93f7d0cfa20d7dc7add84586fe06d258d6dd54422d8015c584ff3b9e: done           |++++++++++++++++++++++++++++++++++++++| 
config-sha256:50c36f221209ea6829db90eff11db167d8cc22abf7c2c0f1e7f4a0c701c0592f:   done           |++++++++++++++++++++++++++++++++++++++| 
layer-sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1:    done           |++++++++++++++++++++++++++++++++++++++| 
layer-sha256:cb7e3bc996dd3208c5b4560d5f6571486e2fe90b5a1cc384e2d6af0dc07de02a:    done           |++++++++++++++++++++++++++++++++++++++| 
elapsed: 3.6 s                                                                    total:  1.0 Mi (295.1 KiB/s)                                     

                            @@@@@@@@@@@@@@@@@@@                                 
                        @@@@@@@@@@@@    @@@@@@@@@@@                             
                      @@@@@@@                  @@@@@@@                          
                    @@@@@@                        @@@@@@                        
                  @@@@@@                            @@@@@                       
                 @@@@@                      @@@#     @@@@@@@@@                  
                @@@@@                     @@   @@@       @@@@@@@@@@             
                @@@@%                     @     @@            @@@@@@@@@@@       
                @@@@                                               @@@@@@@@     
                @@@@                                         @@@@@@@@@@@&       
                @@@@@                                  &@@@@@@@@@@@             
                 @@@@@                               @@@@@@@@                   
                  @@@@@                            @@@@@(                       
                   @@@@@@                        @@@@@@                         
                     @@@@@@@                  @@@@@@@                           
                        @@@@@@@@@@@@@@@@@@@@@@@@@@                              
                            @@@@@@@@@@@@@@@@@@


Hello from Finch!

Visit us @ github.com/runfinch

5．アーキテクチャの確認
$ uname -sm

Darwin x86_64

6．linux/amd64を指定して実行

$ finch run --rm --platform=linux/amd64 public.ecr.aws/amazonlinux/amazonlinux uname -sm

public.ecr.aws/amazonlinux/amazonlinux:latest:                                    resolved       |++++++++++++++++++++++++++++++++++++++| 
index-sha256:f4b5d60ba75876e8a8b02618fd0de3ab796eb210ab37075adf56eab38ec8c3bf:    done           |++++++++++++++++++++++++++++++++++++++| 
manifest-sha256:7e4c85064f9fde50cd075fd80244455fd56bdf6481eaace42e3768cc3d466c6a: done           |++++++++++++++++++++++++++++++++++++++| 
config-sha256:612cb43999513098b3f71dd264efb846765677926bae54fb6b48643a44156568:   done           |++++++++++++++++++++++++++++++++++++++| 
layer-sha256:68028ec3b506bca4d81560b5fcbd408dc7cc49f4b1717a69d5396ff22700f80a:    done           |++++++++++++++++++++++++++++++++++++++| 
elapsed: 56.0s                                                                    total:  59.4 M (1.1 MiB/s)                                       
Linux x86_64

7．--rmを省いて実行

$ finch run --platform=linux/amd64 public.ecr.aws/amazonlinux/amazonlinux uname -sm

8．プロセスの確認
$ finch ps -a

CONTAINER ID    IMAGE                                            COMMAND        CREATED          STATUS                      PORTS    NAMES
a4f675b9f4ff    public.ecr.aws/amazonlinux/amazonlinux:latest    "/bin/bash"    8 minutes ago    Exited (0) 8 minutes ago             amazonlinux-a4f67

9．コンテナの削除
$ finch rm a4f675b9f4ff

10．イメージの確認
$ finch images

REPOSITORY                                TAG       IMAGE ID        CREATED           PLATFORM       SIZE         BLOB SIZE
public.ecr.aws/amazonlinux/amazonlinux    latest    f4b5d60ba758    30 minutes ago    linux/amd64    170.9 MiB    59.4 MiB
public.ecr.aws/finch/hello-finch          latest    a71e474da9ff    25 hours ago      linux/amd64    1.7 MiB      1.0 MiB

11．イメージの削除
$ finch rmi public.ecr.aws/finch/hello-finch

Untagged: public.ecr.aws/finch/hello-finch:latest@sha256:a71e474da9ffd6ec3f8236dbf4ef807dd54531d6f05047edaeefa758f1b1bb7e
Deleted: sha256:2e05c12f4142e6af2317b4097ab105857eb97781088895f41479a9b27378bdcf
Deleted: sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef

$ finch images

REPOSITORY                                TAG       IMAGE ID        CREATED           PLATFORM       SIZE         BLOB SIZE
public.ecr.aws/amazonlinux/amazonlinux    latest    f4b5d60ba758    31 minutes ago    linux/amd64    170.9 MiB    59.4 MiB

12．httpdを実行
$ finch run -d -p 8080:80 httpd

image.png

13．プロセスの確認
$ finch ps

CONTAINER ID    IMAGE                             COMMAND               CREATED           STATUS    PORTS                   NAMES
5cdd48166b71    docker.io/library/httpd:latest    "httpd-foreground"    54 seconds ago    Up        0.0.0.0:8080->80/tcp    httpd-5cdd4

14．コンテナを停止
$ finch stop 5cdd48166b71

15．pythonのバージョン確認
$ finch run --platform=linux/amd64 public.ecr.aws/amazonlinux/amazonlinux python --version

Python 2.7.18

16．下記の構成でDockerfileを作成

finch-python/
  ├ Dockerfile
  └ docker-compose.yml

17．Dockerfileの中身

FROM public.ecr.aws/amazonlinux/amazonlinux:latest
WORKDIR /workdir
RUN amazon-linux-extras install python3
RUN yum install -y gcc python3-devel
RUN pip3 install jupyterlab

18．docker-compose.ymlの中身

docker-compose.yml
version: '3'
services:
  python:
    build: .
    ports:
    - "8888:8888"
    volumes:
    - .:/workdir
    command: jupyter-lab --ip=0.0.0.0 --port=8888 --allow-root --no-browser

19．composeする
$ cd finch-python/
$ finch compose up -d --build

20．コンテナの確認
$ finch container ls

CONTAINER ID    IMAGE                                           COMMAND                   CREATED          STATUS    PORTS                     NAMES
6db5eadcdc73    docker.io/library/finch-python_python:latest    "jupyter-lab --ip=0.…"    5 minutes ago    Up        0.0.0.0:8888->8888/tcp    finch-python_python_1

21．JupyterLabが起動していることを確認
image.png

Lambdaレイヤーの設定に使ってみる

1．コンテナの中に入り/bin/bashを起動、yumを入れようとして断念

$ finch run --name finch-python-run -it finch-python /bin/bash
root@ec70d3c96eed:/workdir# ls
root@ec70d3c96eed:/workdir# python --version
Python 3.11.0
root@ec70d3c96eed:/workdir# mkdir python
root@ec70d3c96eed:/workdir# pip install -t ./python requests
Collecting requests
  Using cached requests-2.28.1-py3-none-any.whl (62 kB)
Collecting charset-normalizer<3,>=2
  Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.13-py2.py3-none-any.whl (140 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2022.9.24-py3-none-any.whl (161 kB)
Installing collected packages: urllib3, idna, charset-normalizer, certifi, requests
Successfully installed certifi-2022.9.24 charset-normalizer-2.1.1 idna-3.4 requests-2.28.1 urllib3-1.26.13
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip available: 22.3 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
root@ec70d3c96eed:/workdir# zip -r requests.zip python
bash: zip: command not found
root@ec70d3c96eed:/workdir# yum install zip
bash: yum: command not found
root@765daf0bf673:/workdir# apt install yum
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package yum

2．finch execを用いてコンテナ上でpython3を実行
$ finch exec -it finch-python_python_1 python3

Python 3.7.10 (default, Jun  3 2021, 00:02:01) 
[GCC 7.3.1 20180712 (Red Hat 7.3.1-13)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

3．finch execを用いてコンテナ上でbashを起動、pythonフォルダにrequestsをpip install
$ finch exec -it finch-python_python_1 bash

bash-4.2# ls
Dockerfile  docker-compose.yml	sample.py
bash-4.2# mkdir python
bash-4.2# python --version
Python 2.7.18
bash-4.2# python3 --version
Python 3.7.10
bash-4.2# pip3 install -t ./python requests
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting requests
  Using cached requests-2.28.1-py3-none-any.whl (62 kB)
Collecting charset-normalizer<3,>=2
  Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2022.9.24-py3-none-any.whl (161 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.13-py2.py3-none-any.whl (140 kB)
Installing collected packages: charset-normalizer, certifi, idna, urllib3, requests
Successfully installed certifi-2022.9.24 charset-normalizer-2.1.1 idna-3.4 requests-2.28.1 urllib3-1.26.13
bash-4.2# cat /etc/system-release
Amazon Linux release 2 (Karoo)
bash-4.2# exit

4．Lambda関数を作成

image.png

5．先ほど作成したpythonフォルダをzip化して、Lambda Layersに登録。ローカルからアップロードできてうれしい！

image.png

7．Lambda関数を実行

lambda_function.py
import json
import requests

def lambda_handler(event, context):
    # TODO implement
    response = requests.get("https://weather.tsukumijima.net/api/forecast/city/400040")
    
    return {
        'statusCode': 200,
        'body': json.dumps(response.json())
    }

image.png

8．プロセスの確認
$ finch ps

CONTAINER ID    IMAGE                                           COMMAND                   CREATED           STATUS    PORTS                     NAMES
49463efb8c15    docker.io/library/finch-python_python:latest    "jupyter-lab --ip=0.…"    35 minutes ago    Up        0.0.0.0:8888->8888/tcp    finch-python_python_1

9．コンテナを停止
$ finch stop 49463efb8c15

$ finch container ls

CONTAINER ID    IMAGE    COMMAND    CREATED    STATUS    PORTS    NAMES

10．finchを停止
$ finch vm stop

INFO[0000] Stopping existing Finch virtual machine...   
INFO[0009] Finch virtual machine stopped successfully

お疲れ様でした

参考文献

3
1
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
3
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?