LoginSignup
9
10

More than 3 years have passed since last update.

[AWS] SAM + Lambda(Python)で、PythonライブラリをLayerに追加して利用してみる

Last updated at Posted at 2020-08-11

SAMのインストール

[AWS] Serverless Application Model (SAM) でAPI Gateway + Lambda + DynamoDBなサンプルを作成してみる」を参照ください。

なお、今回のサンプルに限っていうと、DynamoDBをローカルに持つ必要はないので、Dockerのインストールは不要です。

プロジェクトの作成

では、例によって、HelloWorldベースのプロジェクトを作成していきましょう。

$ sam init --runtime=python3.8
Which template source would you like to use?
    1 - AWS Quick Start Templates
    2 - Custom Template Location
Choice: 1

Project name [sam-app]:

Cloning app templates from https://github.com/awslabs/aws-sam-cli-app-templates.git

AWS quick start application templates:
    1 - Hello World Example
    2 - EventBridge Hello World
    3 - EventBridge App from scratch (100+ Event Schemas)
    4 - Step Functions Sample App (Stock Trader)
    5 - Elastic File System Sample App
Template selection: 1

-----------------------
Generating application:
-----------------------
Name: sam-app
Runtime: python3.8
Dependency Manager: pip
Application Template: hello-world
Output Directory: .

Next steps can be found in the README file at ./sam-app/README.md

Layerを定義する

まず、Layer用のディレクトリを作成します。
ディレクトリは、プロジェクト直下で、ディレクトリ名は後ほど定義に記述するので、自由につけて大丈夫です。
とりあえず「hello_world_layer」とでもしておきます。

$ mkdir hello_world_layer

続いて、template.yamlのResources配下に、Layerの定義を追記します。
今回は「HelloWorldLayer」という名称で作成しておきます。

template.yml変更前
Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Runtime: python3.8
      Events:
        HelloWorld:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /hello
            Method: get
template.yml変更後
Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Runtime: python3.8
      Events:
        HelloWorld:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /hello
            Method: get
      Layers:
        - !Ref HelloWorldLayer
  HelloWorldLayer:
    Type: AWS::Serverless::LayerVersion
    Properties:
      Description: Layer description
      ContentUri: hello_world_layer/
      CompatibleRuntimes:
        - python3.8
    Metadata:
      BuildMethod: python3.8

NumPyを追加してみる

次に、ライブラリを追加します。
利用したいライブラリを記述してください。
今回のサンプルでは、機械学習でよく使用する数値計算モジュールのNumPyを追加してみたいと思います。

hello_world_layerに、requirements.txtを作成し、以下のように記述して、ライブラリを追加します。

$ touch hello_world_layer/requirements.txt
hello_world_layer/requirements.txt
numpy

Layerクラスを作成する

hello_world_layer配下に、numpyの処理をするユーザクラスを作成してみましょう。
名前は適当にuser_numpyとでもしておきましょうか。

$ touch hello_world_layer/user_numpy.py
hello_world_layer/user_numpy.py
import numpy as np

class UserNumpy():
    def __init__(self):
        super().__init__()

    def array(self):
        return np.array([0,1,2])

Layerクラスのメソッドを呼び出す

最後に、Functionの方で、追加したLayerメソッドを呼び出してみましょう。
既存のhello_world/app.pyを以下のように変更してみましょう。

helloi_world/app.py
import json
from user_numpy import UserNumpy

def lambda_handler(event, context):
    un = UserNumpy()

    un_str = [str(n) for n in un.array()]
    return {
        "statusCode": 200,
        "body": json.dumps({
            "message": ','.join(un_str)
        }),
    }

UserNumpyのarrayメソッドは、Numpyのarray([0,1,2])の結果を返します。
これは数値の配列なので、一度文字列の配列に変換したあとで、文字列かしてJsonデータとして返すようにしています。

ビルド

では、ビルドしてみましょう。

$ sam build
Building function 'HelloWorldFunction'
Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource
Building layer 'HelloWorldLayer'
Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Deploy: sam deploy --guided

デプロイ

次にデプロイです。
事前に、AWSのクレデンシャルを設定しておいてください。

$ sam deploy --guided

Configuring SAM deploy
======================

    Looking for samconfig.toml :  Found
    Reading default arguments  :  Success

    Setting default arguments for 'sam deploy'
    =========================================
    Stack Name [sam-app]:
    AWS Region [us-east-1]: ap-northeast-1
    #Shows you resources changes to be deployed and require a 'Y' to initiate deploy
    Confirm changes before deploy [Y/n]: y
    #SAM needs permission to be able to create roles to connect to the resources in your template
    Allow SAM CLI IAM role creation [Y/n]: y
    HelloWorldFunction may not have authorization defined, Is this okay? [y/N]: y
    Save arguments to samconfig.toml [Y/n]: y

    Looking for resources needed for deployment: Not found.
    Creating the required resources...
    Successfully created!

        Managed S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-fqjmg5s30i1w
        A different default S3 bucket can be set in samconfig.toml

    Deploying with following values
    ===============================
    Stack name                 : sam-app
    Region                     : ap-northeast-1
    Confirm changeset          : True
    Deployment s3 bucket       : aws-sam-cli-managed-default-samclisourcebucket-fqjmg5s30i1w
    Capabilities               : ["CAPABILITY_IAM"]
    Parameter overrides        : {}

Initiating deployment
=====================

    Saved arguments to config file
    Running 'sam deploy' for future deployments will use the parameters saved above.
    The above parameters can be changed by modifying samconfig.toml
    Learn more about samconfig.toml syntax at
    https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-config.html
Uploading to sam-app/ec3e7cb3e8873a2b56d81ae66ba15ec5  262144 / 538577.0  (48.67Uploading to sam-app/ec3e7cb3e8873a2b56d81ae66ba15ec5  524288 / 538577.0  (97.35Uploading to sam-app/ec3e7cb3e8873a2b56d81ae66ba15ec5  538577 / 538577.0  (100.00%)
Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  262144 / 14534393.0  (1.8Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  524288 / 14534393.0  (3.6Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  786432 / 14534393.0  (5.4Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  1048576 / 14534393.0  (7.Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  1310720 / 14534393.0  (9.Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  1572864 / 14534393.0  (10Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  1835008 / 14534393.0  (12Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  2097152 / 14534393.0  (14Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  2359296 / 14534393.0  (16Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  2621440 / 14534393.0  (18Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  2883584 / 14534393.0  (19Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  3145728 / 14534393.0  (21Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  3407872 / 14534393.0  (23Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  3670016 / 14534393.0  (25Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  3932160 / 14534393.0  (27Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  4194304 / 14534393.0  (28Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  4456448 / 14534393.0  (30Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  4718592 / 14534393.0  (32Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  4980736 / 14534393.0  (34Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  5242880 / 14534393.0  (36Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  5505024 / 14534393.0  (37Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  5767168 / 14534393.0  (39Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  6029312 / 14534393.0  (41Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  6145785 / 14534393.0  (42Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  6407929 / 14534393.0  (44Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  6670073 / 14534393.0  (45Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  6932217 / 14534393.0  (47Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  7194361 / 14534393.0  (49Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  7456505 / 14534393.0  (51Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  7718649 / 14534393.0  (53Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  7980793 / 14534393.0  (54Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  8242937 / 14534393.0  (56Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  8505081 / 14534393.0  (58Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  8767225 / 14534393.0  (60Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  9029369 / 14534393.0  (62Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  9291513 / 14534393.0  (63Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  9553657 / 14534393.0  (65Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  9815801 / 14534393.0  (67Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  10077945 / 14534393.0  (6Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  10340089 / 14534393.0  (7Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  10602233 / 14534393.0  (7Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  10864377 / 14534393.0  (7Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  11126521 / 14534393.0  (7Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  11388665 / 14534393.0  (7Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  11650809 / 14534393.0  (8Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  11912953 / 14534393.0  (8Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  12175097 / 14534393.0  (8Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  12437241 / 14534393.0  (8Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  12699385 / 14534393.0  (8Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  12961529 / 14534393.0  (8Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  13223673 / 14534393.0  (9Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  13485817 / 14534393.0  (9Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  13747961 / 14534393.0  (9Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  14010105 / 14534393.0  (9Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  14272249 / 14534393.0  (9Uploading to sam-app/04efcd43f42b7344dff7e02a5e9d321c  14534393 / 14534393.0  (100.00%)

    Deploying with following values
    ===============================
    Stack name                 : sam-app
    Region                     : ap-northeast-1
    Confirm changeset          : True
    Deployment s3 bucket       : aws-sam-cli-managed-default-samclisourcebucket-fqjmg5s30i1w
    Capabilities               : ["CAPABILITY_IAM"]
    Parameter overrides        : {}

Initiating deployment
=====================
HelloWorldFunction may not have authorization defined.
Uploading to sam-app/5c2143d527c3bab330c5c2145019dcb1.template  1455 / 1455.0  (100.00%)

Waiting for changeset to be created..

CloudFormation stack changeset
------------------------------------------------------------------------------------------------
Operation                        LogicalResourceId                ResourceType
------------------------------------------------------------------------------------------------
+ Add                            HelloWorldFunctionHelloWorldPe   AWS::Lambda::Permission
                                 rmissionProd
+ Add                            HelloWorldFunctionRole           AWS::IAM::Role
+ Add                            HelloWorldFunction               AWS::Lambda::Function
+ Add                            HelloWorldLayer8a10103d68        AWS::Lambda::LayerVersion
+ Add                            ServerlessRestApiDeployment47f   AWS::ApiGateway::Deployment
                                 c2d5f9d
+ Add                            ServerlessRestApiProdStage       AWS::ApiGateway::Stage
+ Add                            ServerlessRestApi                AWS::ApiGateway::RestApi
------------------------------------------------------------------------------------------------

Changeset created successfully. arn:aws:cloudformation:ap-northeast-1:************:changeSet/samcli-deploy1597147509/8e325896-aa90-4379-afa2-44147e907291


Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]: y

2020-08-11 21:05:21 - Waiting for stack create/update to complete

CloudFormation events from changeset
-------------------------------------------------------------------------------------------------
ResourceStatus           ResourceType             LogicalResourceId        ResourceStatusReason
-------------------------------------------------------------------------------------------------
CREATE_IN_PROGRESS       AWS::Lambda::LayerVers   HelloWorldLayer8a10103   -
                         ion                      d68
CREATE_IN_PROGRESS       AWS::IAM::Role           HelloWorldFunctionRole   Resource creation
                                                                           Initiated
CREATE_IN_PROGRESS       AWS::IAM::Role           HelloWorldFunctionRole   -
CREATE_IN_PROGRESS       AWS::Lambda::LayerVers   HelloWorldLayer8a10103   Resource creation
                         ion                      d68                      Initiated
CREATE_COMPLETE          AWS::Lambda::LayerVers   HelloWorldLayer8a10103   -
                         ion                      d68
CREATE_COMPLETE          AWS::IAM::Role           HelloWorldFunctionRole   -
CREATE_IN_PROGRESS       AWS::Lambda::Function    HelloWorldFunction       -
CREATE_IN_PROGRESS       AWS::Lambda::Function    HelloWorldFunction       Resource creation
                                                                           Initiated
CREATE_COMPLETE          AWS::Lambda::Function    HelloWorldFunction       -
CREATE_IN_PROGRESS       AWS::ApiGateway::RestA   ServerlessRestApi        -
                         pi
CREATE_IN_PROGRESS       AWS::ApiGateway::RestA   ServerlessRestApi        Resource creation
                         pi                                                Initiated
CREATE_COMPLETE          AWS::ApiGateway::RestA   ServerlessRestApi        -
                         pi
CREATE_IN_PROGRESS       AWS::ApiGateway::Deplo   ServerlessRestApiDeplo   -
                         yment                    yment47fc2d5f9d
CREATE_IN_PROGRESS       AWS::Lambda::Permissio   HelloWorldFunctionHell   Resource creation
                         n                        oWorldPermissionProd     Initiated
CREATE_IN_PROGRESS       AWS::Lambda::Permissio   HelloWorldFunctionHell   -
                         n                        oWorldPermissionProd
CREATE_COMPLETE          AWS::ApiGateway::Deplo   ServerlessRestApiDeplo   -
                         yment                    yment47fc2d5f9d
CREATE_IN_PROGRESS       AWS::ApiGateway::Deplo   ServerlessRestApiDeplo   Resource creation
                         yment                    yment47fc2d5f9d          Initiated
CREATE_IN_PROGRESS       AWS::ApiGateway::Stage   ServerlessRestApiProdS   -
                                                  tage
CREATE_IN_PROGRESS       AWS::ApiGateway::Stage   ServerlessRestApiProdS   Resource creation
                                                  tage                     Initiated
CREATE_COMPLETE          AWS::ApiGateway::Stage   ServerlessRestApiProdS   -
                                                  tage
CREATE_COMPLETE          AWS::Lambda::Permissio   HelloWorldFunctionHell   -
                         n                        oWorldPermissionProd
CREATE_COMPLETE          AWS::CloudFormation::S   sam-app                  -
                         tack
-------------------------------------------------------------------------------------------------

CloudFormation outputs from deployed stack
-------------------------------------------------------------------------------------------------
Outputs
-------------------------------------------------------------------------------------------------
Key                 HelloWorldFunctionIamRole
Description         Implicit IAM Role created for Hello World function
Value               arn:aws:iam::************:role/sam-app-HelloWorldFunctionRole-9AHTS9BZMUQL

Key                 HelloWorldApi
Description         API Gateway endpoint URL for Prod stage for Hello World function
Value               https://co2snvo0lk.execute-api.ap-northeast-1.amazonaws.com/Prod/hello/

Key                 HelloWorldFunction
Description         Hello World Lambda Function ARN
Value               arn:aws:lambda:ap-northeast-1:************:function:sam-app-
HelloWorldFunction-1GVL3Y25TQYGZ
-------------------------------------------------------------------------------------------------

Successfully created/updated stack - sam-app in ap-northeast-1

実行

最後に、Deploy時に出力されるAPI Gatewayのエンドポイントにアクセスしてみます。

$ curl https://co2snvo0lk.execute-api.ap-northeast-1.amazonaws.com/Prod/hello/
{"message": "0,1,2"}

レスポンスに、array([0,1,2])の結果が文字列として返されていることが確認できましたね!!

まとめ

Layerにライブラリを追加し、それを呼び出す共通モジュールを作ることは、簡単であることがわかったと思います。
サーバーレスの場合、なるべく疎結合に設計するのが望ましいですが、同一ドメイン内での共通ロジックをLayerに配置すること自体は間違いではないと思います。

ただし、Lambdaの制約として

  • デプロイパッケージサイズ
    • 圧縮済みで50MB
    • 解凍後さで250MB

という制約がありますので、これを考慮しながら、必要なライブラリのみを取り込むようにしましょう。

サンプルコードリポジトリ

9
10
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
9
10