Edited at

How to use Lake Formation ① (Firsttime Lake Formation)


Lake Formation for the first time

You are prompted to add an administrator user. Since Lake Formation has its own user management and authority management system, it is necessary to add an administrator user for Lake Formation (cooperation with IAM is possible)

https___qiita-image-store.s3.ap-northeast-1.amazonaws.com_0_27932_935dd3d9-7131-899e-61f7-45a74a32b2fe.jpeg

You can link with IAM users and IAM roles. After selecting and adding an IAM user etc., you can start Lake Formation by clicking [Save]

IAM user "uehara" is selected.

https___qiita-image-store.s3.ap-northeast-1.amazonaws.com_0_27932_ea415087-6c78-3b2a-2b18-71110ef9fbee.jpeg

When the addition is over, it will look like this

You can also see that an admin user has been added

https___qiita-image-store.s3.ap-northeast-1.amazonaws.com_0_27932_78ee3aad-e515-27ab-4057-efcd92a6201a.jpeg

By the way, Lake Formation Data Catalog is integrated with Glue's data catalog, and the data catalog that is browsed from Lake Formation and Glue has the same contents. Of course from Athena


Lake Formation

https___qiita-image-store.s3.ap-northeast-1.amazonaws.com_0_27932_3171687d-bb1d-c804-8dcc-17823052d97f.jpeg


Glue DataCatalog


  • On the Glue side, the function of ML Transforms job has been added along with LakeFormation GA (per left bottom of the screen)

https___qiita-image-store.s3.ap-northeast-1.amazonaws.com_0_27932_ff38f742-68ae-e8b0-019f-4f1d0cc7fe17.jpeg


Athena

Athena basically uses Glue's data catalog, so of course it is natural

https___qiita-image-store.s3.ap-northeast-1.amazonaws.com_0_27932_fac6405a-2371-d0d8-1296-173df0a6baaa.jpeg


The screen looks like this

The main menu of Lake Formation Composed

DataCatalog, Register and Ingest, Permissions.

We are responsible for the overall construction of the data lake by extracting and converting data with Register and Ingest, registering metadata, and controlling the user and operation authority with DataPermission.

The link (Crawler, Jobs) on the screen is a link to Glue, and it works strongly with Glue.

Because Lake Formation has its own authority management, the operation log on Lake Formation is displayed at the bottom of the screen

https___qiita-image-store.s3.ap-northeast-1.amazonaws.com_0_27932_af88d468-e2b0-923d-c2d2-0a8dbdbbce9b.jpeg

I'll try various things in the future!


Come here

Official document

https://docs.aws.amazon.com/lake-formation/latest/dg/what-is-lake-formation.html

How to use Lake Formation

https://qiita.com/pioho07/items/093d971dcb5656730ea3

How to use Glue(Japanese)

https://qiita.com/pioho07/items/32f76a16cbf49f9f712f