Lake Formation allows fine-grained authority control in the data catalog.
For each user, you can prohibit adding tables to the database of the data catalog, hide certain columns, and limit the SQL that can be executed. I'll try.
File name: person.csv
Upload the following CSV file to s3://test-lf01/in00
taro, 20190101, 101, 00001
jiro, 20190202, 102, 00001
ojiro, 20190303, 101, 00002
hanako, 20190404, 103, 00004
Schema assumes the following
Click "Database" from the left menu on the Lake Formation screen, and click [Create database] in the upper right
Enter "lf01" in Name, uncheck "Grant All to Everyone for new tables in this database", and click [Create database] at the bottom right
※"Everyone" is an element of the Lake Formation user group. "Everyone" includes all principals with IAM permissions on Lake Formation, not the general public.
"Grant All to Everyone for new tables in this database" is a setting that grants all privileges to "Everyone" for new tables created in this database. The default is checked.
The database was ready.
Click "Tables" from the menu on the left side of the Lake Formation screen, and click [Create table] in the upper right (this time manually create the table)
Enter the following (following)
Include path to s3://lf01/in00/
* It seems that you cannot select a bucket in another region
Click [Add column] in the upper right.
Add columns. The column to add↓
Click [Submit] at the bottom right when you finish adding.
The table was ready.
For the database, the administrator has full privileges and "Everyone" has full privileges (by default, full privileges are given to "Everyone" when creating a database)
For the table, the administrator has full authority and no "Everyone" (as described above, this time when creating a database, the newly created table is set not to give full authority to "Everyone"). So only the administrator can operate it.
First of all, the administrator can confirm that the data catalog created by Lake Formation can be viewed on Athena.
Confirm authority control at the column level, which is one of the sales of Lake Formation
This time, I don't want to display the storeid and accountnumber columns for some users. I don't want to drop the table. Put the control.
The procedure is omitted, but test_user1 with Administrator policy was created like this.
test_user1 is administrator at IAM
When you access Athena with test_user1, you can see lf01 of Database, but the table is not displayed because there is no authority to the table yet. Of course, queries are also rejected.
You could see it before Lake Formation appeared. You can think of this feature as a Lake Formation resource-based policy.
Set by Data Permissions of Lake Formation. Hide only the specific columns that you originally wanted to do
I don't want to display the storeid and accountnumber columns for the test_user1 user. I don't want to drop the table. Put the control.
Click "Data Permissions" in the left menu on the LakeFormation screen, then click [Grant] in the upper right.
Enter the following (following)
IAM user add roles : test_user1
Enter the following and click [Grant] at the bottom right
Select "Include columns" for Column
Select "name" and "date" for Include columns
Check "select" for Table permissions
The result looks something like this
For test_user1, only the name and date columns are visible.
DDL is rejected properly
You can see a message that looks like a permission error as follows.
MetaException(message:Insufficient Lake Formation permission(s):...
The Lake Access Recent Access Activity log looks something like this
* Since it is a CloudTrail log, it will be output after about 10 minutes. Also on this screen, the filter is quite weak ...
Click on View event in the upper right to check details
You can enable or disable the following two default permissions for newly created databases and tables (default is enabled):
- Give All permission to "Everyone" for newly created database
- Grant ALL permission to "Everyone" on newly created table
If you don't need it, uncheck it and click [Save] at the bottom right. As a result, the newly created database or table is not automatically granted to "Everyone" (it can be operated with IAM authority), and operations that give explicit fine-grained permissions on Lake Formation can be performed. I can do it.
"Everyone" is an entity that exists for migration from GlueDataCatalog.
How to use lake formation
How to use Glue(Japanese)