What is AWS Transfer family?
The AWS Transfer Family provides fully managed support for file transfers directly into and out of Amazon S3 or Amazon EFS. With support for Secure File Transfer Protocol (SFTP), File Transfer Protocol over SSL (FTPS), and File Transfer Protocol (FTP), the AWS Transfer Family helps you seamlessly migrate your file transfer workflows to AWS by integrating with existing authentication systems, and providing DNS routing with Amazon Route 53 so nothing changes for your customers and partners, or their applications. With your data in Amazon S3 or Amazon EFS, you can use it with AWS services for processing, analytics, machine learning, archiving, as well as home directories and developer tools. Getting started with the AWS Transfer Family is easy; there is no infrastructure to buy and set up.
- It can be used to transfer files into and out of S3 and EFS
- SFTP, FTPS, and FTP protocols are supported
How do we use it?
Demo
Visit https://ap-northeast-1.console.aws.amazon.com/transfer/home?region=ap-northeast-1#/
→ Click Create server
Set up configurations
Choose protocols
We can select one of the following options
- SFTP (SSH File Transfer Protocol)
- FTPS (File Transfer Protocol Secure)
- FTP (File Transfer Protocol)
Check SFTP (SSH File Transfer Protocol) - file transfer over Secure Shell and click Next
Choose an identity provider
We can select one of the following protocols
- Service managed
- Custom
Check Service managed and click Next
Choose an endpoint
We need to configure endpoint type and custom hostname. We have the following options for an endpoint.
- Publicly accessible
- VPC hosted
Check Publicly accessible and click Next
Choose a domain
We can use S3 and EFS as the destination or starting point of transfer of files. Click Amazon S3 and click Next.
Configure additional details
We can attach an existing role or create a new role to allow the transfer family to put logs in CloudWatch Logs. Choose to Create a new role and click Next
Review and create
Click Create server below.
Check on console
Add a service managed user
Click Server ID of the server that you want to add a user to. Click Add user.
User configuration
Attach to the role AWSTransferLoggingAccess
Input Username transfer-family-test, choose the role AWSTransferLoggingAccess. Then, choose S3 bucket where the files are transferred and
Generate SSH key
> ssh-keygen -P "" -m PEM -f ~/.ssh/transfer-family-test
Generating public/private rsa key pair.
Your identification has been saved in /Users/Kanata-Miyahana/.ssh/transfer-family-test.
Your public key has been saved in /Users/Kanata-Miyahana/.ssh/transfer-family-test.pub.
The key fingerprint is:
SHA256:zsBLQFnZn7PL+Q8ioX68yge7Nqo2lpCAgE4VcvDzM5A kanata-miyahana@ip-192-168-3-2.ap-northeast-1.compute.internal
The key's randomart image is:
+---[RSA 3072]----+
|.oo+oo.o |
|o.+.o . . |
|= E. . . |
|o. +o + |
|.. ++ S o |
|o .+* .. |
| . . o++..o. |
| = o+ +.+. . |
| o.o.o=*.. .... |
+----[SHA256]-----+
> cat ~/.ssh/transfer-family-test.pub | pbcopy
enter the public SSH key and click Add.
※ You can follow the instruction here to create an SSH public key.
Confirm that the user is successfully added
Install client application
Install Cyberduck
> brew install cyberduck --cask
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 3 taps (homebrew/cask-versions, homebrew/core and homebrew/cask).
==> New Formulae
snowpack
==> Updated Formulae
Updated 28 formulae.
==> New Casks
devbook goldenpassport netnewswire-beta
==> Updated Casks
Updated 26 casks.
==> Downloading https://update.cyberduck.io/Cyberduck-7.8.2.34203.zip
==> Downloading from https://a48823c7ec3cf4539564-60c534a1284a12ce74ef84032e9b4e46.ssl.cf1.rackcdn.com/Cyberduck-7.8.2.
######################################################################## 100.0%
==> Installing Cask cyberduck
==> Moving App 'Cyberduck.app' to '/Applications/Cyberduck.app'
🍺 cyberduck was successfully installed!
Set up configurations
If Cyberduck is successfully installed, open the application. Click + button.
Set the following configuration
- SFTP
- endpoint → you can get it on AWS console
Test connection
Check that there is no file on S3
> aws s3 ls s3://transfer-family-test-miyahana-local
# There is nothing because we have never transferred any files
Open directory where the file you want to transfer locates, drag, and drop the file. You can copy the file from local to S3!!!
Check S3
> aws s3 ls s3://transfer-family-test-miyahana-local/
2021-03-09 00:04:58 12 test.txt
> aws s3 cp s3://transfer-family-test-miyahana-local/test.txt .
download: s3://transfer-family-test-miyahana-local/test.txt to ./test.txt
> cat test.txt
hello world
Clean up
- Delete the server
- Delete ssh key pairs
- Delete the bucket
You can follow instructions