データエンジニアとしてデータ調査のお仕事
この記事は freeeデータに関わる人たち Advent Calendar 2020 11日目のエントリーです。 データエンジニアという仕事をやってます。今年もいろいろなデータを調べました。その...
10
6
Comment0
97 search resultsShowing 1~20 results
Stocked
You need to log-in
AWS Glue で 億超えレコードなテーブルからETLする
この記事はfreee データに関わる人たち Advent Calendar 2019の11日目です。 シンプルにAWS Glueで RDB(MySQLとか)から巨大なテーブルデータを取り出すとき...
16
12
Comment0
debian & ubuntu memo
Where are utils? dig = dnsutils ps,pstree,top = procps ss = iproute2 Ignore apt install interacti...
1
1
Comment0
ECS note
ECS Network mode bridge ECS network mode = bridge means both its name and driver. So, we have to ...
1
0
Comment0
Docker note
ECS move https://qiita.com/h-imaoka/items/0bc296e8b0084e897cfa warning: cannot increase max open ...
1
1
Comment0
How to build Linux client
hibernation root> swapoff /swapfile root> fallocate -l XGB /swapfile root> chmod 600 /sw...
0
0
Comment0
S3 maniacs
Sample bucket policies receive by ses { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowSE...
2
3
Comment0
Amazon Linux note
Security update Amazon Linux Security Center v2 Install old versions @ amazon-extras. $ amazon-li...
0
0
Comment0
supervisor note
in Docker supervisor @ docker means a single container/multi process. So if you want to use pytho...
0
0
Comment0
Python programing note
urllib (python3) ignore redirect Via class import urllib.request class HTTPNoRedirectHandler(urll...
0
0
Comment0
embulk note
S3 incremental in: type: s3 bucket: [bucket] path_prefix: [prefix] ... incremental: true parser: ...
1
1
Comment0
PostgreSQL note
with docker Connection via tcp create users via postgres user. mydb's owner will be postgres. sud...
0
0
Comment0
powershellメモ
add $PATH temporary $Env:Path += ";C:¥hoge¥fuga¥" service Get-Service | Format-Table -AutoSize re...
8
9
Comment0
putty note
proxy command putty via GUI create a session to bastion. create new session to server over bastio...
0
0
Comment0
Linux memo
diagnosis smtp-auth from smtplib import SMTP smtp = SMTP('smtp.ecample.com') smtp.login('<user...
1
1
Comment0
corretto memo
install @ amazon v2 amazon-linux-extras install corretto8 # if you want to use jdk too yum instal...
0
0
Comment0
aws note
AWS IoT Use it simple MQTT broker? Yes! Rule Engine for bin-data? Support. but no filter rule, ex...
1
0
Comment0
mac note
karabiner-Elements key layout JP to US ~/.config/karabiner/assets/jptojskey.json { "title": "Rema...
0
0
Comment0
My Windows-Client
Terminal Cmder (comemu) http://cmder.net/ Shells CMD,Powershell Via Cmder. For Windows Native bin...
0
0
Comment0
AWS del ami/snapshot
via ssm (bash) export list=$(aws ec2 describe-images --owner self --query 'Images[][ImageId]' --o...
0
0
Comment0
97 search resultsShowing 1~20 results
Qiita is a knowledge sharing service for engineers.
- You can follow users and tags
- You can stock useful information
- You can make edit suggestions for articles