LoginSignup
0
1

More than 5 years have passed since last update.

Athenaのパーティションを一括作成して一括削除

Posted at
#!/bin/sh

# ex.
# $ ./xx.sh create 2018-01-01 2018-12-31
# $ ./xx.sh delete 2018-01-01 2018-12-31

METHOD=$1
STARTDATE=$2
ENDDATE=$3
CURRENTDATE=$STARTDATE
REGION=us-east-1
INPUTS3BUCKET=s3://xxx
OUTPUTS3BUCKET=s3://xxx
DBTABLE=データベース名.テーブル名

while [ 1 ] ; do

  CURRENTYEAR=`date -j -f "%Y-%m-%d" "$CURRENTDATE" "+%Y"`
  CURRENTMONTH=`date -j -f "%Y-%m-%d" "$CURRENTDATE" "+%m"`
  CURRENTDAY=`date -j -f "%Y-%m-%d" "$CURRENTDATE" "+%d"`

  if [ $METHOD = "create" ] ; then
    COMMAND="ALTER TABLE $DBTABLE ADD PARTITION (year='$CURRENTYEAR',month='$CURRENTMONTH',day='$CURRENTDAY') location '$INPUTS3BUCKET/$CURRENTYEAR/$CURRENTMONTH/$CURRENTDAY/'"
  elif [ $METHOD = "delete" ] ; then
    COMMAND="ALTER TABLE $DBTABLE DROP PARTITION (year='$CURRENTYEAR',month='$CURRENTMONTH',day='$CURRENTDAY');"
  else
    echo please input create or delete.
    exit 0
  fi

  aws --region $REGION athena start-query-execution --query-string "$COMMAND" --result-configuration OutputLocation=$OUTPUTS3BUCKET

  if [ $CURRENTDATE = $ENDDATE ] ; then
    break
  fi

  CURRENTDATE=`date -v+1d -j -f "%Y-%m-%d" "$CURRENTDATE" "+%Y-%m-%d"`
done

参考にさせてもらいました。
https://qiita.com/ssossan/items/6a4320b9e40af9a132c3

0
1
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
0
1