AnyPointSystem (powered by Qiitadon) > 2019-05-01 タグごと、日付ごとのカウントリスト作成

動作環境

Ubuntu 18.04 LTs
Jupyter Notebook v5.7.8 
(Python 3.6.7 (default, Oct 22 2018, 11:32:17) )

関連

• Qiitadon (Mastodon) > create_app()のto_fileで保存されるclient_id.txtの中身 | AnyPointSystem始めました
• AnyPointSystem (powered by Qiitadon) > 2019-04-27 システム設計

前準備

1. Mastodon.pyのセットアップ
2. アプリの登録 (Qiitadonの開発にて登録「AnyPointSystem」）
3. client_id.txtの生成 (client_idとclient_secret: 手順2の登録情報から)
4. access_token.txtの生成 (手順2の登録情報から)

概要

• Qiitadonをデータベースとするポイントシステム
• 今回
  • タグごと、日付ごとにカウントしリストにする

実装

PEP8 チェック未実施

Qiitadon_API_190425.ipynb

import sys
from mastodon import Mastodon
import datetime
import csv

# [ ] PEP8 checked

# 2019-05-01 08:14
# 2019-04-29 08:49
# 2019-04-27 09:17

url = "https://qiitadon.com"
cid_file = 'client_id.txt'
token_file = 'access_token.txt'

mastodon = Mastodon(
    client_id=cid_file,
    access_token=token_file,
    api_base_url=url
)

# print(user_toots[0])  # トゥートを全て表示する

NUM_LOOP = 40  # for getting 40 toot each
#NUM_LOOP = 0 # for debug


def isAnyPointData(inlist):
    for elem in inlist:
        # print('***', elem)  # debug
        if 'anypointdata' in elem['name']:
            return True
    return False


def getPointTag(inlist):
    res = []
    for elem in inlist:
        if 'anypointdata' in elem['name']:
            continue
        res += [elem['name']]
    return res


max_id = None
fulllist = []

for loop in range(NUM_LOOP):
    user_dict = mastodon.account_verify_credentials()
    # 40 seems limit as of 2019-04-27
    user_toots = mastodon.account_statuses(user_dict['id'],
                                           max_id=max_id, limit=40)

    max_id = user_toots[-1]['id']
    # print(len(user_toots))
    for idx in range(len(user_toots)):
        # print(user_toots[idx])  # to print what the toot looks like
        atoot = user_toots[idx]['tags']
        atime = user_toots[idx]['created_at']

        # to avoid [TypeError: can't subtract offset-naive and offset-aware datetimes ]
        # https://stackoverflow.com/questions/796008/cant-subtract-offset-naive-and-offset-aware-datetimes
        atime = atime.replace(tzinfo=None)

        if len(atoot) == 0:
            continue
        if not isAnyPointData(atoot):
            continue
        res = getPointTag(atoot)
        fulllist += [[atime, res]]
        print('.', end='')
        #print(res)
        #print(atime)

# for debug
if NUM_LOOP == 0:
    fulllist = [
    [datetime.datetime(2019, 4, 30, 23, 8, 29, 983000), ['中国語学習']],
    [datetime.datetime(2019, 4, 30, 23, 1, 52, 859000), ['英語復習']],
    [datetime.datetime(2019, 4, 30, 13, 25, 46, 588000), ['中国語復習']],
    [datetime.datetime(2019, 4, 30, 13, 1, 31, 340000), ['ツール作成']],
    [datetime.datetime(2019, 4, 30, 12, 58, 27, 258000), ['animalbyaurora']],
    [datetime.datetime(2019, 4, 30, 9, 40, 31, 145000), ['チカッとチカ千花っ']],
    [datetime.datetime(2019, 4, 30, 9, 21, 52, 735000), ['farbymarieueda']],
    [datetime.datetime(2019, 4, 30, 9, 14, 35, 939000), ['queendombyaudora']],
    [datetime.datetime(2019, 4, 30, 9, 10, 58, 67000), ['animalbyaurora']],
    [datetime.datetime(2019, 4, 30, 2, 59, 50, 350000), ['英語復習']],
    [datetime.datetime(2019, 4, 29, 23, 32, 33, 842000), ['grenning学習', 'tdd学習']],
    [datetime.datetime(2019, 4, 29, 23, 27, 7, 216000), ['英語復習']],
    [datetime.datetime(2019, 4, 29, 13, 21, 5, 351000), ['英語学習']],
    [datetime.datetime(2019, 4, 29, 13, 5, 28, 552000), ['chinese']],
    [datetime.datetime(2019, 4, 29, 12, 30, 45, 331000), ['中国語学習']],
    [datetime.datetime(2019, 4, 29, 11, 57, 15, 318000), ['farbymarieueda']],
    [datetime.datetime(2019, 4, 29, 9, 51, 2, 24000), ['チカッとチカ千花っ']],
    [datetime.datetime(2019, 4, 29, 9, 33, 31, 33000), ['queendombyaudora']],
    [datetime.datetime(2019, 4, 29, 9, 32, 7, 586000), ['英語学習']],
    [datetime.datetime(2019, 4, 29, 9, 26, 30, 169000), ['farbymarieueda']],
    [datetime.datetime(2019, 4, 29, 9, 22, 31, 857000), ['animalbyaurora']],
    [datetime.datetime(2019, 4, 29, 1, 30, 51, 103000), ['ツール作成']],
    [datetime.datetime(2019, 4, 29, 1, 29, 41, 267000), ['チカッとチカ千花っ']],
    [datetime.datetime(2019, 4, 28, 23, 25, 35, 632000), ['queendombyaudora']],
    [datetime.datetime(2019, 4, 28, 23, 18, 31, 362000), ['animalbyaurora']],
    [datetime.datetime(2019, 4, 28, 23, 0, 21, 950000), ['英語復習']],
    [datetime.datetime(2019, 4, 28, 12, 36, 59, 80000), ['中国語復習']],
    [datetime.datetime(2019, 4, 28, 3, 34, 23, 115000), ['queendombyaurora']],
    [datetime.datetime(2019, 4, 28, 3, 30, 38, 750000), ['animalbyaurora']],
    [datetime.datetime(2019, 4, 28, 3, 29, 39, 44000), ['中国語学習']],
    [datetime.datetime(2019, 4, 28, 1, 18, 9, 823000), ['grenning学習', 'tdd学習']],
    [datetime.datetime(2019, 4, 28, 1, 17, 45, 883000), ['チョコ食べる']],
    ]


print()  # for new line

# add 1st tag
tagcsv = fulllist[0][1][0]  # 0: 1st toot, 1: tag, 0: expand from list
#print(tagcsv)
for elem in fulllist:
    #print(elem[1])
    atag = elem[1][0]
    if atag not in tagcsv:
        tagcsv += "," + str(elem[1][0])
taglist = tagcsv.split(",")

DAYS_PAST = 40
BASE_DATE = datetime.datetime(2019,4,1,0,0,0) 

for trgtag in taglist:  # loop by target tag
    wrklist = [0 for i in range(DAYS_PAST)]
    print(trgtag)
    for elem in fulllist:
        adate = elem[0]
        tags = elem[1]
        for atag in tags:
            if trgtag not in atag:
                continue
            elpsd = (int)((elem[0] - BASE_DATE).total_seconds() / 60 / 60 / 24)
            wrklist[elpsd] += 1
            #print(elpsd, adate, atag)
    print(wrklist)

実行例

................................................................
中国語学習
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 1, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
英語復習
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
中国語復習
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
ツール作成
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
animalbyaurora
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 4, 0, 2, 1, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
チカッとチカ千花っ
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
farbymarieueda
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
queendombyaudora
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
grenning学習
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
英語学習
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
chinese
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
queendombyaurora
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
チョコ食べる
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
英語資料整理
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
英語リスニング
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
cerafilter
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
ツール開発
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
珈琲飲まない
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
仮眠
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

実装 (PEP8適用後)

Line too longは未対応

Qiitadon_API_190425.ipynb

import sys
from mastodon import Mastodon
import datetime
import csv

# [x] PEP8 checked

# 2019-05-01 09:55
# 2019-05-01 08:14
# 2019-04-29 08:49
# 2019-04-27 09:17

url = "https://qiitadon.com"
cid_file = 'client_id.txt'
token_file = 'access_token.txt'

mastodon = Mastodon(
    client_id=cid_file,
    access_token=token_file,
    api_base_url=url
)

# print(user_toots[0])  # トゥートを全て表示する

NUM_LOOP = 40  # for getting 40 toot each
# NUM_LOOP = 0 # for debug


def isAnyPointData(inlist):
    for elem in inlist:
        # print('***', elem)  # debug
        if 'anypointdata' in elem['name']:
            return True
    return False


def getPointTag(inlist):
    res = []
    for elem in inlist:
        if 'anypointdata' in elem['name']:
            continue
        res += [elem['name']]
    return res


max_id = None
fulllist = []

for loop in range(NUM_LOOP):
    user_dict = mastodon.account_verify_credentials()
    # 40 seems limit as of 2019-04-27
    user_toots = mastodon.account_statuses(user_dict['id'],
                                           max_id=max_id, limit=40)

    max_id = user_toots[-1]['id']
    # print(len(user_toots))
    for idx in range(len(user_toots)):
        # print(user_toots[idx])  # to print what the toot looks like
        atoot = user_toots[idx]['tags']
        atime = user_toots[idx]['created_at']

        # to avoid [TypeError: can't subtract offset-naive and offset-aware datetimes ]
        # https://stackoverflow.com/questions/796008/cant-subtract-offset-naive-and-offset-aware-datetimes
        atime = atime.replace(tzinfo=None)

        if len(atoot) == 0:
            continue
        if not isAnyPointData(atoot):
            continue
        res = getPointTag(atoot)
        fulllist += [[atime, res]]
        print('.', end='')
        # print(res)
        # print(atime)

# for debug
if NUM_LOOP == 0:
    fulllist = [
        [datetime.datetime(2019, 4, 30, 23, 8, 29, 983000), ['中国語学習']],
        [datetime.datetime(2019, 4, 30, 23, 1, 52, 859000), ['英語復習']],
        [datetime.datetime(2019, 4, 30, 13, 25, 46, 588000), ['中国語復習']],
        [datetime.datetime(2019, 4, 30, 13, 1, 31, 340000), ['ツール作成']],
        [datetime.datetime(2019, 4, 30, 12, 58, 27, 258000), ['animalbyaurora']],
        [datetime.datetime(2019, 4, 30, 9, 40, 31, 145000), ['チカッとチカ千花っ']],
        [datetime.datetime(2019, 4, 30, 9, 21, 52, 735000), ['farbymarieueda']],
        [datetime.datetime(2019, 4, 30, 9, 14, 35, 939000), ['queendombyaudora']],
        [datetime.datetime(2019, 4, 30, 9, 10, 58, 67000), ['animalbyaurora']],
        [datetime.datetime(2019, 4, 30, 2, 59, 50, 350000), ['英語復習']],
        [datetime.datetime(2019, 4, 29, 23, 32, 33, 842000), ['grenning学習', 'tdd学習']],
        [datetime.datetime(2019, 4, 29, 23, 27, 7, 216000), ['英語復習']],
        [datetime.datetime(2019, 4, 29, 13, 21, 5, 351000), ['英語学習']],
        [datetime.datetime(2019, 4, 29, 13, 5, 28, 552000), ['chinese']],
        [datetime.datetime(2019, 4, 29, 12, 30, 45, 331000), ['中国語学習']],
        [datetime.datetime(2019, 4, 29, 11, 57, 15, 318000), ['farbymarieueda']],
        [datetime.datetime(2019, 4, 29, 9, 51, 2, 24000), ['チカッとチカ千花っ']],
        [datetime.datetime(2019, 4, 29, 9, 33, 31, 33000), ['queendombyaudora']],
        [datetime.datetime(2019, 4, 29, 9, 32, 7, 586000), ['英語学習']],
        [datetime.datetime(2019, 4, 29, 9, 26, 30, 169000), ['farbymarieueda']],
        [datetime.datetime(2019, 4, 29, 9, 22, 31, 857000), ['animalbyaurora']],
        [datetime.datetime(2019, 4, 29, 1, 30, 51, 103000), ['ツール作成']],
        [datetime.datetime(2019, 4, 29, 1, 29, 41, 267000), ['チカッとチカ千花っ']],
        [datetime.datetime(2019, 4, 28, 23, 25, 35, 632000), ['queendombyaudora']],
        [datetime.datetime(2019, 4, 28, 23, 18, 31, 362000), ['animalbyaurora']],
        [datetime.datetime(2019, 4, 28, 23, 0, 21, 950000), ['英語復習']],
        [datetime.datetime(2019, 4, 28, 12, 36, 59, 80000), ['中国語復習']],
        [datetime.datetime(2019, 4, 28, 3, 34, 23, 115000), ['queendombyaurora']],
        [datetime.datetime(2019, 4, 28, 3, 30, 38, 750000), ['animalbyaurora']],
        [datetime.datetime(2019, 4, 28, 3, 29, 39, 44000), ['中国語学習']],
        [datetime.datetime(2019, 4, 28, 1, 18, 9, 823000), ['grenning学習', 'tdd学習']],
        [datetime.datetime(2019, 4, 28, 1, 17, 45, 883000), ['チョコ食べる']],
    ]

print()  # for new line

# add 1st tag
tagcsv = fulllist[0][1][0]  # 0: 1st toot, 1: tag, 0: expand from list
# print(tagcsv)
for elem in fulllist:
    # print(elem[1])
    atag = elem[1][0]
    if atag not in tagcsv:
        tagcsv += "," + str(elem[1][0])
taglist = tagcsv.split(",")

DAYS_PAST = 40
BASE_DATE = datetime.datetime(2019, 4, 1, 0, 0, 0)

for trgtag in taglist:  # loop by target tag
    wrklist = [0 for i in range(DAYS_PAST)]
    print(trgtag)
    for elem in fulllist:
        adate = elem[0]
        tags = elem[1]
        for atag in tags:
            if trgtag not in atag:
                continue
            elpsd = (int)((elem[0] - BASE_DATE).total_seconds() / 60 / 60 / 24)
            wrklist[elpsd] += 1
            # print(elpsd, adate, atag)
    print(wrklist)