Help us understand the problem. What is going on with this article?

時系列データの前処理例

メモ的に。
1分毎のデータから、欠損値を埋めて15分毎の集計データをつくる。

こんな感じの計測データ(変数名:data, 型:DataFrame)がある。

created_at value
2019-09-01 00:00:00 2
2019-09-01 00:01:00 3
2019-09-01 00:02:00 6
2019-09-01 00:05:00 5
2019-09-01 00:07:00 11
2019-09-01 01:00:00 2

対象範囲の1分毎のタイムフレームをつくる。

import pandas as pd

start = '2019-9-01 00:00:00'
end = '2019-9-01 23:59:59'
dr = pd.date_range(start, end, freq='1min')
tf = pd.DataFrame(dr, columns=['created_at'])

タイムフレームと計測データをマージする。タイムフレームから左結合することによって、計測データが無いタイムフレームのvalueはNaNになる。

df = pd.merge(tf, data, how='left')

先頭行の消費量が欠損の場合は0をセットする。

if df.iat[0, 1] is None:
    df.iat[0, 1] = 0

欠損値を前の行の値で埋める。この埋め方はいろいろある。

df.fillna(method='ffill', inplace=True)

対象範囲の15分毎のタイムフレームをつくる。

dr15 = pd.date_range(start, end, freq='15min')

集計したデータをつくる。

import pandas.tseries.offsets as offsets

values = []

for tr in dr15:
    created_from = tr + offsets.Minute(15)
    created_to = created_from + offsets.Minute(15)

    s = df.query('@created_from <= created_at < @created_to')['value'].sum()
    values.append((tr, s))

df15 = pd.DataFrame(values, columns=['created_at', 'value'])
Why do not you register as a user and use Qiita more conveniently?
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away
Comments
Sign up for free and join this conversation.
If you already have a Qiita account
Why do not you register as a user and use Qiita more conveniently?
You need to log in to use this function. Qiita can be used more conveniently after logging in.
You seem to be reading articles frequently this month. Qiita can be used more conveniently after logging in.
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away