準備

easy_install line_profiler

cProfile

コード全体をプロファイリング
どの関数が一番時間を食っているかを明らかにする

python -m cProfile -s cumulative chi2.py > profile
less profile

cProfileの結果

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.002    0.002    7.516    7.516 chi2.py:3(<module>)
        1    0.000    0.000    6.099    6.099 chi2.py:101(chi2)
     4959    0.229    0.000    3.511    0.001 chi2.py:11(detect_step)
       29    0.001    0.000    3.291    0.113 chi2.py:84(calculate_border_bestfit)
      435    0.008    0.000    3.289    0.008 chi2.py:35(detect_step_all_ranges)
    76661    0.219    0.000    2.738    0.000 fromnumeric.py:2830(var)
        1    0.000    0.000    2.538    2.538 chi2.py:53(plot)
    76661    1.215    0.000    2.519    0.000 _methods.py:77(_var)
        1    0.000    0.000    2.421    2.421 pyplot.py:138(show)
        1    0.000    0.000    2.421    2.421 backend_bases.py:140(__call__)
        1    0.000    0.000    2.417    2.417 backend_macosx.py:28(mainloop)
        1    2.323    2.323    2.417    2.417 {matplotlib.backends._macosx.show}
        1    0.001    0.001    1.251    1.251 pyplot.py:17(<module>)

結果を見ると、chi2関数が一番時間を食っている
chi2関数をよく調べてみる

Line profile

chi2関数に@profileを付加

chi2.py

@profile
def chi2(trajectory, t):
    s = []
    border_bestfit_best = []
    max_s = 0.0

    for nstep in range(1, 30):
        border_bestfit = calculate_border_bestfit(trajectory, nstep)
        border_counter = calculate_border_counter(trajectory, border_bestfit)
        s = calculate_chi2(trajectory, border_counter) / calculate_chi2(trajectory, border_bestfit)
        if max_s < s:
            max_s = s
            border_bestfit_best = border_bestfit

    # Calculate average trajectory
    average_trajectory = [np.average(trajectory[border_bestfit_best[i]:border_bestfit_best[i+1]])
                          for i in range(0, len(border_bestfit_best) - 1)]

    # Plot
    plot(trajectory, border_bestfit_best, average_trajectory)

    # Calculate step size
    step_size = [average_trajectory[i+1] - average_trajectory[i]
                 for i in range(0, len(average_trajectory)-1)]

    return step_size

kernprof.pyを使ってlprofファイルを作成

kernprof.py -l chi2.py

Line profilerの結果

python -m line_profiler chi2.py.lprof

Timer unit: 1e-06 s

Total time: 6.75531 s
File: chi2.py
Function: chi2 at line 101

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
   101                                           @profile
   102                                           def chi2(trajectory, t):
   103         1            3      3.0      0.0      s = []
   104         1            1      1.0      0.0      border_bestfit_best = []
   105         1            1      1.0      0.0      max_s = 0.0
   106
   107        30           23      0.8      0.0      for nstep in range(1, 30):
   108        29      4188417 144428.2     62.0          border_bestfit = calculate_border_bestfit(trajectory, nstep)
   109        29       310665  10712.6      4.6          border_counter = calculate_border_counter(trajectory, border_bestfit)
   110        29        46667   1609.2      0.7          s = calculate_chi2(trajectory, border_counter) / calculate_chi2(trajectory, border_bestfit)
   111        29           39      1.3      0.0          if max_s < s:
   112         7            5      0.7      0.0              max_s = s
   113         7            7      1.0      0.0              border_bestfit_best = border_bestfit
   114
   115                                               # Calculate average trajectory
   116         1            1      1.0      0.0      average_trajectory = [np.average(trajectory[border_bestfit_best[i]:border_bestfit_best[i+1]])
   117        12          289     24.1      0.0                            for i in range(0, len(border_bestfit_best) - 1)]
   118
   119                                               # Plot
   120         1      2209171 2209171.0     32.7      plot(trajectory, border_bestfit_best, average_trajectory)
   121
   122                                               # Calculate step size
   123         1            2      2.0      0.0      step_size = [average_trajectory[i+1] - average_trajectory[i]
   124        11           18      1.6      0.0                   for i in range(0, len(average_trajectory)-1)]
   125
   126         1            0      0.0      0.0      return step_size

行ごとにかかった時間が表示されている
chi2関数の計算時間の62%はcalculate_border_bestfit関数の計算に使用されている
今度は、calculate_border_bestfit関数をline profilerにかける
この作業を繰り返すことでどの部分の計算に時間を浪費しているかを明らかにする