準備
easy_install line_profiler
cProfile
- コード全体をプロファイリング
- どの関数が一番時間を食っているかを明らかにする
python -m cProfile -s cumulative chi2.py > profile
less profile
cProfileの結果
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.002 0.002 7.516 7.516 chi2.py:3(<module>)
1 0.000 0.000 6.099 6.099 chi2.py:101(chi2)
4959 0.229 0.000 3.511 0.001 chi2.py:11(detect_step)
29 0.001 0.000 3.291 0.113 chi2.py:84(calculate_border_bestfit)
435 0.008 0.000 3.289 0.008 chi2.py:35(detect_step_all_ranges)
76661 0.219 0.000 2.738 0.000 fromnumeric.py:2830(var)
1 0.000 0.000 2.538 2.538 chi2.py:53(plot)
76661 1.215 0.000 2.519 0.000 _methods.py:77(_var)
1 0.000 0.000 2.421 2.421 pyplot.py:138(show)
1 0.000 0.000 2.421 2.421 backend_bases.py:140(__call__)
1 0.000 0.000 2.417 2.417 backend_macosx.py:28(mainloop)
1 2.323 2.323 2.417 2.417 {matplotlib.backends._macosx.show}
1 0.001 0.001 1.251 1.251 pyplot.py:17(<module>)
- 結果を見ると、chi2関数が一番時間を食っている
- chi2関数をよく調べてみる
Line profile
- chi2関数に@profileを付加
chi2.py
@profile
def chi2(trajectory, t):
s = []
border_bestfit_best = []
max_s = 0.0
for nstep in range(1, 30):
border_bestfit = calculate_border_bestfit(trajectory, nstep)
border_counter = calculate_border_counter(trajectory, border_bestfit)
s = calculate_chi2(trajectory, border_counter) / calculate_chi2(trajectory, border_bestfit)
if max_s < s:
max_s = s
border_bestfit_best = border_bestfit
# Calculate average trajectory
average_trajectory = [np.average(trajectory[border_bestfit_best[i]:border_bestfit_best[i+1]])
for i in range(0, len(border_bestfit_best) - 1)]
# Plot
plot(trajectory, border_bestfit_best, average_trajectory)
# Calculate step size
step_size = [average_trajectory[i+1] - average_trajectory[i]
for i in range(0, len(average_trajectory)-1)]
return step_size
- kernprof.pyを使ってlprofファイルを作成
kernprof.py -l chi2.py
Line profilerの結果
python -m line_profiler chi2.py.lprof
Timer unit: 1e-06 s
Total time: 6.75531 s
File: chi2.py
Function: chi2 at line 101
Line # Hits Time Per Hit % Time Line Contents
==============================================================
101 @profile
102 def chi2(trajectory, t):
103 1 3 3.0 0.0 s = []
104 1 1 1.0 0.0 border_bestfit_best = []
105 1 1 1.0 0.0 max_s = 0.0
106
107 30 23 0.8 0.0 for nstep in range(1, 30):
108 29 4188417 144428.2 62.0 border_bestfit = calculate_border_bestfit(trajectory, nstep)
109 29 310665 10712.6 4.6 border_counter = calculate_border_counter(trajectory, border_bestfit)
110 29 46667 1609.2 0.7 s = calculate_chi2(trajectory, border_counter) / calculate_chi2(trajectory, border_bestfit)
111 29 39 1.3 0.0 if max_s < s:
112 7 5 0.7 0.0 max_s = s
113 7 7 1.0 0.0 border_bestfit_best = border_bestfit
114
115 # Calculate average trajectory
116 1 1 1.0 0.0 average_trajectory = [np.average(trajectory[border_bestfit_best[i]:border_bestfit_best[i+1]])
117 12 289 24.1 0.0 for i in range(0, len(border_bestfit_best) - 1)]
118
119 # Plot
120 1 2209171 2209171.0 32.7 plot(trajectory, border_bestfit_best, average_trajectory)
121
122 # Calculate step size
123 1 2 2.0 0.0 step_size = [average_trajectory[i+1] - average_trajectory[i]
124 11 18 1.6 0.0 for i in range(0, len(average_trajectory)-1)]
125
126 1 0 0.0 0.0 return step_size
- 行ごとにかかった時間が表示されている
- chi2関数の計算時間の62%はcalculate_border_bestfit関数の計算に使用されている
- 今度は、calculate_border_bestfit関数をline profilerにかける
- この作業を繰り返すことでどの部分の計算に時間を浪費しているかを明らかにする