1
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 1 year has passed since last update.

Numbaは爆速だが、Dict(辞書型)は遅い

Last updated at Posted at 2022-07-22

Numbaの高速ぶりには感謝しております。

下記サイトでもNumba使われてますが、Numbaが断然速い。本PCでもJavaScriptコードでは約3秒。Numbaは掲載記事通りのパラなら3.5秒程度。fastmathやparallelパラ付けで0.7秒位。断然速い。
https://itnext.io/compare-c-js-python-python-numba-php7-php8-and-golang-in-prime-number-calculation-55e82b6f82a9

PythonのコードをよりNumbaへ置き換えたいのですがDictがあると高速化はできていません。
わかる方がいらしたら教えて頂けると幸いです。

キーは数字/Numba有り

  • NoPythonモードでビルドできます
import numpy as np
from numba import njit
import time
start = time.time()


@njit(cache=True, parallel=True, fastmath=True)
def dict_try4nb():
    d = dict()
    k = {1: np.arange(1), 2: np.arange(2)}
    # The following tells the compiler what the key type and the
    # value
    # type are for `d`.
    d[3] = np.arange(3)
    d[5] = np.arange(5)
    return d, k


d, k = dict_try4nb()
print(d)    # {3: [0 1 2], 5: [0 1 2 3 4]}
print(k)    # {1: [0], 2: [0 1]}
print(f"Cost: {time.time()-start:.1f}sec")

結果

Cost: 14.6sec # Cache前なので遅い!?
Cost: 4.3sec
Cost: 4.1sec

キーは数字/Numba無し

import numpy as np
from numba import njit
import time
start = time.time()


def dict_try4nb():
    d = dict()
    k = {1: np.arange(1), 2: np.arange(2)}
    # The following tells the compiler what the key type and the
    # value
    # type are for `d`.
    d[3] = np.arange(3)
    d[5] = np.arange(5)
    return d, k


d, k = dict_try4nb()
print(d)    # {3: [0 1 2], 5: [0 1 2 3 4]}
print(k)    # {1: [0], 2: [0 1]}
print(f"Cost: {time.time()-start:.1f}sec")
  • 結果
Cost: 0.0sec
Cost: 0.0sec
Cost: 0.0sec

キーは文字列/Numba有り

  • NoPythonモードはエラー生じます。解決策ご存知でしたら教えて頂けるとたすかります。

from random import randint
import numpy as np
from numba import njit, prange, jit
import time


@jit(cache=True, parallel=True, forceobj=True)
def dict_str_try4nb(n):
    start = time.time()
    d = dict()
    k = {1: np.arange(1), 2: np.arange(2)}
    # The following tells the compiler what the key type and the
    # value
    # type are for `d`.
    d[3] = np.arange(3)
    d[5] = np.arange(5)
    for i in prange(n):
        # d[i]=np.arange(10)
        d[f"テスト{i}"] = randint(1, 1000)
    return start, d, k


start, d, k = dict_str_try4nb(100_000)
# print(d)    # {3: [0 1 2], 5: [0 1 2 3 4]}
# print(k)    # {1: [0], 2: [0 1]}
print(f"Cost: {time.time()-start:.1f}sec")
  • 結果
    徐々に早くなっている?
Cost: 2.1sec
Cost: 1.7sec
Cost: 1.5sec

キーは文字列/Numba無し

from random import randint
import numpy as np
from numba import njit, prange, jit
import time


def dict_str_try4nb(n):
    start = time.time()
    d = dict()
    k = {1: np.arange(1), 2: np.arange(2)}
    # The following tells the compiler what the key type and the
    # value
    # type are for `d`.
    d[3] = np.arange(3)
    d[5] = np.arange(5)
    for i in prange(n):
        # d[i]=np.arange(10)
        d[f"テスト{i}"] = randint(1, 1000)
    return start, d, k


start, d, k = dict_str_try4nb(100_000)
# print(d)    # {3: [0 1 2], 5: [0 1 2 3 4]}
# print(k)    # {1: [0], 2: [0 1]}
print(f"Cost: {time.time()-start:.1f}sec")
  • 結果
Cost: 1.0sec
Cost: 1.1sec
Cost: 1.0sec

np.arrangeをprimitive型値へ/キーは数値

Normal(Numba無し)

from random import randint
import numpy as np
from numba import njit, prange, jit
import time
start = time.time()

def dict_str_try4nb(n):

    d = dict()
    k = {1: 1, 2: 2}
    # The following tells the compiler what the key type and the
    # value
    # type are for `d`.
    d[3] = 3
    d[5] = 5
    for i in prange(n):
        # d[i]=np.arange(10)
        d[1] = randint(1, 1000)
    return start, d, k


start, d, k = dict_str_try4nb(100_000)
# print(d)    # {3: [0 1 2], 5: [0 1 2 3 4]}
# print(k)    # {1: [0], 2: [0 1]}
print(f"Cost: {time.time()-start:.1f}sec")

結果

Cost: 0.1sec
Cost: 0.1sec
Cost: 0.1sec

Numba有り

  • Cacheやfastmathパラを印加して最適化
from random import randint
import numpy as np
from numba import njit, prange, jit
import time

start = time.time()


@njit(cache=True, parallel=True, fastmath=True)
def dict_str_try4nb(n):
    d = dict()
    k = {1: 1, 2: 2}
    # The following tells the compiler what the key type and the
    # value
    # type are for `d`.
    d[3] = 3
    d[5] = 5
    for i in prange(n):
        # d[i]=np.arange(10)
        d[1] = randint(1, 1000)
    return d, k


d, k = dict_str_try4nb(100_000)
# print(d)    # {3: [0 1 2], 5: [0 1 2 3 4]}
# print(k)    # {1: [0], 2: [0 1]}
print(f"Cost: {time.time()-start:.1f}sec")
  • 結果
    キーを数値でもNumba無しより遅い。
Cost: 0.4sec
Cost: 0.2sec
Cost: 0.2sec
1
1
3

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
1
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?