redis zcount 計算量 100万件調査結果

  • 0
    Like
  • 2
    Comment

    redisのzcountって本番で使って良い悪いがあまり自信がなくて調査
    結論として100万件程度のデータ量なら使って問題ない

    redis 公式ドキュメントを読む

    ZCOUNT について読むと計算量は O(log(N))+O(M) 要素数Mによって線形に計算量が増加。線形に増加は怖いので調査してみた

    結果

    問題なし!

    スクリーンショット 2017-09-13 13.47.35.png

    コード

    from __future__ import absolute_import, unicode_literals
    import redis
    import random
    
    KEY = "SORTED_SET_KEY"
    
    def time(func):
        import functools
        import datetime
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            start = datetime.datetime.today()
            result = func(*args, **kwargs)
            end = datetime.datetime.today()
            return end - start
        return wrapper
    
    def main():
        print "start"
    
        redis_client = redis.StrictRedis(host='localhost', port=6379, db=0)
    
        # redis initial data
        redis_client.delete(KEY)
    
        # set data
        for h in xrange(1, 100):
            for i in xrange(1, 10000):
                player_key = 'player:{}:{}'.format(h, i)
                redis_client.zadd(KEY, random.randint(1, 10000), player_key)
    
            zscore_time_10000 = zscore_time(redis_client, player_key)
            score = 10
            zcount_time_10000 = zcount_time(redis_client, score)
            print "----- {}times zcount:{}".format(h, redis_client.zcount(KEY, score, 'inf'))
            print "10000times zscore:{}sec, zcount:{}sec".format(zscore_time_10000, zcount_time_10000)
    
    @time
    def zscore_time(redis_client, unique_key, count=10000):
        [redis_client.zscore(KEY, unique_key) for x in xrange(count)]
    
    @time
    def zcount_time(redis_client, score, count=10000):
        [redis_client.zcount(KEY, score, 'inf') for x in xrange(count)]
    
    main()
    
    

    実行結果

    > python set_redis_data.py 
    start
    ----- 1times zcount:9992
    10000times zscore:0:00:00.749579sec, zcount:0:00:00.735492sec
    ----- 2times zcount:19978
    10000times zscore:0:00:00.757828sec, zcount:0:00:00.733708sec
    ----- 3times zcount:29973
    10000times zscore:0:00:00.745015sec, zcount:0:00:00.736602sec
    ----- 4times zcount:39962
    10000times zscore:0:00:00.768604sec, zcount:0:00:00.733056sec
    ----- 5times zcount:49951
    10000times zscore:0:00:00.781331sec, zcount:0:00:00.727951sec
    ----- 6times zcount:59940
    10000times zscore:0:00:00.744400sec, zcount:0:00:00.727242sec
    ----- 7times zcount:69930
    10000times zscore:0:00:00.754257sec, zcount:0:00:00.728895sec
    ----- 8times zcount:79920
    10000times zscore:0:00:00.749809sec, zcount:0:00:00.736275sec
    ----- 9times zcount:89908
    10000times zscore:0:00:00.739206sec, zcount:0:00:00.732444sec
    ----- 10times zcount:99901
    10000times zscore:0:00:00.754966sec, zcount:0:00:00.753912sec
    ----- 11times zcount:109891
    10000times zscore:0:00:00.766370sec, zcount:0:00:00.734423sec
    ----- 12times zcount:119885
    10000times zscore:0:00:00.751999sec, zcount:0:00:00.739062sec
    ----- 13times zcount:129872
    10000times zscore:0:00:00.743560sec, zcount:0:00:00.723789sec
    ----- 14times zcount:139857
    10000times zscore:0:00:00.754542sec, zcount:0:00:00.749401sec
    ----- 15times zcount:149852
    10000times zscore:0:00:00.749797sec, zcount:0:00:00.761494sec
    ----- 16times zcount:159845
    10000times zscore:0:00:00.963314sec, zcount:0:00:00.904765sec
    ----- 17times zcount:169834
    10000times zscore:0:00:00.852251sec, zcount:0:00:00.804572sec
    ----- 18times zcount:179828
    10000times zscore:0:00:00.832594sec, zcount:0:00:00.830622sec
    ----- 19times zcount:189812
    10000times zscore:0:00:00.913706sec, zcount:0:00:01.014552sec
    ----- 20times zcount:199798
    10000times zscore:0:00:00.834342sec, zcount:0:00:00.754801sec
    ----- 21times zcount:209786
    10000times zscore:0:00:00.781016sec, zcount:0:00:00.777940sec
    ----- 22times zcount:219780
    10000times zscore:0:00:00.942493sec, zcount:0:00:00.811952sec
    ----- 23times zcount:229768
    10000times zscore:0:00:00.800323sec, zcount:0:00:00.791843sec
    ----- 24times zcount:239761
    10000times zscore:0:00:00.871959sec, zcount:0:00:00.809339sec
    ----- 25times zcount:249747
    10000times zscore:0:00:00.808262sec, zcount:0:00:00.782781sec
    ----- 26times zcount:259740
    10000times zscore:0:00:00.798491sec, zcount:0:00:00.747983sec
    ----- 27times zcount:269729
    10000times zscore:0:00:00.796869sec, zcount:0:00:00.767065sec
    ----- 28times zcount:279724
    10000times zscore:0:00:00.875296sec, zcount:0:00:00.807199sec
    ----- 29times zcount:289715
    10000times zscore:0:00:00.778267sec, zcount:0:00:00.773806sec
    ----- 30times zcount:299704
    10000times zscore:0:00:00.817535sec, zcount:0:00:00.856725sec
    ----- 31times zcount:309689
    10000times zscore:0:00:00.779981sec, zcount:0:00:00.802634sec
    ----- 32times zcount:319679
    10000times zscore:0:00:00.779044sec, zcount:0:00:00.885543sec
    ----- 33times zcount:329668
    10000times zscore:0:00:00.801946sec, zcount:0:00:00.920779sec
    ----- 34times zcount:339659
    10000times zscore:0:00:00.840763sec, zcount:0:00:00.862033sec
    ----- 35times zcount:349647
    10000times zscore:0:00:00.848629sec, zcount:0:00:00.839048sec
    ----- 36times zcount:359635
    10000times zscore:0:00:00.796536sec, zcount:0:00:00.757579sec
    ----- 37times zcount:369628
    10000times zscore:0:00:00.761311sec, zcount:0:00:00.743290sec
    ----- 38times zcount:379620
    10000times zscore:0:00:00.787605sec, zcount:0:00:00.793797sec
    ----- 39times zcount:389610
    10000times zscore:0:00:00.756282sec, zcount:0:00:00.753933sec
    ----- 40times zcount:399601
    10000times zscore:0:00:00.762800sec, zcount:0:00:00.748836sec
    ----- 41times zcount:409588
    10000times zscore:0:00:00.756001sec, zcount:0:00:00.730211sec
    ----- 42times zcount:419579
    10000times zscore:0:00:00.744252sec, zcount:0:00:00.732069sec
    ----- 43times zcount:429565
    10000times zscore:0:00:00.752527sec, zcount:0:00:00.733024sec
    ----- 44times zcount:439545
    10000times zscore:0:00:00.742493sec, zcount:0:00:00.729422sec
    ----- 45times zcount:449538
    10000times zscore:0:00:00.754651sec, zcount:0:00:00.731426sec
    ----- 46times zcount:459530
    10000times zscore:0:00:00.751480sec, zcount:0:00:00.751284sec
    ----- 47times zcount:469518
    10000times zscore:0:00:00.848265sec, zcount:0:00:00.810607sec
    ----- 48times zcount:479504
    10000times zscore:0:00:00.765747sec, zcount:0:00:00.739687sec
    ----- 49times zcount:489496
    10000times zscore:0:00:00.752537sec, zcount:0:00:00.736055sec
    ----- 50times zcount:499486
    10000times zscore:0:00:00.765538sec, zcount:0:00:00.728143sec
    ----- 51times zcount:509477
    10000times zscore:0:00:00.748288sec, zcount:0:00:00.725538sec
    ----- 52times zcount:519468
    10000times zscore:0:00:00.742314sec, zcount:0:00:00.741281sec
    ----- 53times zcount:529456
    10000times zscore:0:00:00.782913sec, zcount:0:00:00.744661sec
    ----- 54times zcount:539448
    10000times zscore:0:00:00.758629sec, zcount:0:00:00.764310sec
    ----- 55times zcount:549439
    10000times zscore:0:00:00.743581sec, zcount:0:00:00.729405sec
    ----- 56times zcount:559430
    10000times zscore:0:00:00.752376sec, zcount:0:00:00.738607sec
    ----- 57times zcount:569419
    10000times zscore:0:00:00.770094sec, zcount:0:00:00.745846sec
    ----- 58times zcount:579408
    10000times zscore:0:00:00.745116sec, zcount:0:00:00.736532sec
    ----- 59times zcount:589397
    10000times zscore:0:00:00.747376sec, zcount:0:00:00.772593sec
    ----- 60times zcount:599386
    10000times zscore:0:00:00.838002sec, zcount:0:00:00.740173sec
    ----- 61times zcount:609379
    10000times zscore:0:00:00.749521sec, zcount:0:00:00.728057sec
    ----- 62times zcount:619375
    10000times zscore:0:00:00.749353sec, zcount:0:00:00.729088sec
    ----- 63times zcount:629359
    10000times zscore:0:00:00.768890sec, zcount:0:00:00.753640sec
    ----- 64times zcount:639350
    10000times zscore:0:00:00.746590sec, zcount:0:00:00.728816sec
    ----- 65times zcount:649343
    10000times zscore:0:00:00.739419sec, zcount:0:00:00.742850sec
    ----- 66times zcount:659335
    10000times zscore:0:00:00.866874sec, zcount:0:00:00.773135sec
    ----- 67times zcount:669329
    10000times zscore:0:00:00.766303sec, zcount:0:00:00.751412sec
    ----- 68times zcount:679319
    10000times zscore:0:00:00.774909sec, zcount:0:00:00.731371sec
    ----- 69times zcount:689309
    10000times zscore:0:00:00.753021sec, zcount:0:00:00.738191sec
    ----- 70times zcount:699298
    10000times zscore:0:00:00.738990sec, zcount:0:00:00.723266sec
    ----- 71times zcount:709287
    10000times zscore:0:00:00.887564sec, zcount:0:00:00.871255sec
    ----- 72times zcount:719281
    10000times zscore:0:00:00.774284sec, zcount:0:00:00.780908sec
    ----- 73times zcount:729272
    10000times zscore:0:00:00.779932sec, zcount:0:00:00.827605sec
    ----- 74times zcount:739263
    10000times zscore:0:00:00.768705sec, zcount:0:00:00.734963sec
    ----- 75times zcount:749252
    10000times zscore:0:00:00.776220sec, zcount:0:00:00.897861sec
    ----- 76times zcount:759246
    10000times zscore:0:00:01.110917sec, zcount:0:00:00.730362sec
    ----- 77times zcount:769235
    10000times zscore:0:00:00.751864sec, zcount:0:00:00.736097sec
    ----- 78times zcount:779224
    10000times zscore:0:00:00.932775sec, zcount:0:00:00.823374sec
    ----- 79times zcount:789206
    10000times zscore:0:00:00.790337sec, zcount:0:00:00.836322sec
    ----- 80times zcount:799188
    10000times zscore:0:00:00.820177sec, zcount:0:00:00.771782sec
    ----- 81times zcount:809177
    10000times zscore:0:00:00.760560sec, zcount:0:00:00.738159sec
    ----- 82times zcount:819170
    10000times zscore:0:00:00.823681sec, zcount:0:00:00.793130sec
    ----- 83times zcount:829159
    10000times zscore:0:00:00.792423sec, zcount:0:00:00.778726sec
    ----- 84times zcount:839149
    10000times zscore:0:00:00.849077sec, zcount:0:00:00.786689sec
    ----- 85times zcount:849133
    10000times zscore:0:00:00.821718sec, zcount:0:00:00.743985sec
    ----- 86times zcount:859119
    10000times zscore:0:00:00.752505sec, zcount:0:00:00.752729sec
    ----- 87times zcount:869106
    10000times zscore:0:00:00.804886sec, zcount:0:00:00.801518sec
    ----- 88times zcount:879095
    10000times zscore:0:00:00.830245sec, zcount:0:00:00.745599sec
    ----- 89times zcount:889085
    10000times zscore:0:00:00.812706sec, zcount:0:00:00.800208sec
    ----- 90times zcount:899072
    10000times zscore:0:00:00.827639sec, zcount:0:00:00.911494sec
    ----- 91times zcount:909064
    10000times zscore:0:00:00.819233sec, zcount:0:00:00.772907sec
    ----- 92times zcount:919055
    10000times zscore:0:00:00.766537sec, zcount:0:00:00.778436sec
    ----- 93times zcount:929043
    10000times zscore:0:00:00.805694sec, zcount:0:00:00.846116sec
    ----- 94times zcount:939033
    10000times zscore:0:00:00.845019sec, zcount:0:00:00.832113sec
    ----- 95times zcount:949024
    10000times zscore:0:00:00.773874sec, zcount:0:00:00.766136sec
    ----- 96times zcount:959013
    10000times zscore:0:00:00.785367sec, zcount:0:00:00.774693sec
    ----- 97times zcount:969002
    10000times zscore:0:00:00.828814sec, zcount:0:00:00.910054sec
    ----- 98times zcount:978993
    10000times zscore:0:00:00.855522sec, zcount:0:00:00.855757sec
    ----- 99times zcount:988978
    10000times zscore:0:00:00.855399sec, zcount:0:00:00.804692sec