Play Python async

There are a few post about python asyncio introduced since Python 3.3. I played around and this is the note for myself.

Summary

I evaluated thread and coroutine and find out the fact that the performace does not have a huge difference. Though, async is faster a little.(delimiter: I tested them with a really simple model. I do not say this is the ground truth.) Perhaps, the advantage of coroutine is chaining using yield from, which is easy to write compared with using queue and thread

Spec

• Python 3.5dev
• Mac OSX 10.10
• Memory DDR3 16GB
• CPU 2.3 GHz Intel Core i7

Python async module was introduced in 3.3.

I'm not trying to reinvent the wheel. I'm trying to build a good one.

Guido van Rossum

import asyncio

• Reference implementation for PEP-3156.
• Basic components for doing async i/o
• Works officially on Python >= 3.3

Goals

• Modern implementation of async i/o for Python
• Use yield from (PEP-380)
• Don't use anything that requires Python > 3.3
• Interoperabililty with other frameworks
• Unix and Windows support
• IPv4 and IPv6
• TCP, UCP and pipes
• Basic SSL
• Subporcess

Non goals

• Perfection
• Replacing current framwork
• Protocol implementations
• Replace http, smtplib, ...
• Make it work on Python < 3.3

Coroutine, Futures and Tasks

Coroutines

• generator function, can receive values
• decorated with @coroutine
  Future
• promise of a result or an error
  Task
• Future which runs a coroutine

Thread vs Coroutine

import time
import asyncio
import threading
import numpy as np


def fib(n):
    if n in (1, 2):
        return 1
    return fib(n - 1) + fib(n - 2)

@asyncio.coroutine
def co_fib(n):
    if n in (1, 2):
        return 1
    return fib(n - 1) + fib(n - 2)

def coroutine_test(n):
    tasks = [asyncio.ensure_future(co_fib(20)) for _ in range(n)]
    loop = asyncio.get_event_loop()
    loop.run_until_complete(asyncio.wait(tasks))

def thread_test(n):
    threads = [threading.Thread(target=fib, args=(20,)) for _ in range(n)]
    for thread in threads:
        thread.start()

    for thread in threads:
        thread.join()

def evaluate(n, m):
    r = []
    print("thread start")
    for _ in range(n):
        s = time.time()
        thread_test(m)
        process_time = time.time() - s
        r.append(process_time)
    print("thread average: ", np.average(r), " secs")
    print("thread distribution: ", np.var(r))

    r = []
    print("coroutine start")
    for _ in range(n):
        s = time.time()
        coroutine_test(m)
        process_time = time.time() - s
        r.append(process_time)
    print("coroutine average: ", np.average(r), " secs")
    print("coroutine distribution: ", np.var(r))


if __name__ == "__main__":
    evaluate(100, 1000)

Result

thread start
thread average:  2.26430218458  secs
thread distribution:  0.000516381104439
coroutine start
coroutine average:  2.17451543093  secs
coroutine distribution:  0.00377455202351

reference