0
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 1 year has passed since last update.

非同期並列実行版 map 関数(Python)

Last updated at Posted at 2023-03-22

こんにちは。
非同期並列実行版 map関数を、asyncio を利用して作りました(下記ソース内の map_async)。

動作確認

最大並列数(limit)を 2 へ指定し、実行時間が 1/2 となることを確かめました。
output.gif

ソース
test_asyncio.py
#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import asyncio

def map_async(func, data, limit):
	loop = asyncio.get_event_loop()
	async def run_all():	
		sem = asyncio.Semaphore(limit)
		async def run_each(d):
			async with sem:
				return await loop.run_in_executor(None, func, d)
		return await asyncio.gather(*[run_each(d) for d in data])
	return loop.run_until_complete(run_all())  # list

def map_async_test(data, limit):
	def func_blocking(x):
		import time
		print(x)
		time.sleep(x)
		return x
	
	print(map_async(func_blocking, data, limit))

# synchronous
# def map_sync_test(data):
# 	print(list(map(func_blocking, data)))

def main():
	import argparse
	parser = argparse.ArgumentParser()
	parser.add_argument('--limit', type=int, default=2)
	parser.add_argument('data', type=float, nargs='*')
	args = parser.parse_args()
	map_async_test(args.data, args.limit)

if __name__ == '__main__':
	main()
0
1
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
0
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?