Exactly-Once Initialization in Asynchronous Python

栏目: IT技术 · 发布时间: 4年前

内容简介:A common situation inThe naive “solution” would be to track the initialization state in a variable:The reasoning for

A common situation in asyncio Python programs is asynchronous initialization. Some resource must be initialized exactly once before it can be used, but the initialization itself is asynchronous — such as an asyncpg database. Let’s talk about a couple of solutions.

The naive “solution” would be to track the initialization state in a variable:

initialized = False

async def one_time_setup():
    "Do not call more than once!"
    ...

async def maybe_initialize():
    global initialized
    if not initialized:
        await one_time_setup()
        initialized = True

The reasoning for initialized is the expectation of calling the function more than once. However, if it might be called from concurrent tasks there’s a race condition . If the second caller arrives while the first is awaiting one_time_setup() , the function will be called a second time.

Switching the order of the call and the assignment won’t help:

async def maybe_initialize():
    global initialized
    if not initialized:
        initialized = True
        await one_time_setup()

Since asyncio is cooperative, the first caller doesn’t give up control until to other tasks until the await , meaning one_time_setup() will never be called twice. However, the second caller may return before one_time_setup() has completed. What we want is for one_time_setup() to be called exactly once, but for no caller to return until it has returned.

Mutual exclusion

My first thought was to use a mutex lock . This will protect the variable and prevent followup callers from progressing too soon. Tasks arriving while one_time_setup() is still running will block on the lock.

initialized = False
initialized_lock = asyncio.Lock()

async def maybe_initialize():
    global initialized
    async with initialized_lock:
        if not initialized:
            await one_time_setup()
            initialized = True

Unfortunately this has a serious downside: asyncio locks are associated with the loop where they were created . Since the lock variable is global, maybe_initialize() can only be called from the same loop that loaded the module. asyncio.run() creates a new loop so it’s incompatible.

# create a loop: always an error
asyncio.run(maybe_initialize())

# reuse the loop: maybe an error
loop = asyncio.get_event_loop()
loop.run_until_complete((maybe_initialize()))

(IMHO, it was a mistake for the asyncio API to include explicit loop objects. It’s a low-level concept that unavoidably leaks through most high-level abstractions.)

A workaround is to create the lock lazily. Thank goodness creating a lock isn’t itself asynchronous!

initialized = False
initialized_lock = None

async def maybe_initialize():
    global initialized, initialized_lock
    if not initialized_lock:
        initialized_lock = asyncio.Lock()
    async with initialized_lock:
        if not initialized:
            await one_time_setup()
            initialized = True

This is better, but maybe_initialize() can still only ever be called from a single loop.

asyncio.run(maybe_initialize()) # ok
asyncio.run(maybe_initialize()) # error!

Once

The pthreads API provides pthread_once to solve this problem. C++11 has similarly has std::call_once . We can build something similar using a future-like object.

future = None

async def maybe_initialize():
    if not future:
        future = asyncio.create_task(one_time_setup())
    await future

Awaiting a coroutine more than once is an error, but tasks are future-like objects and can be awaited more than once. At least on CPython, they can also be awaited in other loops! So not only is this simpler, it also solves the loop problem!

asyncio.run(maybe_initialize()) # ok
asyncio.run(maybe_initialize()) # still ok

This can be tidied up nicely in a @once decorator:

def once(func):
    future = None
    async def once_wrapper(*args, **kwargs):
        nonlocal future
        if not future:
            future = asyncio.create_task(func(*args, **kwargs))
        return await future
    return once_wrapper

No more need for maybe_initialize() , just decorate the original one_time_setup() :

@once
async def one_time_setup():
    ...

以上所述就是小编给大家介绍的《Exactly-Once Initialization in Asynchronous Python》,希望对大家有所帮助,如果大家有任何疑问请给我留言,小编会及时回复大家的。在此也非常感谢大家对 码农网 的支持!

查看所有标签

猜你喜欢:

本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们

让大象飞

让大象飞

[美] 史蒂文·霍夫曼 / 周海云、陈耿宣 / 中信出版社 / 2017-3 / 69.00

这是一本为中国创业者量身定做的创业指南,将帮助创业者理解创新的基本方法、模式和硅谷的创业理念。作者霍夫曼频繁地穿梭于中美两地,与不同的创业者、投资人、政府负责人进行对话,积累了大量的来自中国创业者的第一手经验。在这本书里,从创业团队的人员配备到创业融资的成败再到团队的高效管理,从创业者的心理素质到创业者的独到眼光再到企业赖以生存的根本,360度无死角地呈现了一家公司从初创到惊艳到立足再到稳定的全过......一起来看看 《让大象飞》 这本书的介绍吧!

HTML 压缩/解压工具
HTML 压缩/解压工具

在线压缩/解压 HTML 代码

RGB HSV 转换
RGB HSV 转换

RGB HSV 互转工具

HEX CMYK 转换工具
HEX CMYK 转换工具

HEX CMYK 互转工具