Contextvars and Thread local
Here in the post, I will share some examples about the contextvars (new in Python 3.7) and thread local.
Default Value
In the module level, use ContextVar.set or directly setattr for a thread local variable, won't successfully set a default value, the value set won't take effect in another thread.
To ensure a default value, for contextvars
1import contextvars
2context_var = contextvars.ContextVar("context_var", default=0)
for thread local, a sub class of thread.local need to be declared
1class ValueLocal(threading.local):
2 def __init__(self, value):
3 self.value = value
4
5thread_local = ValueLocal(0)
Behavior with coroutines
If used in the multi threading projects, contextvars and thread local would behave pretty much the same. But with coroutines, using thread local is dangerous and contextvars is the aid.
The example below will show you the different behaviors for contextvars and thread local. First let's define some base variables and functions for examples for both threads and coroutines.
1import time
2import contextvars
3import threading
4import asyncio
5import random
6import functools
7
8context_var = contextvars.ContextVar("test", default=['root'])
9
10# Set default values for thread.local in new threads
11class ValueLocal(threading.local):
12 def __init__(self, value):
13 self.value = value
14
15thread_local = ValueLocal(['root'])
16
17outputs = {
18 "contextvars": "",
19 "thread.local": ""
20}
21
22def dump_context(x):
23 '''
24 Dump the contextvars and thread.local to outputs.
25 '''
26 outputs["contextvars"] += "%s\t%s\n" % (x, '\t'.join(context_var.get()))
27 outputs["thread.local"] += "%s\t%s\n" % (x, '\t'.join(thread_local.value))
28
29def async_function_context(f):
30 '''
31 A wrapper for async/await functions.
32 '''
33 @functools.wraps(f)
34 async def wrapper(*sub, **kwargs):
35 context_var.set(context_var.get()[:] + [f.__name__])
36 thread_local.value = thread_local.value[:] + [f.__name__]
37 r = await f(*sub, **kwargs)
38 context_var.set(context_var.get()[:-1])
39 thread_local.value = thread_local.value[:-1]
40 return r
41 return wrapper
42
43def function_context(f):
44 '''
45 A wrapper of normal functions.
46 '''
47 @functools.wraps(f)
48 def wrapper(*sub, **kwargs):
49 context_var.set(context_var.get()[:] + [f.__name__])
50 thread_local.value = thread_local.value[:] + [f.__name__]
51 r = f(*sub, **kwargs)
52 context_var.set(context_var.get()[:-1])
53 thread_local.value = thread_local.value[:-1]
54 return r
55 return wrapper
The first example is for multi threads:
1from base import *
2
3@function_context
4def foo(x):
5 time.sleep(random.random())
6 dump_context(x)
7 bar(x)
8 dump_context(x)
9
10@function_context
11def bar(x):
12 time.sleep(random.random())
13 dump_context(x)
14 baz(x)
15 dump_context(x)
16
17@function_context
18def baz(x):
19 time.sleep(random.random())
20 dump_context(x)
21
22def main():
23 threads = [threading.Thread(target=foo, args=(i,)) for i in [1, 2, 3]]
24 for t in threads:
25 t.start()
26 for t in threads:
27 t.join()
28 for k, v in outputs.items():
29 print("***%s***\n%s" % (k, v))
30
31main()
The output results of the above example would be:
1***contextvars***
22 root foo
31 root foo
43 root foo
52 root foo bar
61 root foo bar
73 root foo bar
83 root foo bar baz
93 root foo bar
103 root foo
112 root foo bar baz
122 root foo bar
132 root foo
141 root foo bar baz
151 root foo bar
161 root foo
17
18***thread.local***
192 root foo
201 root foo
213 root foo
222 root foo bar
231 root foo bar
243 root foo bar
253 root foo bar baz
263 root foo bar
273 root foo
282 root foo bar baz
292 root foo bar
302 root foo
311 root foo bar baz
321 root foo bar
331 root foo
Results from both contextvars and thread local works are as we expected.
And the second example are for coroutines:
1from base import *
2
3@async_function_context
4async def foo(x):
5 await asyncio.sleep(random.random())
6 dump_context(x)
7 await bar(x)
8 dump_context(x)
9
10@async_function_context
11async def bar(x):
12 await asyncio.sleep(random.random())
13 dump_context(x)
14 await baz(x)
15 dump_context(x)
16
17@async_function_context
18async def baz(x):
19 await asyncio.sleep(random.random())
20 dump_context(x)
21
22async def main():
23 await asyncio.gather(
24 foo(1),
25 foo(2),
26 foo(3)
27 )
28 for k, v in outputs.items():
29 print("***%s***\n%s" % (k, v))
30
31asyncio.run(main())
The results of the above example would be like:
1***contextvars***
22 root foo
31 root foo
41 root foo bar
53 root foo
61 root foo bar baz
71 root foo bar
81 root foo
92 root foo bar
103 root foo bar
112 root foo bar baz
122 root foo bar
132 root foo
143 root foo bar baz
153 root foo bar
163 root foo
17
18***thread.local***
192 root foo foo foo
201 root foo foo foo bar
211 root foo foo foo bar bar
223 root foo foo foo bar bar baz
231 root foo foo foo bar bar baz bar
241 root foo foo foo bar bar baz
251 root foo foo foo bar bar
262 root foo foo foo bar
273 root foo foo foo bar baz
282 root foo foo foo bar baz baz
292 root foo foo foo bar baz
302 root foo foo foo bar
313 root foo foo foo
323 root foo foo
333 root foo
This time, the results from contextvars are still as expected. But results from thread local is messed, because different coroutines shares the same thread, breaks the safety of thread local mechanism.