_thread context switching.

[peterhinch]

Assume two threads running on a single core. Can context switches only occur at Python instruction boundaries?

Put it another way, if a single Python instruction on thread A updates a shared variable of arbitrary complexity, is it guaranteed that thread B will see a valid variable? Or is it necessary to use a lock?

[dlech]

I think it depends on the port and if GIL is enabled or not. For example, on the unix port with GIL disabled, a context switch can happen at any time, but on the stm32 port with GIL and threading enabled, the context switch is only going to happen between bytecodes (when MP_THREAD_GIL_EXIT is called) and in long blocking operations that call MP_THREAD_GIL_EXIT, MICROPY_EVENT_POLL_HOOK or MICROPY_THREAD_YIELD.

[jimmo]

I guess there are four main cases: STM32, ESP32, RP2040, Unix, but really these fall into two categories:

• STM32 & ESP32 (pre-emptive, single-core, GIL)
• RP2040 (multi-core, non-pre-emptive, no GIL) & Unix (multi-core, pre-emptive, no GIL)

Neither of these provide a guarantee that context switches occur at Python instruction boundaries. (I don't think what @dlech said is quite right -- on STM32 there are three mechanisms that can trigger a context switch: pre-emptive (via systick expiring the time slice), lock acquisition, and the various event poll hook and VM hooks. His answer only applies to the last two. The systick-triggered switches can occur at any time).

However, the GIL effectively provides this guarantee. So on ESP32 and STM32, the guarantee that @peterhinch is after does hold -- Python instructions are effectively atomic on STM32 and ESP32.

(What this means is that a thread can be mid-bytecode-op and holding the GIL, get pre-empted at the end of its 4ms time slice, and then the next thread immediately tries to acquire the GIL but immediately yields back to the first thread in lock acquisition. I've actually never looked into the details of threading much before, and my first thought is that I'm not sure how this allows fairness, but I suspect that the event poll hook yield as well as the "vm divisor GIL bounce" largely address this).