`call_soon_threadsafe` callbacks from executor no longer run before `await run_in_executor()` returns in 3.14.3

[sairon]

Bug report

Bug description:

Python 3.14.3 introduced a behavior change of how run_in_executor interacts with call_soon_threadsafe. Previously, callbacks scheduled via call_soon_threadsafe from within an executor job were guaranteed to run before await run_in_executor() returned.

This seems to be a direct cause of changes from gh-141696, as monkey-patching the asyncio.futures._chain_future reverts to the old behavior.

Reproducer:

import asyncio
import sys
import threading
from concurrent.futures import ThreadPoolExecutor

print(f"Python version: {sys.version}")


async def test_timing():
    loop = asyncio.get_running_loop()
    executor = ThreadPoolExecutor(max_workers=1)
    results = []


    def final_callback():
        results.append("final_callback")

    def intermediate_callback():
        results.append("intermediate_callback")
        loop.call_soon(final_callback)

    def thread_work():
        results.append("thread_work")
        loop.call_soon_threadsafe(intermediate_callback)

    await loop.run_in_executor(executor, thread_work)
    results.append("executor_done")

    await asyncio.sleep(0)
    results.append("after_sleep_0")

    executor.shutdown(wait=False)

    final_ran_before_sleep = (
        "final_callback" in results
        and results.index("final_callback") < results.index("after_sleep_0")
    )
    return final_ran_before_sleep, results


async def main():
    for i in range(20):
        success, results = await test_timing()
        print(f"  Iteration {i+1} {'PASS' if success else 'FAIL'}: {results}")


asyncio.run(main())

3.14.2:

Python version: 3.14.2 (main, Feb  5 2026, 13:08:22) [GCC 15.2.1 20260103]
  Iteration 1 PASS: ['thread_work', 'intermediate_callback', 'final_callback', 'executor_done', 'after_sleep_0']
  (rest is the same)

3.14.3:

Python version: 3.14.3 (main, Feb  5 2026, 12:15:05) [GCC 15.2.1 20260103]
  Iteration 1 PASS: ['thread_work', 'intermediate_callback', 'final_callback', 'executor_done', 'after_sleep_0']
  Iteration 2 FAIL: ['thread_work', 'executor_done', 'intermediate_callback', 'after_sleep_0']
  Iteration 3 FAIL: ['thread_work', 'executor_done', 'intermediate_callback', 'after_sleep_0']
  Iteration 4 FAIL: ['thread_work', 'executor_done', 'intermediate_callback', 'after_sleep_0']
  Iteration 5 PASS: ['thread_work', 'intermediate_callback', 'final_callback', 'executor_done', 'after_sleep_0']
  Iteration 6 FAIL: ['thread_work', 'executor_done', 'intermediate_callback', 'after_sleep_0']
  Iteration 7 PASS: ['thread_work', 'intermediate_callback', 'final_callback', 'executor_done', 'after_sleep_0']
  Iteration 8 PASS: ['thread_work', 'intermediate_callback', 'final_callback', 'executor_done', 'after_sleep_0']
  Iteration 9 FAIL: ['thread_work', 'executor_done', 'intermediate_callback', 'after_sleep_0']
  Iteration 10 PASS: ['thread_work', 'intermediate_callback', 'final_callback', 'executor_done', 'after_sleep_0']
  Iteration 11 FAIL: ['thread_work', 'executor_done', 'intermediate_callback', 'after_sleep_0']
  Iteration 12 PASS: ['thread_work', 'intermediate_callback', 'final_callback', 'executor_done', 'after_sleep_0']
  Iteration 13 FAIL: ['thread_work', 'executor_done', 'intermediate_callback', 'after_sleep_0']
  Iteration 14 FAIL: ['thread_work', 'executor_done', 'intermediate_callback', 'after_sleep_0']
  Iteration 15 FAIL: ['thread_work', 'executor_done', 'intermediate_callback', 'after_sleep_0']
  Iteration 16 PASS: ['thread_work', 'intermediate_callback', 'final_callback', 'executor_done', 'after_sleep_0']
  Iteration 17 FAIL: ['thread_work', 'executor_done', 'intermediate_callback', 'after_sleep_0']
  Iteration 18 PASS: ['thread_work', 'intermediate_callback', 'final_callback', 'executor_done', 'after_sleep_0']
  Iteration 19 FAIL: ['thread_work', 'executor_done', 'intermediate_callback', 'after_sleep_0']
  Iteration 20 FAIL: ['thread_work', 'executor_done', 'intermediate_callback', 'after_sleep_0']

3.14.3, with asyncio.sleep(1e-15), all callbacks run but order changes:

Python version: 3.14.3 (main, Feb  5 2026, 12:15:05) [GCC 15.2.1 20260103]
  Iteration 1 PASS: ['thread_work', 'intermediate_callback', 'final_callback', 'executor_done', 'after_sleep_0']
  Iteration 2 PASS: ['thread_work', 'intermediate_callback', 'final_callback', 'executor_done', 'after_sleep_0']
  Iteration 3 PASS: ['thread_work', 'executor_done', 'intermediate_callback', 'final_callback', 'after_sleep_0']
  Iteration 4 PASS: ['thread_work', 'intermediate_callback', 'final_callback', 'executor_done', 'after_sleep_0']
  Iteration 5 PASS: ['thread_work', 'intermediate_callback', 'final_callback', 'executor_done', 'after_sleep_0']
  Iteration 6 PASS: ['thread_work', 'executor_done', 'intermediate_callback', 'final_callback', 'after_sleep_0']
  Iteration 7 PASS: ['thread_work', 'executor_done', 'intermediate_callback', 'final_callback', 'after_sleep_0']
  Iteration 8 PASS: ['thread_work', 'executor_done', 'intermediate_callback', 'final_callback', 'after_sleep_0']
  Iteration 9 PASS: ['thread_work', 'executor_done', 'intermediate_callback', 'final_callback', 'after_sleep_0']
  Iteration 10 PASS: ['thread_work', 'intermediate_callback', 'final_callback', 'executor_done', 'after_sleep_0']
  Iteration 11 PASS: ['thread_work', 'executor_done', 'intermediate_callback', 'final_callback', 'after_sleep_0']
  Iteration 12 PASS: ['thread_work', 'executor_done', 'intermediate_callback', 'final_callback', 'after_sleep_0']
  Iteration 13 PASS: ['thread_work', 'executor_done', 'intermediate_callback', 'final_callback', 'after_sleep_0']
  Iteration 14 PASS: ['thread_work', 'executor_done', 'intermediate_callback', 'final_callback', 'after_sleep_0']
  Iteration 15 PASS: ['thread_work', 'executor_done', 'intermediate_callback', 'final_callback', 'after_sleep_0']
  Iteration 16 PASS: ['thread_work', 'executor_done', 'intermediate_callback', 'final_callback', 'after_sleep_0']
  Iteration 17 PASS: ['thread_work', 'executor_done', 'intermediate_callback', 'final_callback', 'after_sleep_0']
  Iteration 18 PASS: ['thread_work', 'executor_done', 'intermediate_callback', 'final_callback', 'after_sleep_0']
  Iteration 19 PASS: ['thread_work', 'intermediate_callback', 'final_callback', 'executor_done', 'after_sleep_0']
  Iteration 20 PASS: ['thread_work', 'executor_done', 'intermediate_callback', 'final_callback', 'after_sleep_0']

CPython versions tested on:

3.14

Operating systems tested on:

Linux

[kumaraditya303]

Previously, callbacks scheduled via call_soon_threadsafe from within an executor job were guaranteed to run before await run_in_executor() returned.

Hmm, AFAIK this was never guaranteed, it just happened to work like that on previous versions. If you change the call_soon to call_later with small delay such as 0.001 in intermediate_callback then you would see this behaviour on older versions as well.

[kumaraditya303]

I'll close this as this is being fixed on home-assistant side.