asyncio should use signal.set_wakeup_fd on Windows

[njsmith]

BPO	33192
Nosy	@vstinner, @giampaolo, @njsmith, @asvetlov, @1st1

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2018-03-31.22:44:55.378>
labels = ['3.7', '3.8', 'type-bug', 'library']
title = 'asyncio should use signal.set_wakeup_fd on Windows'
updated_at = <Date 2018-04-01.23:35:09.554>
user = 'https://github.com/njsmith'

bugs.python.org fields:

activity = <Date 2018-04-01.23:35:09.554>
actor = 'pitrou'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2018-03-31.22:44:55.378>
creator = 'njs'
dependencies = []
files = []
hgrepos = []
issue_num = 33192
keywords = []
message_count = 1.0
messages = ['314749']
nosy_count = 5.0
nosy_names = ['vstinner', 'giampaolo.rodola', 'njs', 'asvetlov', 'yselivanov']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue33192'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[njsmith]

I thought there was already a bug for this, but it came up in conversation again and I can't find one, so, here you go...

It looks like originally there was this bug for making control-C wake up the asyncio event loop in Windows: python/asyncio#191

This required some changes to signal.set_wakeup_fd to work on Windows, which was done in bpo-22018.

But I guess the last step got lost in the shuffle: right now signal.set_wakeup_fd works fine on Windows, but asyncio doesn't actually use it. This means that on Windows you can't wake up this program using control-C:

>> import asyncio
>> asyncio.run(asyncio.sleep(100000000))

Both of the Windows event loops should register a wakeup socket with signal.set_wakeup_fd, and arrange for the loop to wake up when data arrives on that socket, and read from it until it's empty again. (And once the loop is awake, Python's normal control-C handling will kick in.) That will make control-C on Windows work similarly to how it does on Unix.

[gvanrossum]

It looks like BaseProactorEventLoop (and hence ProactorEventLoop) does use signal.set_wakeup_fd(), but _WindowsSelectorEventLoop does not. So there's work needed on that.

[vstinner]

Is _WindowsSelectorEventLoop still relevant in Python 3.12, since ProactorEventLoop is now feature complete? (CTRL+C, signals, subprocesses, TLS sockets, etc.)

[gvanrossum]

Is _WindowsSelectorEventLoop still relevant in Python 3.12, since ProactorEventLoop is now feature complete? (CTRL+C, signals, subprocesses, TLS sockets, etc.)

IIUC the (sockets) selector event loop is still needed if you want to wait for a socket fd that was created outside of asyncio. Those use integers but they are not in the same namespace as file fds: both are ints, but the set of C-level APIs that accept one or the other are distinct, so you can't pass a socket fd to read(), you must use recv().

If I am wrong we should just start the deprecation process for _WindowsSelectorEventLoop, and then we can close this.

[vstinner]

IIUC the (sockets) selector event loop is still needed if you want to wait for a socket fd that was created outside of asyncio

Ah right, ProactorEventLoop lacks add_reader() and add_writer() methods: https://docs.python.org/dev/library/asyncio-platforms.html#asyncio-platform-support

I don't know if it's an issue in practice.

ProactorEventLoop has methods like sock_recv() which take a Python socket object. For example, sock_recv(sock, nbytes) calls ov.WSARecv(sock.fileno(), nbytes, flags).

[gvanrossum]

I don't use asyncio on Windows (or anywhere else, but don't tell anyone, I'm trying to pose as an expert :-), so fine to start the deprecation process. Doesn't it require a PEP nowadays?

[vstinner]

Doesn't it require a PEP nowadays?

To deprecate an asyncio feature, check if anyone uses it. If the usage is limited, PEP 387 gives the process.

I ran a quick code search:

$ ./search_pypi_top.py PYPI-2020-09-01/ 'add_reader|add_writer' -o asyncio -q
(...)
Found 461 matching lines in 51 projects

It's used in multiple projects. Some examples:

aiohttp-3.8.1.tar.gz: aiohttp-3.8.1/examples/client_ws.py: loop.add_reader(sys.stdin.fileno(), stdin_callback)
prompt_toolkit-3.0.30.tar.gz: prompt_toolkit-3.0.30/src/prompt_toolkit/input/vt100.py: loop.add_reader(fd, callback_wrapper)
prompt_toolkit-3.0.30.tar.gz: prompt_toolkit-3.0.30/src/prompt_toolkit/input/vt100.py: loop.add_reader(fd, previous)

In these examples, the argument is a pipe, not a socket.

But I don't know if such code is used on Windows, or only on Unix-like operating systems which are based on file descriptors (select, epoll, kqueue, etc.).

[vstinner]

Related issue: issue #71019 "ProactorEventLoop doesn't support stdin/stdout nor files with connect_read_pipe/connect_write_pipe"

[gvanrossum]

Someone would have to do more research. :-(