bpo-36801: Fix waiting in StreamWriter.drain for closing SSL transport

[asvetlov]

https://bugs.python.org/issue36801

[asvetlov]

The backport to 3.7 is desired.
The problem not in SSL connections only.
Working on the patch I've found that await subproc.stdin.wait_closed() is broken on 3.7
Probably nobody executes this call though, nobody blames.

[1st1]

Do we want to raise an error from drain()? Maybe it should just return? What's the point of this error?

[asvetlov]

That's how protocol._drain_helper() works now.
The future is set to exception only if the connection is closed with a failure.
But await writer.drain() raises ConnectionResetError.
I believe this is good: await writer.write(b'data') should fail loudly if the socket is closed.

[asvetlov]

Maybe the code could be simplified by setting ConnectionResetError by connection_lost() callback handler but I'd like to keep the PR small.
Future improvement worth another pull request.

[1st1]

OK, got it.

[miss-islington]

Thanks @asvetlov for the PR 🌮🎉.. I'm working now to backport this PR to: 3.7.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

[bedevere-bot]

GH-13176 is a backport of this pull request to the 3.7 branch.

[nullstd]

Hi @asvetlov , I ran into a BrokenPipeError error when testing the following code:

#!/usr/bin/env python

import sys
import os
import asyncio





async def main_task():

    print(sys.version)


    xxx = "x" * 196428
    incoming_original_data = xxx.encode()

    print(f"stdin len: {len(incoming_original_data)}")

    proc = await asyncio.create_subprocess_shell(
        f"sleep 2 && xxd -l 1",
        stdin=asyncio.subprocess.PIPE,
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE)
    
    await proc.communicate(input = incoming_original_data)

    print("subprocess ended")

def main():

    opts = asyncio.run(main_task())

    print(f"main END")

    return opts




if __name__ == '__main__':

    main()

    

Note the subprocess ends very soon, and it seems _stdin_closed isn't properly handled because I see the following output:

3.11.3 (main, Apr  7 2023, 19:25:52) [Clang 14.0.0 (clang-1400.0.29.202)]
stdin len: 196428
subprocess ended
main END
Future exception was never retrieved
future: <Future finished exception=BrokenPipeError(32, 'Broken pipe')>
Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/subprocess.py", line 152, in _feed_stdin
    await self.stdin.drain()
  File "/usr/local/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/streams.py", line 378, in drain
    await self._protocol._drain_helper()
  File "/usr/local/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/streams.py", line 173, in _drain_helper
    await waiter
  File "/usr/local/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/unix_events.py", line 709, in _write_ready
    n = os.write(self._fileno, self._buffer)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
BrokenPipeError: [Errno 32] Broken pipe

[nullstd]

BTW, here are some links I think might be helpful:

https://stackoverflow.com/questions/14207708/ioerror-errno-32-broken-pipe-when-piping-prog-py-othercmd

https://stackoverflow.com/questions/23688492/oserror-errno-22-invalid-argument-in-subprocess#28020096

[nullstd]

cc @1st1

[gpshead]

please do not post questions and cc people on long closed/merged PRs. Please use the discuss.python.org Help category - and only when it seems an actual CPython issue has been found, file a new issue in the issue tracker.