gh-110012: Fix RuntimeWarning in test_socket

[sobolevn]

After:

Ran 483 tests in 27.474s

OK (skipped=245)

== Tests result: SUCCESS ==

1 test OK.

Total duration: 27.6 sec
Total tests: run=483 skipped=245
Total test files: run=1/1
Result: SUCCESS

I haven't checked if it also generates this warning on other platforms (different from macos sonoma), but in case it is not, I will just change quiet to True

• Issue: test_socket generates several warnings #110012

[AlexWaygood]

You mean "malformed" and "quiet", not "mailformed" and "quite" :-)

[sobolevn]

@AlexWaygood I am really into tpyos, thanks! :)

[AlexWaygood]

Yuo're wellcom!

[vstinner]

Does it catch them or ignore them? What is the behavior of quiet=False?

[sobolevn]

Yes

cpython/Lib/test/support/warnings_helper.py

Lines 152 to 188 in 8f324b7

	def _filterwarnings(filters, quiet=False):
	"""Catch the warnings, then check if all the expected
	warnings have been raised and re-raise unexpected warnings.
	If 'quiet' is True, only re-raise the unexpected warnings.
	"""
	# Clear the warning registry of the calling module
	# in order to re-raise the warnings.
	frame = sys._getframe(2)
	registry = frame.f_globals.get('__warningregistry__')
	if registry:
	registry.clear()
	with warnings.catch_warnings(record=True) as w:
	# Set filter "always" to record all warnings. Because
	# test_warnings swap the module, we need to look up in
	# the sys.modules dictionary.
	sys.modules['warnings'].simplefilter("always")
	yield WarningsRecorder(w)
	# Filter the recorded warnings
	reraise = list(w)
	missing = []
	for msg, cat in filters:
	seen = False
	for w in reraise[:]:
	warning = w.message
	# Filter out the matching messages
	if (re.match(msg, str(warning), re.I) and
	issubclass(warning.__class__, cat)):
	seen = True
	reraise.remove(w)
	if not seen and not quiet:
	# This filter caught nothing
	missing.append((msg, cat.__name__))
	if reraise:
	raise AssertionError("unhandled warning %s" % reraise[0])
	if missing:
	raise AssertionError("filter (%r, %s) did not catch any warning" %
	missing[0])

1. It catches warnings with warnings.catch_warnings(record=True)
2. Then it filters out ones that matched ("received malformed or improperly-truncated ancillary data", RuntimeWarning)
3. If there are any left, the test fails
4. quiet=True handles the cases where no warning was raised - just by ignoring the last check, if quiet=False we are required to have at least one matched warning

[serhiy-storchaka]

Is there a correlation between a warning and the emptiness of ancdata?

[vstinner]

"received malformed or improperly-truncated ancillary data" message comes from Python if get_cmsg_data_len() returns non-zero. Can you please check in which case your are?

static int 
get_cmsg_data_len(struct msghdr *msg, struct cmsghdr *cmsgh, size_t *data_len)
{
    size_t space, cmsg_data_len;

    if (!cmsg_min_space(msg, cmsgh, CMSG_LEN(0)) ||
        cmsgh->cmsg_len < CMSG_LEN(0))
        return -1; 
    cmsg_data_len = cmsgh->cmsg_len - CMSG_LEN(0);
    if (!get_cmsg_data_space(msg, cmsgh, &space))
        return -1;
    if (space >= cmsg_data_len) {
        *data_len = cmsg_data_len;
        return 0;
    }
    *data_len = space;
    return 1;
}   

[sobolevn]

Local debug shows that I face

printf("3");
*data_len = space;
return 1;

[sobolevn]

Hm, I agree that this does not look like a proper fix. There's something else going on inside, for some reason there's not enough space to write the data.

[serhiy-storchaka]

The code was added in 2011 (bpo-6560, #50809, 96fe56a). It does not emit a warning on Linux (at least I never see it). Were the warnings always on Windows, or did they appear in some version? Are they always expected in these tests, do they signal a problem? Shouldn't these tests fail if there was not a warning on Windows?

[sobolevn]

We should ask @ncoghlan for help, I guess :)

[serhiy-storchaka]

@sobolevn Could you test whether warnings were present in old Python versions (it was 3.2 or 3.3 I guess). Since most core developers do not use Windows, it might just slip past their notice.

[sobolevn]

I cannot build anything below 3.7 on mac sonora (m2) :(

3.7:

~/Desktop/cpython2  0f56adb8d7 ✗                                                          
» ./python.exe -m test test_socket                         
0:00:00 load avg: 5.09 Run tests sequentially
0:00:00 load avg: 5.09 [1/1] test_socket
/Users/sobolev/Desktop/cpython2/Lib/test/test_socket.py:2323: RuntimeWarning: received malformed or improperly-truncated ancillary data
  result = sock.recvmsg(bufsize, *args)
/Users/sobolev/Desktop/cpython2/Lib/test/test_socket.py:2414: RuntimeWarning: received malformed or improperly-truncated ancillary data
  result = sock.recvmsg_into([buf], *args)

== Tests result: SUCCESS ==

1 test OK.

Total duration: 24.0 sec
Tests result: SUCCESS

3.8:

~/Desktop/cpython2  3.8 ✔                                                                 
» ./python.exe -m test test_socket                         
0:00:00 load avg: 6.11 Run tests sequentially
0:00:00 load avg: 6.11 [1/1] test_socket
/Users/sobolev/Desktop/cpython2/Lib/test/test_socket.py:2438: RuntimeWarning: received malformed or improperly-truncated ancillary data
  result = sock.recvmsg(bufsize, *args)
/Users/sobolev/Desktop/cpython2/Lib/test/test_socket.py:2529: RuntimeWarning: received malformed or improperly-truncated ancillary data
  result = sock.recvmsg_into([buf], *args)

== Tests result: SUCCESS ==

1 test OK.

Total duration: 23.2 sec
Tests result: SUCCESS

So, this is not new at least.

[vstinner]

@serhiy-storchaka: @sobolevn is trying to make warnings quiet on macOS, not on Windows. I don't know if these warnings are emitted on other platforms than macOS.

[AlexWaygood]

python -We -m test test_socket passes for me on Windows on a reasonably fresh build of Python 3.13, FWIW

[sobolevn]

@vstinner they appear both on Win and Mac, see my first commit:

• ubuntu (failing): https://github.com/python/cpython/actions/runs/6335548712/job/17206922402
• windows passing: https://github.com/python/cpython/actions/runs/6335548712/job/17206922528

[vstinner]

@vstinner they appear both on Win and Mac, see my first commit

Oh ok.

[hugovk]

@AlexWaygood @serhiy-storchaka @vstinner Any further comments on this PR?

[serhiy-storchaka]

I do not think that simply suppressing warnings here is the right solution. What is the warning about? What is the end user supposed to do with it? Non-zero result of get_cmsg_data_len() should be either an error or ignored (note that there are two non-zero returned values). If such situation is common, then perhaps the code just does not handle all cases correctly.

[graingert]

What we can do is set the warning filter to default for these warnings, add a TODO comment and not close this issue

[AlexWaygood]

@AlexWaygood @serhiy-storchaka @vstinner Any further comments on this PR?

I only ever suggested some typo fixes on this one; it's well out of my area of expertise :-)

[graingert]

What we can do is set the warning filter to default for these warnings, add a TODO comment and not close this issue

I've now done this in the warnings as errors PR