Merge branch 'master' into justin39/aiter-c

* master: (129 commits)
  bpo-43452: Micro-optimizations to PyType_Lookup (pythonGH-24804)
  bpo-43517: Fix false positive in detection of circular imports (python#24895)
  bpo-43494: Make some minor changes to lnotab notes (pythonGH-24861)
  Mention that code.co_lnotab is deprecated in what's new for 3.10. (python#24902)
  bpo-43244: Remove symtable.h header file (pythonGH-24910)
  bpo-43466: Add --with-openssl-rpath configure option (pythonGH-24820)
  Fix a typo in c-analyzer (pythonGH-24468)
  bpo-41561: Add workaround for Ubuntu's custom security level (pythonGH-24915)
  bpo-43521: Allow ast.unparse with empty sets and NaN (pythonGH-24897)
  bpo-43244: Remove the PyAST_Validate() function (pythonGH-24911)
  bpo-43541: Fix PyEval_EvalCodeEx() regression (pythonGH-24918)
  bpo-43244: Fix test_peg_generators on Windows (pythonGH-24913)
  bpo-39342: Expose X509_V_FLAG_ALLOW_PROXY_CERTS in ssl module (pythonGH-18011)
  bpo-43244: Fix test_peg_generator for PyAST_Validate() (pythonGH-24912)
  bpo-42128: Add 'missing :' syntax error message to match statements (pythonGH-24733)
  bpo-43244: Add pycore_ast.h header file (pythonGH-24908)
  bpo-43244: Rename pycore_ast.h to pycore_ast_state.h (pythonGH-24907)
  Remove unnecessary imports in the grammar parser (pythonGH-24904)
  bpo-35883: Py_DecodeLocale() escapes invalid Unicode characters (pythonGH-24843)
  Add PEP 626 to what's new in 3.10. (python#24892)
  ...

Author: jab

.github/workflows/build.yml

@@ -127,7 +127,7 @@ jobs:
 
   build_ubuntu:
     name: 'Ubuntu'
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-20.04
     needs: check_source
     if: needs.check_source.outputs.run_tests == 'true'
     env:

Doc/c-api/exceptions.rst

@@ -505,29 +505,73 @@ Signal Handling
       single: SIGINT
       single: KeyboardInterrupt (built-in exception)
 
-   This function interacts with Python's signal handling.  It checks whether a
-   signal has been sent to the processes and if so, invokes the corresponding
-   signal handler.  If the :mod:`signal` module is supported, this can invoke a
-   signal handler written in Python.  In all cases, the default effect for
-   :const:`SIGINT` is to raise the  :exc:`KeyboardInterrupt` exception.  If an
-   exception is raised the error indicator is set and the function returns ``-1``;
-   otherwise the function returns ``0``.  The error indicator may or may not be
-   cleared if it was previously set.
+   This function interacts with Python's signal handling.
+
+   If the function is called from the main thread and under the main Python
+   interpreter, it checks whether a signal has been sent to the processes
+   and if so, invokes the corresponding signal handler.  If the :mod:`signal`
+   module is supported, this can invoke a signal handler written in Python.
+
+   The function attemps to handle all pending signals, and then returns ``0``.
+   However, if a Python signal handler raises an exception, the error
+   indicator is set and the function returns ``-1`` immediately (such that
+   other pending signals may not have been handled yet: they will be on the
+   next :c:func:`PyErr_CheckSignals()` invocation).
+
+   If the function is called from a non-main thread, or under a non-main
+   Python interpreter, it does nothing and returns ``0``.
+
+   This function can be called by long-running C code that wants to
+   be interruptible by user requests (such as by pressing Ctrl-C).
+
+   .. note::
+      The default Python signal handler for :const:`SIGINT` raises the
+      :exc:`KeyboardInterrupt` exception.
 
 
 .. c:function:: void PyErr_SetInterrupt()
 
    .. index::
+      module: signal
       single: SIGINT
       single: KeyboardInterrupt (built-in exception)
 
-   Simulate the effect of a :const:`SIGINT` signal arriving. The next time
+   Simulate the effect of a :const:`SIGINT` signal arriving.
+   This is equivalent to ``PyErr_SetInterruptEx(SIGINT)``.
+
+   .. note::
+      This function is async-signal-safe.  It can be called without
+      the :term:`GIL` and from a C signal handler.
+
+
+.. c:function:: int PyErr_SetInterruptEx(int signum)
+
+   .. index::
+      module: signal
+      single: KeyboardInterrupt (built-in exception)
+
+   Simulate the effect of a signal arriving. The next time
    :c:func:`PyErr_CheckSignals` is called,  the Python signal handler for
-   :const:`SIGINT` will be called.
+   the given signal number will be called.
+
+   This function can be called by C code that sets up its own signal handling
+   and wants Python signal handlers to be invoked as expected when an
+   interruption is requested (for example when the user presses Ctrl-C
+   to interrupt an operation).
+
+   If the given signal isn't handled by Python (it was set to
+   :data:`signal.SIG_DFL` or :data:`signal.SIG_IGN`), it will be ignored.
+
+   If *signum* is outside of the allowed range of signal numbers, ``-1``
+   is returned.  Otherwise, ``0`` is returned.  The error indicator is
+   never changed by this function.
+
+   .. note::
+      This function is async-signal-safe.  It can be called without
+      the :term:`GIL` and from a C signal handler.
+
+   .. versionadded:: 3.10
 
-   If :const:`SIGINT` isn't handled by Python (it was set to
-   :data:`signal.SIG_DFL` or :data:`signal.SIG_IGN`), this function does
-   nothing.
 
 .. c:function:: int PySignal_SetWakeupFd(int fd)
 

Doc/c-api/memory.rst

@@ -555,7 +555,7 @@ Customize pymalloc Arena Allocator
    +--------------------------------------------------+---------------------------------------+
    | ``void* alloc(void *ctx, size_t size)``          | allocate an arena of size bytes       |
    +--------------------------------------------------+---------------------------------------+
-   | ``void free(void *ctx, size_t size, void *ptr)`` | free an arena                         |
+   | ``void free(void *ctx, void *ptr, size_t size)`` | free an arena                         |
    +--------------------------------------------------+---------------------------------------+
 
 .. c:function:: void PyObject_GetArenaAllocator(PyObjectArenaAllocator *allocator)

Doc/c-api/unicode.rst

@@ -725,6 +725,10 @@ Extension modules can continue using them, as they will not be removed in Python
    :c:type:`Py_UNICODE` buffer of the given *size* by ASCII digits 0--9
    according to their decimal value.  Return ``NULL`` if an exception occurs.
 
+   .. deprecated-removed:: 3.3 3.11
+      Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
+      :c:func:`Py_UNICODE_TODECIMAL`.
+
 
 .. c:function:: Py_UNICODE* PyUnicode_AsUnicodeAndSize(PyObject *unicode, Py_ssize_t *size)
 
@@ -1043,7 +1047,7 @@ These are the generic codec APIs:
    to be used is looked up using the Python codec registry.  Return ``NULL`` if an
    exception was raised by the codec.
 
-   .. deprecated-removed:: 3.3 4.0
+   .. deprecated-removed:: 3.3 3.11
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
       :c:func:`PyUnicode_AsEncodedString`.
 
@@ -1116,7 +1120,7 @@ These are the UTF-8 codec APIs:
    return a Python bytes object.  Return ``NULL`` if an exception was raised by
    the codec.
 
-   .. deprecated-removed:: 3.3 4.0
+   .. deprecated-removed:: 3.3 3.11
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
       :c:func:`PyUnicode_AsUTF8String`, :c:func:`PyUnicode_AsUTF8AndSize` or
       :c:func:`PyUnicode_AsEncodedString`.
@@ -1190,7 +1194,7 @@ These are the UTF-32 codec APIs:
 
    Return ``NULL`` if an exception was raised by the codec.
 
-   .. deprecated-removed:: 3.3 4.0
+   .. deprecated-removed:: 3.3 3.11
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
       :c:func:`PyUnicode_AsUTF32String` or :c:func:`PyUnicode_AsEncodedString`.
 
@@ -1265,7 +1269,7 @@ These are the UTF-16 codec APIs:
 
    Return ``NULL`` if an exception was raised by the codec.
 
-   .. deprecated-removed:: 3.3 4.0
+   .. deprecated-removed:: 3.3 3.11
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
       :c:func:`PyUnicode_AsUTF16String` or :c:func:`PyUnicode_AsEncodedString`.
 
@@ -1303,7 +1307,7 @@ These are the UTF-7 codec APIs:
    nonzero, whitespace will be encoded in base-64.  Both are set to zero for the
    Python "utf-7" codec.
 
-   .. deprecated-removed:: 3.3 4.0
+   .. deprecated-removed:: 3.3 3.11
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
       :c:func:`PyUnicode_AsEncodedString`.
 
@@ -1333,7 +1337,7 @@ These are the "Unicode Escape" codec APIs:
    Encode the :c:type:`Py_UNICODE` buffer of the given *size* using Unicode-Escape and
    return a bytes object.  Return ``NULL`` if an exception was raised by the codec.
 
-   .. deprecated-removed:: 3.3 4.0
+   .. deprecated-removed:: 3.3 3.11
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
       :c:func:`PyUnicode_AsUnicodeEscapeString`.
 
@@ -1364,7 +1368,7 @@ These are the "Raw Unicode Escape" codec APIs:
    Encode the :c:type:`Py_UNICODE` buffer of the given *size* using Raw-Unicode-Escape
    and return a bytes object.  Return ``NULL`` if an exception was raised by the codec.
 
-   .. deprecated-removed:: 3.3 4.0
+   .. deprecated-removed:: 3.3 3.11
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
       :c:func:`PyUnicode_AsRawUnicodeEscapeString` or
       :c:func:`PyUnicode_AsEncodedString`.
@@ -1396,7 +1400,7 @@ ordinals and only these are accepted by the codecs during encoding.
    return a Python bytes object.  Return ``NULL`` if an exception was raised by
    the codec.
 
-   .. deprecated-removed:: 3.3 4.0
+   .. deprecated-removed:: 3.3 3.11
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
       :c:func:`PyUnicode_AsLatin1String` or
       :c:func:`PyUnicode_AsEncodedString`.
@@ -1428,7 +1432,7 @@ codes generate errors.
    return a Python bytes object.  Return ``NULL`` if an exception was raised by
    the codec.
 
-   .. deprecated-removed:: 3.3 4.0
+   .. deprecated-removed:: 3.3 3.11
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
       :c:func:`PyUnicode_AsASCIIString` or
       :c:func:`PyUnicode_AsEncodedString`.
@@ -1480,7 +1484,7 @@ These are the mapping codec APIs:
    *mapping* object and return the result as a bytes object.  Return ``NULL`` if
    an exception was raised by the codec.
 
-   .. deprecated-removed:: 3.3 4.0
+   .. deprecated-removed:: 3.3 3.11
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
       :c:func:`PyUnicode_AsCharmapString` or
       :c:func:`PyUnicode_AsEncodedString`.
@@ -1512,7 +1516,7 @@ The following codec API is special in that maps Unicode to Unicode.
    character *mapping* table to it and return the resulting Unicode object.
    Return ``NULL`` when an exception was raised by the codec.
 
-   .. deprecated-removed:: 3.3 4.0
+   .. deprecated-removed:: 3.3 3.11
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
       :c:func:`PyUnicode_Translate`. or :ref:`generic codec based API
       <codec-registry>`

Doc/data/stable_abi.dat

@@ -36,6 +36,7 @@ PyCFunction_Call
 PyCFunction_GetFlags
 PyCFunction_GetFunction
 PyCFunction_GetSelf
+PyCFunction_New
 PyCFunction_NewEx
 PyCFunction_Type
 PyCMethod_New
@@ -144,6 +145,7 @@ PyErr_SetFromErrnoWithFilenameObjects
 PyErr_SetImportError
 PyErr_SetImportErrorSubclass
 PyErr_SetInterrupt
+PyErr_SetInterruptEx
 PyErr_SetNone
 PyErr_SetObject
 PyErr_SetString
@@ -774,7 +776,6 @@ Py_SetPath
 Py_SetProgramName
 Py_SetPythonHome
 Py_SetRecursionLimit
-Py_SymtableString
 Py_UTF8Mode
 Py_VaBuildValue
 Py_XNewRef

Doc/faq/design.rst

@@ -259,9 +259,8 @@ Why isn't there a switch or case statement in Python?
 -----------------------------------------------------
 
 You can do this easily enough with a sequence of ``if... elif... elif... else``.
-There have been some proposals for switch statement syntax, but there is no
-consensus (yet) on whether and how to do range tests.  See :pep:`275` for
-complete details and the current status.
+For literal values, or constants within a namespace, you can also use a
+``match ... case`` statement.
 
 For cases where you need to choose from a very large number of possibilities,
 you can create a dictionary mapping case values to functions to call.  For
@@ -601,7 +600,15 @@ test cases at all.
 Why is there no goto?
 ---------------------
 
-You can use exceptions to provide a "structured goto" that even works across
+In the 1970s people realized that unrestricted goto could lead
+to messy "spaghetti" code that was hard to understand and revise.
+In a high-level language, it is also unneeded as long as there
+are ways to branch (in Python, with ``if`` statements and ``or``,
+``and``, and ``if-else`` expressions) and loop (with ``while``
+and ``for`` statements, possibly containing ``continue`` and ``break``).
+
+One can also use exceptions to provide a "structured goto"
+that works even across
 function calls.  Many feel that exceptions can conveniently emulate all
 reasonable uses of the "go" or "goto" constructs of C, Fortran, and other
 languages.  For example::

Doc/howto/descriptor.rst

@@ -1139,8 +1139,8 @@ If you have ever wondered where *self* comes from in regular methods or where
 *cls* comes from in class methods, this is it!
 
 
-Static methods
---------------
+Other kinds of methods
+----------------------
 
 Non-data descriptors provide a simple mechanism for variations on the usual
 patterns of binding functions into methods.
@@ -1163,6 +1163,10 @@ This chart summarizes the binding and its two most useful variants:
       | classmethod     | f(type(obj), \*args) | f(cls, \*args)   |
       +-----------------+----------------------+------------------+
 
+
+Static methods
+--------------
+
 Static methods return the underlying function without changes.  Calling either
 ``c.f`` or ``C.f`` is the equivalent of a direct lookup into
 ``object.__getattribute__(c, "f")`` or ``object.__getattribute__(C, "f")``. As a

Doc/library/_thread.rst

@@ -61,15 +61,27 @@ This module defines the following constants and functions:
       :func:`sys.unraisablehook` is now used to handle unhandled exceptions.
 
 
-.. function:: interrupt_main()
+.. function:: interrupt_main(signum=signal.SIGINT, /)
 
-   Simulate the effect of a :data:`signal.SIGINT` signal arriving in the main
-   thread. A thread can use this function to interrupt the main thread.
+   Simulate the effect of a signal arriving in the main thread.
+   A thread can use this function to interrupt the main thread, though
+   there is no guarantee that the interruption will happen immediately.
 
-   If :data:`signal.SIGINT` isn't handled by Python (it was set to
+   If given, *signum* is the number of the signal to simulate.
+   If *signum* is not given, :data:`signal.SIGINT` is simulated.
+
+   If the given signal isn't handled by Python (it was set to
    :data:`signal.SIG_DFL` or :data:`signal.SIG_IGN`), this function does
    nothing.
 
+   .. versionchanged:: 3.10
+      The *signum* argument is added to customize the signal number.
+
+   .. note::
+      This does not emit the corresponding signal but schedules a call to
+      the associated handler (if it exists).
+      If you want to truly emit the signal, use :func:`signal.raise_signal`.
+
 
 .. function:: exit()