argparse - make return type from `add_subparsers` public

[highvelcty]

This issue was brought up and closed as part of issue # 41592.

The add_subparsers method returns a protected _SubParsersAction instance. To add a sub-parser, the public add_parser method of the protected _SubParsersAction instance must be used.

1. It is unexpected to require the use of a public method of a protected class instance.
2. The protected return type breaks type hinting because
   1. protected and should not be imported
   2. it is not available for import

Real world use case example:

sub_parsers_action = parser.add_subparsers(...)

def build_subcmd(sub_parsers_action: _SubParsersAction):
    ...

The ask is to make the type of whatever is returned from add_subparsers public.

[hpaulj]

Do you really have to include type hinting when creating a utility function like this?

I'll admit that I haven't paid much attention to typing, but as it stands there's nothing in argparse that makes any concession to that.

My arguments (as paulj3) in the earlier issue are still relevant.

[highvelcty]

It is not required, but it is a development aide.

It was a bit difficult for me to understand how sub-command tree branches can be formed with the following construct. However, I was able to figure it out from the good documentation surrounding it. Type hint coherency would help to further communicate this.

ArgumentParser(...).add_subparsers(...)  -> _SubParsersAction
_SubParsersAction.add_parser(...) -> ArgumentParser

[hpaulj]

There are several other functions that return "private" objects

 a = parser.add_argument(...)

a will be some subclass of Action, e.g argparse._StoreAction. This is just a variation on the subparsers case. Normally we don't hang on to reference to this Action object, but it is available for later use. The Action is stored in the parser._actions list. On occasion programs use this list to do specific things with one or more of the Actions.

  g = parser.add_argument_group(....)

return an argparse._ArgumentGroup object. Same for mutually exclusive groups.

These groups, and the parser itself are subclasses of argparse._ActionsContainer. Most of the class hierarchy behind argparse objects is "hidden", that is, not documented or listed in the argparse.__all__, but they are none the less available to the programmer.

argparse was created for py2, long before typing and even annotations were around. It's a clean OOP structure, but the documentation is use oriented, Normally ArgumentParser is the only class that's created directly (via class name and its init. Nearly everything else is produced by a factory function or method. Those are what's documented, not the classes.

Also you can substitute your own classes, or subclasses, in many places. The HelpFormatter, the action parameter of add_argument, even the parser class returned by add_parser, By fiddling with the registry you can change the class returned by add_subparsers. And nearly all of the substitutions use duck typing.

Many methods use (*args, **kwargs) signatures, So the signature itself tells us, or any 'inspector', little about the parameters. In the case of add_argument, the details about how **kwargs are handled depend on the Action subclass that's being used. There are issues where people complain about add_argument accepting parameters that don't work, or don't even make sense. In some cases the 'errors' are ignored, in others they they are raised at low levels. In general the more you push the boundaries of what's documented, the more important it is to be able read the argparse.py code yourself.

[jmracek]

Do you really have to include type hinting when creating a utility function like this?

I am forced to do this due to company wide lint rules. I had the exact same use case as @highvelcty. I tried to track down the type returned by add_subparsers() in the public documentation to no success, and further Googling landed me on this issue.

[Warchant]

+1 - please make that type public.
add_subparsers now returns _SubParsersAction[ArgumentParser]

[wvenialbo]

I also require this feature to comply with my company's coding lint rules.

[wajda]

Do you really have to include type hinting when creating a utility function like this?

Oh yes, of course I do, and I would encourage every Python developer to use type hints as much as possible!
We write code primarily for humans, not machines. Think about this way - when you are reading a piece of code does it really matter if it's a piece of a simple 50-line utility function or a complex 50,000-line project - the code is code and you need to be able to easily understand it, change it and maintain it. And from that perspective proper type hints are vital.
I completely agree with @highvelcty, a public function or a method can only return a public type otherwise it's simply drives the type hints on this part of API useless.
Please make the _SubParsersAction public or return another public abstraction instead.

[hpaulj]

I don't think changes to public/protected names, as well as typing and annotations should be made on a piecemeal basis. IMO, `argparse` was well written according to the PEP8 guidelines used 20 years ago. All public classes, functions, and constants are listed in the import `__all__`. These are names that could be imported with `*`, and that users would normally be expected to use directly, such as `ArgumentParser` itself. Same goes for all the documented methods. But the same standards do not apply to objects that methods return. As I noted above, `add_argument` as well as `add_subparsers` returns a `_Action` subclass instance. Users shouldn't try to instantiate such classes directly. `_Action` instances need to be properly 'vetted' before they are created, and stored (by reference) in the parser (and parser groups). There is a `_registries` dictionary that lets us specify actions by name (`action='store'`) without actually using the protected class name. Adherence to more modern guidelines like typing might be nice, but this is legacy code. Searching pypi for 'typed' 'argparse' returns 180 projects. Surely something there meets your needs :) Paul

…

On Mon, Sep 23, 2024 at 11:30 AM Alex Vajda ***@***.***> wrote: Do you really have to include type hinting when creating a utility function like this? Oh yes, of course I do, and I would encourage every Python developer use type hints as much as possible! We write code primarily for humans, not machines. Think about this way - when you are reading a piece of code does it really matter if it's a piece of a simple 50-line utility function or a complex 50,000-line project - the code is code and you need to able easily understand it, change it and maintain it. And from that perspective proper type hints a vital. I completely agree with @highvelcty <https://github.com/highvelcty>, a *public* function or a method can only return a *public* type otherwise it's simply drives the type hints on this part of API useless. Please make the _SubParsersAction public or return another public abstraction instead. — Reply to this email directly, view it on GitHub <#101503 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAITB6AFWV46KSGGV6OAVADZYBM5BAVCNFSM6AAAAABOWU6N4GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRZGA2TSNRRGI> . You are receiving this because you commented.Message ID: ***@***.***>

[hpaulj]

`_ArgumentGroup` and `_MutuallyExclusiveGroup` are other protected subclasses that users might reference (but not instantiate directly).

…

On Mon, Sep 23, 2024 at 1:52 PM paulj ***@***.***> wrote: I don't think changes to public/protected names, as well as typing and annotations should be made on a piecemeal basis. IMO, `argparse` was well written according to the PEP8 guidelines used 20 years ago. All public classes, functions, and constants are listed in the import `__all__`. These are names that could be imported with `*`, and that users would normally be expected to use directly, such as `ArgumentParser` itself. Same goes for all the documented methods. But the same standards do not apply to objects that methods return. As I noted above, `add_argument` as well as `add_subparsers` returns a `_Action` subclass instance. Users shouldn't try to instantiate such classes directly. `_Action` instances need to be properly 'vetted' before they are created, and stored (by reference) in the parser (and parser groups). There is a `_registries` dictionary that lets us specify actions by name (`action='store'`) without actually using the protected class name. Adherence to more modern guidelines like typing might be nice, but this is legacy code. Searching pypi for 'typed' 'argparse' returns 180 projects. Surely something there meets your needs :) Paul On Mon, Sep 23, 2024 at 11:30 AM Alex Vajda ***@***.***> wrote: > Do you really have to include type hinting when creating a utility > function like this? > > Oh yes, of course I do, and I would encourage every Python developer use > type hints as much as possible! > We write code primarily for humans, not machines. Think about this way - > when you are reading a piece of code does it really matter if it's a piece > of a simple 50-line utility function or a complex 50,000-line project - the > code is code and you need to able easily understand it, change it and > maintain it. And from that perspective proper type hints a vital. > I completely agree with @highvelcty <https://github.com/highvelcty>, a > *public* function or a method can only return a *public* type otherwise > it's simply drives the type hints on this part of API useless. > Please make the _SubParsersAction public or return another public > abstraction instead. > > — > Reply to this email directly, view it on GitHub > <#101503 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAITB6AFWV46KSGGV6OAVADZYBM5BAVCNFSM6AAAAABOWU6N4GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRZGA2TSNRRGI> > . > You are receiving this because you commented.Message ID: > ***@***.***> >

[smheidrich]

add_argument as well as add_subparsers returns a _Action
subclass instance. Users shouldn't try to instantiate such classes
directly. _Action instances need to be properly 'vetted' before they are
created

The type hint wouldn't have to be the exact class that is returned, a Protocol containing the publicly documented methods would be perfectly fine.