Recent version 2.28.2 cannot be used in PyInstaller because of missing charset_normalizer files

[ikus060]

Requests is not working in PyInstaller packages because of missing file from charset_normalizer module.

Expected Result

I'm expecting requests to be working within PyInstaller package.

Actual Result

Once the package is compile within an executable, the executable fail to start as follow:

Traceback (most recent call last):
  File "requests\compat.py", line 11, in <module>
ModuleNotFoundError: No module named 'chardet'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "minarca_client\main.py", line 15, in <module>
    from minarca_client.core import (
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "minarca_client\core\__init__.py", line 23, in <module>
    import requests
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "requests\__init__.py", line 45, in <module>
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "requests\exceptions.py", line 9, in <module>
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "requests\compat.py", line 13, in <module>
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "charset_normalizer\__init__.py", line 24, in <module>
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "charset_normalizer\api.py", line 5, in <module>
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "charset_normalizer\cd.py", line 9, in <module>
ModuleNotFoundError: No module named 'charset_normalizer.md__mypyc'

Reproduction Steps

test.py:

import requests

print(requests.get('http://example.com'))

pip install requests==2.28.2 charset-normalizer==3.0.1 pyinstaller
pyinstaller test.py
dist/test/test

System Information

$ python -m requests.help

{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.0.1"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.6"
  },
  "platform": {
    "release": "6.0.12-76060006-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.28.2"
  },
  "system_ssl": {
    "version": "30000020"
  },
  "urllib3": {
    "version": "1.26.10"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}

[sigmavirus24]

Charset normalizer is a dependency of requests, not a project maintained by us. This presents another good reason to ditch it though and stop trying to auto detect character sets.

We can't fix this for you since it's not our code. Closing as a result

[ddffnn]

I'm disagree that this can't be fixed in requests code. The issue seems to be that the try except block around importing the charset_normalizer module doesn't work as intended under pyinstaller builds.

Specifically, in compat.py

try:
    import chardet
except ImportError:
    import charset_normalizer as chardet

I see this throw a ModuleNotFoundError instead of ImportError so it doesn't even look for the charset_normalizer when it should.

[nateprewitt]

I see this throw a ModuleNotFoundError instead of ImportError so it doesn't even look for the charset_normalizer when it should.

It's generally good practice to validate assumptions before asserting them as fact. It avoids unnecessary tangents and wasted time.

try:
    raise ModuleNotFoundError('module not found')
except ImportError as e:
    print(f"Error Type: {type(e)}, {e}")

The stack trace provided shows we are handling the ModuleNotFoundError correctly because the call stack has moved into import charset_normalizer as chardet.

During the handling of that import, PyInstaller is unable to resolve charset_normalizer.md__mypyc for the reasons mentioned in the PyInstaller ticket. It looks like they're working on a fix from their end. I don't believe there's anything else to be done here.

[bwoodsend]

This presents another good reason to ditch it though and stop trying to auto detect character sets.

Oohh, please do. If an API is using a non UTF8 encoding without sending a Content-Encoding header then I'd rather requests just let my know by erroring out rather than guessing.

[bwilso1]

for other people frantically trying to google this issue for PyInstaller, see the workaround in this comment pyinstaller/pyinstaller-hooks-contrib#534 (comment)

still broken for me in PyInstaller 5.10.1, but was reported against 5.7
Edit: the actual solution was upgrade pyinstaller-hooks-contrib, perhaps I did not pip upgrade properly when I updated PyInstaller. But the workaround worked too.

Thanks requests devs for all your hard work!