gh-110932: Fix regrtest for SOURCE_DATE_EPOCH by vstinner · Pull Request #111143 · python/cpython

vstinner · 2023-10-20T22:26:56Z

If the SOURCE_DATE_EPOCH environment variable is defined, use its value as the random seed.

Issue: SOURCE_DATE_EPOCH: AssertionError: 'Using random seed ([0-9]+)' not found in 'Using random seed None...' #110932

If the SOURCE_DATE_EPOCH environment variable is defined, use its value as the random seed.

vstinner · 2023-10-20T22:30:12Z

SOURCE_DATE_EPOCH intent is to make programs reproducible. Without this change, if SOURCE_DATE_EPOCH is set, regrtest uses a random seed which makes tests less reproducible.

With this change, the random seed is constant and so the behavior should be more reproducible.

--randseed=custom_string command line option is invalid: the option value must be an integer. Only SOURCE_DATE_EPOCH is used to use a string as the random seed.

vstinner · 2023-10-20T22:30:50Z

@sobolevn: Would you mind to review this change?

sobolevn

Generally looks good to me, just one idea.

sobolevn · 2023-10-21T04:59:47Z

Lib/test/libregrtest/main.py

-            else random.getrandbits(32)
-        )
-        if 'SOURCE_DATE_EPOCH' in os.environ:
+        if ('SOURCE_DATE_EPOCH' in os.environ


nit

Suggested change

if ('SOURCE_DATE_EPOCH' in os.environ

if (

'SOURCE_DATE_EPOCH' in os.environ

sobolevn · 2023-10-21T05:00:25Z

Lib/test/libregrtest/main.py

+            # SOURCE_DATE_EPOCH should be an integer, but use a string to not
+            # fail if it's not integer. random.seed() accepts a string.
+            # https://reproducible-builds.org/docs/source-date-epoch/
+            self.random_seed: int | str = os.environ['SOURCE_DATE_EPOCH']


Myabe we should always use str? There's no real reason to use int here.

The reason is subtle. It avoids to import hashlib to convert a string to an integer in Random.seed(). Right now, hashlib is always imported, but I have a local branch to reduce the number of imports at Python startup.

I'm talking about the second code path, where we create a random seed as an integer.

sobolevn · 2023-10-21T05:00:49Z

Lib/test/libregrtest/main.py

+            # https://reproducible-builds.org/docs/source-date-epoch/
+            self.random_seed: int | str = os.environ['SOURCE_DATE_EPOCH']
+        elif ns.random_seed is None:
+            self.random_seed = random.getrandbits(32)


If so, it would become:

Suggested change

self.random_seed = random.getrandbits(32)

self.random_seed = str(random.getrandbits(32))

miss-islington-app · 2023-10-21T08:38:02Z

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

miss-islington-app · 2023-10-21T08:38:03Z

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

If the SOURCE_DATE_EPOCH environment variable is defined, use its value as the random seed. (cherry picked from commit 7237fb5) Co-authored-by: Victor Stinner <vstinner@python.org>

bedevere-app · 2023-10-21T08:38:24Z

GH-111153 is a backport of this pull request to the 3.11 branch.

bedevere-app · 2023-10-21T08:38:28Z

GH-111154 is a backport of this pull request to the 3.12 branch.

vstinner · 2023-10-21T08:39:17Z

Merged, thanks to reviewing the change @sobolevn.

…1153) gh-110932: Fix regrtest for SOURCE_DATE_EPOCH (GH-111143) If the SOURCE_DATE_EPOCH environment variable is defined, use its value as the random seed. (cherry picked from commit 7237fb5) Co-authored-by: Victor Stinner <vstinner@python.org>

…1154) gh-110932: Fix regrtest for SOURCE_DATE_EPOCH (GH-111143) If the SOURCE_DATE_EPOCH environment variable is defined, use its value as the random seed. (cherry picked from commit 7237fb5) Co-authored-by: Victor Stinner <vstinner@python.org>

If the SOURCE_DATE_EPOCH environment variable is defined, use its value as the random seed.

pythongh-110932: Fix regrtest for SOURCE_DATE_EPOCH

55bf9f7

If the SOURCE_DATE_EPOCH environment variable is defined, use its value as the random seed.

bedevere-app bot mentioned this pull request Oct 20, 2023

SOURCE_DATE_EPOCH: AssertionError: 'Using random seed ([0-9]+)' not found in 'Using random seed None...' #110932

Closed

bedevere-app bot added the awaiting core review label Oct 20, 2023

sobolevn reviewed Oct 21, 2023

View reviewed changes

vstinner merged commit 7237fb5 into python:main Oct 21, 2023

vstinner deleted the regrtest_epoch branch October 21, 2023 08:37

bedevere-app bot removed the awaiting core review label Oct 21, 2023

vstinner added needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Oct 21, 2023

bedevere-app bot removed the needs backport to 3.11 only security fixes label Oct 21, 2023

bedevere-app bot removed the needs backport to 3.12 only security fixes label Oct 21, 2023

aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024

pythongh-110932: Fix regrtest for SOURCE_DATE_EPOCH (python#111143)

9017ebb

If the SOURCE_DATE_EPOCH environment variable is defined, use its value as the random seed.

Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024

pythongh-110932: Fix regrtest for SOURCE_DATE_EPOCH (python#111143)

ee93145

If the SOURCE_DATE_EPOCH environment variable is defined, use its value as the random seed.

	if ('SOURCE_DATE_EPOCH' in os.environ
	if (
	'SOURCE_DATE_EPOCH' in os.environ

	self.random_seed = random.getrandbits(32)
	self.random_seed = str(random.getrandbits(32))

Uh oh!

Conversation

vstinner commented Oct 20, 2023 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vstinner commented Oct 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vstinner commented Oct 20, 2023

Uh oh!

sobolevn left a comment

Choose a reason for hiding this comment

Uh oh!

sobolevn Oct 21, 2023

Choose a reason for hiding this comment

Uh oh!

sobolevn Oct 21, 2023

Choose a reason for hiding this comment

Uh oh!

vstinner Oct 21, 2023

Choose a reason for hiding this comment

Uh oh!

vstinner Oct 21, 2023

Choose a reason for hiding this comment

Uh oh!

sobolevn Oct 21, 2023

Choose a reason for hiding this comment

Uh oh!

miss-islington-app bot commented Oct 21, 2023

Uh oh!

miss-islington-app bot commented Oct 21, 2023

Uh oh!

bedevere-app bot commented Oct 21, 2023

Uh oh!

bedevere-app bot commented Oct 21, 2023

Uh oh!

vstinner commented Oct 21, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

vstinner commented Oct 20, 2023 •

edited by bedevere-app bot

Loading

vstinner commented Oct 20, 2023 •

edited

Loading