Skip to content

Update HPC module templates that load Python < 3.10 (Bridges2, Expanse, Richardson, Ascent) #1270

@sbryngelson

Description

@sbryngelson

Problem

toolchain/bootstrap/python.sh has enforced a Python ≥ 3.10 minimum since before the viz PR:

MFC_PYTHON_MIN_MAJOR=3
MFC_PYTHON_MIN_MINOR=10

./mfc.sh build calls this script early and exits with a clear error if the active Python is older. The pyproject.toml requires-python = ">=3.10" field formalises the same constraint.

However, several entries in toolchain/modules still load Python versions below 3.10. Any user who runs source ./mfc.sh load -c <slug> -m <mode> on these systems and then runs ./mfc.sh build will hit the following error immediately:

error: Python 3.x.y (python3) is out of date. Required >= 3.10.

Affected systems

Slug System Module loaded Version Status
b PSC Bridges2 python/3.8.6 3.8.6
e SDSC Expanse python/3.8.5 3.8.5
r Richardson python/3.7 3.7
a OLCF Ascent python (no version pinned) unknown ⚠️ check default

Systems already loading Python ≥ 3.10 (no change needed):
p (3.12.5), tuo (3.12.2), w (3.10.8), c (3.12.1), cc (cray-python/3.11.7), o (3.13), h (3.12), f/famd/d/dai/san (unversioned python — verify defaults are ≥ 3.10).

Required changes to toolchain/modules

Each affected entry needs to be updated to a Python 3.10+ module that the cluster actually provides. The relevant maintainer for each cluster should verify available versions via module spider python and update the entry.

PSC Bridges2 (b)

-b-all python/3.8.6 hdf5 anaconda3
+b-all python/3.11.x hdf5 anaconda3   # verify available version

Bridges2 ships Python 3.11 via module load python/3.11.x (check with module spider python). If no 3.10+ module exists, the fallback is to use the anaconda3 module and conda activate an environment with Python 3.10+.

SDSC Expanse (e)

-e-all python/3.8.5
+e-all python/3.10.x   # or 3.11/3.12 — check module spider

Expanse provides newer Python via module spider python. As of 2024, Python 3.11 modules are available.

Richardson (r)

-r-all python/3.7
+r-all python/3.10.x   # verify available

This system may be decommissioned or may only have old Python available — if no 3.10+ module exists the entry should be removed or documented as unsupported.

OLCF Ascent (a)

The entry loads python without a version pin. Verify that the default Python module on Ascent is ≥ 3.10, and pin it if possible:

-a-all python cmake/3.22.2
+a-all python/3.x.y cmake/3.22.2   # pin to confirmed 3.10+ version

How to verify

After updating an entry, test end-to-end on the cluster:

source ./mfc.sh load -c <slug> -m c
python3 --version          # must be >= 3.10
./mfc.sh build -j 8

Why Python 3.10?

bootstrap/python.sh has required 3.10 for some time — this is not a new constraint introduced by the viz subcommand. The viz subcommand does use some dependencies (textual) that themselves require 3.8+, so the binding constraint remains the toolchain bootstrap script. Python 3.10 added structural pattern matching and improved type annotation syntax used elsewhere in the toolchain.

Notes

  • If a cluster genuinely has no Python 3.10+ module, the workaround is to install miniconda in the user's home directory and conda activate an environment before running ./mfc.sh. This should be documented in the entry as a comment.
  • Unversioned python entries (a, d, dai, f, famd, san) should be audited to confirm their default is ≥ 3.10, and pinned to a specific version where possible to prevent silent regressions when cluster admins update the default.

Metadata

Metadata

Assignees

No one assigned

    Labels

    HPCbugSomething isn't working or doesn't seem right

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions