Skip to content

Implement $O(n^3)$ flow finding algorithm#294

Closed
Newtech66 wants to merge 13 commits intoTeamGraphix:masterfrom
Newtech66:newflow
Closed

Implement $O(n^3)$ flow finding algorithm#294
Newtech66 wants to merge 13 commits intoTeamGraphix:masterfrom
Newtech66:newflow

Conversation

@Newtech66
Copy link

@Newtech66 Newtech66 commented Jun 4, 2025

Description of the change:

  • Implement faster flow finding algorithm.
  • Figure out how to implement the mode argument of the old flow finding algorithm in the new one.
  • Remove old flow finding algorithm.
  • Add method right_inverse() to linalg.py.
  • Update get_rank to use GF2.row_reduce.
  • Fix incorrect get_pauli_nodes method. (fixed in PR Fix handling of plane YZ in Pauli flow #297)

Related issue:
Closes #279.

Checks:

  • Make sure you have tests for the new code and that test passes (run nox)
  • If applicable, add a line to the [unreleased] part of CHANGELOG.md, following keep-a-changelog.
  • Format added code by ruff
    • See CONTRIBUTING.md for more details
  • Make sure the checks (github actions) pass.

@EarlMilktea
Copy link
Contributor

@Newtech66 (CC: @thierry-martinez @shinich1 )

Thank you for opening this PR.
I'm the creator of fastflow package mentioned in #279 .
I have a few comments related to this PR:

  • Old flow/gflow stuff will soon be replaced by fastflow .
  • In fastflow there are dedicated classes for Pauli measurements and I'm planning to introduce this convention to graphix .
  • Please do not use linalg.py .
    • This is severely slow. How about using the Rust-implemented GF2 solver in fastflow ? It's currently private but I would expose it from fastflow if necessary.

@mgarnier59
Copy link
Contributor

mgarnier59 commented Jun 5, 2025

Hi @EarlMilktea ,

we will comment on this PR soon with @thierry-martinez.

Note that there is also a PauliMeausurement class in measurement.py.

Since this is a Python implementation, I think benchmarking it against Python implementations makes more sense.

Also, @Newtech66 saw a convention issue in the old flow-finding algorithms and we are currently investigating why this has not been seen.

Another suggestion, maybe don't replace the old implementation so that we can compare the two versions.

We'll update you soon.

In any case thanks for the work!

@EarlMilktea
Copy link
Contributor

don't replace the old implementation

I'm strongly opposed to this idea for code quality improvements. It must be replaced, at least after merging this PR.

Anyway keep me updated!

@Newtech66 Newtech66 marked this pull request as ready for review June 7, 2025 20:40
Copy link
Contributor

@EarlMilktea EarlMilktea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MEMO: Reviews work in progress.

graphix/gflow.py Outdated
non_output_idx = np.searchsorted(nodes, non_output_nodes)
non_input_map = {v: i for v, i in zip(non_input_nodes, non_input_idx)}
non_output_map = {v: i for v, i in zip(non_output_nodes, non_output_idx)}
n, nI, nO = len(nodes), len(iset), len(oset)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please respect PEP8 naming convention (ni, no, etc,)

@EarlMilktea
Copy link
Contributor

@Newtech66 (CC: @shinich1 )

Opened TeamGraphix/swiflow#9 to collaborate with this PR in the future.

@Newtech66
Copy link
Author

This comment has to be addressed to proceed with the tests for this PR.

@EarlMilktea
Copy link
Contributor

@Newtech66

At the maintainer meeting, we came to the following conclusions:

  • GF2 solver used in fastflow is not suitable for this PR
    • Because it is not designed to expose the intermediate states (ex. partially-eliminated matrix)
  • General-purpose Pythonic GF2 solver will be provided in this package
  • Current flow/gflow calculation routines will be replaced by fastflow

@mgarnier59
Copy link
Contributor

Hi @Newtech66
to make your code more readable could you had more comments and make the link with the steps in the paper more explicit?

Thanks!

@EarlMilktea
Copy link
Contributor

Additional comments from me, mainly related to maintainability:

  • find_pauliflow is too complex: you need to split it into functions.
  • As the patch is fairly large, this PR should be typechecked before merge.
  • Index conversion (nx.searchsorted) will be well substituted by utility classes defined in fastflow.

@Newtech66
Copy link
Author

find_pauliflow is too complex: you need to split it into functions.

You're right, I planned to do that once I was done writing the function, but I just forgot somehow...

@Newtech66
Copy link
Author

All the tests are passing now.

@shinich1
Copy link
Contributor

@Newtech66 could you resolve conflict with master branch, so that CI can run?

@Newtech66
Copy link
Author

@shinich1 Done.

@codecov
Copy link

codecov bot commented Jun 11, 2025

Codecov Report

Attention: Patch coverage is 97.22222% with 4 lines in your changes missing coverage. Please review.

Project coverage is 80.47%. Comparing base (1dc02d7) to head (1b38df5).

Files with missing lines Patch % Lines
graphix/gflow.py 96.74% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #294      +/-   ##
==========================================
+ Coverage   79.83%   80.47%   +0.64%     
==========================================
  Files          36       36              
  Lines        5429     5414      -15     
==========================================
+ Hits         4334     4357      +23     
+ Misses       1095     1057      -38     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@EarlMilktea
Copy link
Contributor

EarlMilktea commented Jun 11, 2025

@shinich1 @Newtech66 @mgarnier59 @thierry-martinez

My suggestion: why not merge this PR (of course, after ready) at the same time with #280 ?

This is because

  • We need to typecheck this PR because the diff. is big/procedure is very complex.
    • This cannot be done unless gflow.py itself is annotated.
    • Optionally we could create a dedicated file for this PR and merge it into gflow.py later.
  • This PR is 100 % affected by linalg.py refactoring, which requires Add fastflow #280 .

@thierry-martinez
Copy link
Collaborator

Indeed, if there's no urgency to merge this PR (i.e., if we're out of time to merge it before the Unitary Hack), then it's better to wait for #280 to be merged first.

@Newtech66
Copy link
Author

There's only 8.5 hours before the end of Unitary Hack.

@EarlMilktea
Copy link
Contributor

There's only 8.5 hours before the end of Unitary Hack.

8.5 hours is obviously too short for reviewing this PR, but at the same time I personally feel @Newtech66 's effort should somehow be appreciated.
@shinich1 Any comments?

@shinich1
Copy link
Contributor

Hack allows the PR to be open until the 16th. I will assign @Newtech66 to issue so that it's clear you're still working on the bountied issue (to the UF).

@Newtech66
Copy link
Author

@EarlMilktea Actually, why don't we port this new algorithm over to fastflow in the first place? Why bother updating gflow.py at all?

@EarlMilktea
Copy link
Contributor

@EarlMilktea Actually, why don't we port this new algorithm over to fastflow in the first place? Why bother updating gflow.py at all?

There're some good reasons for not doing so.

  • Rust binding is complicated

It should be considered as the last resort only for performance-critical fractions.

  • Solver is not compatible with this PR

fastflow is using a bitset-based GF(2) solver, which only takes up 1 bit for each entry.
While it is indeed fast, flexible operations (ex. concatenation, slicing, partial Gaussian elimination, etc.) are not supported and not even planed in the future.
This is one of the reasons why TeamGraphix/swiflow#9 was abandoned.

@EarlMilktea EarlMilktea mentioned this pull request Jun 13, 2025
@Newtech66
Copy link
Author

Any updates on this?

@shinich1
Copy link
Contributor

Any updates on this?

My understanding is that we tried but are still struggling to finalize our design.
I'm contacting Unitary Foundation to set up discussion thread - I believe you'll hear from them soon. thanks for your patience.

@thierry-martinez
Copy link
Collaborator

#337 subsumes this PR.

matulni added a commit that referenced this pull request Oct 2, 2025
Introduce the `O(N^3)` Pauli flow-finding algorithm presented in [1] which improves the previous version with `O(N^5)`
complexity [2]. Since a Pauli flow in an open graph without Pauli measurements defines a gflow, this algorithm also improves the previous implementation of the gflow-finding algorithm with `O(N^4)` complexity [3].

The implementation proposed here considers the suggestion in [1] to eliminate the rows that are identically 0 in the order-demand matrix to accelerate the flow-finding routine in open graphs with nI ≠ n O.

Finally, the new algorithm solves the issue discussed in qat-inria#17 , whereby the previous flow-finding functions could return a correction function containing input nodes in the codomain, which is not allowed by the definition of gflow or Pauli flow.

In summary:

- The core functions of the algorithm are encapsulated in the new module `graphix.find_pflow`, while `graphix.gflow` and related modules are adapted accordingly to avoid modifying the current API in this PR.
- The module `graphix.linalg` has been renamed `graphix._linalg` and updated with more efficient functions for gaussian elimination, back substitution, kernel-finding and right-inverse. New implementation is GF2-specialized and, where appropriate, just-in-time compiled with numba. The dependence on `sympy` and `galois` is dropped. Tests are simplified and updated accordingly, and module is now typed.
- `pytest-benchmark` is incorporated to allow for easier benchmarking between different commits (courtesy of @thierry-martinez).

Additional comments
---------------------
- This PR addresses the same issue as #294, but is otherwise unrelated.
- The module `graphix.gflow` will be restructured in a future PR.

The implementation exhibits the cubic scaling as described in [1]. See https://gitlab.inria.fr/muldemol/pflow-on3-scaling.git for a benchmarking code.

References
-----------
[1] Mitosek and Backens, 2024 (arXiv:2410.23439)
[2] Simmons, 2021 (arXiv:2109.05654)
[3] Backens et al., Quantum 5, 421 (2021)

---------

Co-authored-by: Thierry Martinez <thierry.martinez@inria.fr>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement a faster Pauli-flow finding algorithm

5 participants