Skip to content
This repository was archived by the owner on Feb 24, 2026. It is now read-only.

[Dev] Improve Dequant performance on CUDA Simt#189

Merged
LeiWang1999 merged 4 commits intomicrosoft:mainfrom
LeiWang1999:v100
Sep 20, 2024
Merged

[Dev] Improve Dequant performance on CUDA Simt#189
LeiWang1999 merged 4 commits intomicrosoft:mainfrom
LeiWang1999:v100

Conversation

@LeiWang1999
Copy link
Contributor

This pull request includes several changes to improve error handling, code clarity, and functionality in the bitblas module. The most important changes include adding error handling when loading runtime modules, fixing a typo in comments, introducing alignment factors for vectorization, and adding new methods for fetching data to local memory.

Error Handling:

  • bitblas/cache/operator.py: Added error handling for loading runtime modules, logging an error message if the module fails to load.

Code Clarity:

Functionality Enhancements:

  • bitblas/gpu/matmul.py: Introduced an align_factor to handle unaligned K and BK values, and expanded the vectorization factor. [1] [2]
  • bitblas/gpu/matmul.py: Added a new method fetch_to_local to fetch data to local memory with alignment considerations.
  • bitblas/ops/operator.py: Modified update_runtime_module to handle cases where rt_mod is None and added conditional evaluation of time_evaluator.

@LeiWang1999 LeiWang1999 merged commit 6333d3b into microsoft:main Sep 20, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant