Skip to content
This repository was archived by the owner on Feb 24, 2026. It is now read-only.

[Dev] Enhance Operator Cache to support multi-thread environments#205

Merged
LeiWang1999 merged 14 commits intomicrosoft:mainfrom
LeiWang1999:tl_ops_dynamic
Oct 1, 2024
Merged

[Dev] Enhance Operator Cache to support multi-thread environments#205
LeiWang1999 merged 14 commits intomicrosoft:mainfrom
LeiWang1999:tl_ops_dynamic

Conversation

@LeiWang1999
Copy link
Contributor

Fix ref to #204 #186

This pull request introduces several changes to improve thread safety, enhance scheduler functionality, and refactor code for better readability and maintainability. The most important changes include adding a lock to the OperatorCache class, modifying the ThreadPoolExecutor to use a variable number of workers, and introducing a new fine-grained matrix multiplication scheduler.

Thread Safety Enhancements:

  • bitblas/cache/operator.py: Added a cache_locker using threading.RLock to synchronize access to the cache in methods like add, get, clear, and save_into_database. [1] [2]

Scheduler Improvements:

Code Refactoring:

Additional Changes:

@LeiWang1999 LeiWang1999 merged commit 22e25c9 into microsoft:main Oct 1, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant