bugfixes to kylesayrs/transform_apply
#375
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Brian Dellabetta <[email protected]>
|
Yep, these are needed as discussed. As for the error, you're trying to load a model with online rotations, but vllm doesn't implement online rotations (yet) |
dsikka
pushed a commit
that referenced
this pull request
Jul 9, 2025
* add utilities Signed-off-by: Kyle Sayers <[email protected]> * add tests Signed-off-by: Kyle Sayers <[email protected]> * add additional tests Signed-off-by: Kyle Sayers <[email protected]> * add utils and tests Signed-off-by: Kyle Sayers <[email protected]> * Implement transform factories Signed-off-by: Kyle Sayers <[email protected]> * add permutations Signed-off-by: Kyle Sayers <[email protected]> * add delete_offload_module Signed-off-by: Kyle Sayers <[email protected]> * key inverses by weight Signed-off-by: Kyle Sayers <[email protected]> * fix tests Signed-off-by: Kyle Sayers <[email protected]> * standardize random hadamard Signed-off-by: Kyle Sayers <[email protected]> * prepend input hooks Signed-off-by: Kyle Sayers <[email protected]> * apply sqrt division first Signed-off-by: Kyle Sayers <[email protected]> * use divided hadamards Signed-off-by: Kyle Sayers <[email protected]> * fix typo Signed-off-by: Kyle Sayers <[email protected]> * add random option Signed-off-by: Kyle Sayers <[email protected]> * use random seeds, rename matrix multiply Signed-off-by: Kyle Sayers <[email protected]> * add deterministic generation to random matrix Signed-off-by: Kyle Sayers <[email protected]> * fix perm math Signed-off-by: Kyle Sayers <[email protected]> * update docstrings Signed-off-by: Kyle Sayers <[email protected]> * update docstrings Signed-off-by: Kyle Sayers <[email protected]> * cleanup Signed-off-by: Kyle Sayers <[email protected]> * cleanup 2 Signed-off-by: Kyle Sayers <[email protected]> * make seed optional Signed-off-by: Kyle Sayers <[email protected]> * remove iterable check and missing return value Signed-off-by: Kyle Sayers <[email protected]> * Remove unrelated changes * simplify code Signed-off-by: Kyle Sayers <[email protected]> * implement apply, use in tests Signed-off-by: Kyle Sayers <[email protected]> * use hadamards database file Signed-off-by: Kyle Sayers <[email protected]> * try manifest Signed-off-by: Kyle Sayers <[email protected]> * try setup, update hadamards list Signed-off-by: Kyle Sayers <[email protected]> * fix setup Signed-off-by: Kyle Sayers <[email protected]> * add docstrings, cleanup Signed-off-by: Kyle Sayers <[email protected]> * fix setup, thank you @dbarbuzzi Signed-off-by: Kyle Sayers <[email protected]> * remove numpy, add tests Signed-off-by: Kyle Sayers <[email protected]> * solidify dtype, add gpu tests Signed-off-by: Kyle Sayers <[email protected]> * fix docstring Signed-off-by: Kyle Sayers <[email protected]> * add device option Signed-off-by: Kyle Sayers <[email protected]> * construct on execution device, cache on offload device Signed-off-by: Kyle Sayers <[email protected]> * save construction device changes for later Signed-off-by: Kyle Sayers <[email protected]> * construct on execution device, cache on offload device * cite nja sloane Signed-off-by: Kyle Sayers <[email protected]> * remove dreg Signed-off-by: Kyle Sayers <[email protected]> * put on device via safe_open Signed-off-by: Kyle Sayers <[email protected]> * nits and docstrings Signed-off-by: Kyle Sayers <[email protected]> * update docstring Signed-off-by: Kyle Sayers <[email protected]> * Merge * merge with construct: construct in float32 Signed-off-by: Kyle Sayers <[email protected]> * construct with same dtype, constructing on fp32 found no difference Signed-off-by: Kyle Sayers <[email protected]> * remove unnecessary imports Signed-off-by: Kyle Sayers <[email protected]> * bugfixes (#375) Signed-off-by: Brian Dellabetta <[email protected]> * use factory_kwargs Signed-off-by: Kyle Sayers <[email protected]> * add frozen dict to deps Signed-off-by: Kyle Sayers <[email protected]> * fix style Signed-off-by: Kyle Sayers <[email protected]> * merge Signed-off-by: Kyle Sayers <[email protected]> * use delete_offload_module Signed-off-by: Kyle Sayers <[email protected]> * add docstrign Signed-off-by: Kyle Sayers <[email protected]> * use parametrize Signed-off-by: Kyle Sayers <[email protected]> * remove random from tests Signed-off-by: Kyle Sayers <[email protected]> --------- Signed-off-by: Kyle Sayers <[email protected]> Signed-off-by: Brian Dellabetta <[email protected]> Co-authored-by: Brian Dellabetta <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
bugfixes to get this branch to work with kylesayrs/transform-modifier.
Currently failing on load-up in vllm: