Skip to content

v0.3.1

Latest

Choose a tag to compare

@hannamw hannamw released this 14 Jan 08:19
· 1 commit to main since this release
e09b5f3

This release fixes two bugs:

  • Error nodes for skip transcoders were computed without accounting for the skip connection, resulting in inflated error nodes and outward edges. This would have affected Llama 3.2 1B graphs made with skip transcoders, as well as Gemma 3 graphs made with skip transcoders.
  • Error nodes for gemma-3 instruct models were only being zeroed out at position 0, rather than at the first 4 positions (corresponding to the 4 static BOS-adjacent tokens that their transcoders were not trained on)