Bump transformers version

[drbh]

This PR simply bumps the nixpkgs to the latest upstream and explicitly adds transformers to the flake.

Python Library Changes

below is the output of a small gist that outputs the difference in python libraries when replacing the nixpkg commit

uv run https://gist.githubusercontent.com/drbh/790d6a147d6a852887448242bedec47f/raw/8b424e9118b478dae03b11bf9f364048bc00d18d/nixpkg-python-diffs.py d38025438a6ee456758dc03188ca6873a415463b 679aa6b2219d272ae97f00adbc3092581680243

Nix Package Comparison Tool
Usage: python compare_nix_packages.py [old_commit] [new_commit] [--verbose] [--show-unchanged]

Comparing packages between:
  Old: d38025438a6ee456758dc03188ca6873a415463b
  New: 679aa6b2219d272ae97f00adbc3092581680243

Running: nix eval .#packages.x86_64-linux.python3Packages --override-input nixpkgs github:nixos/nixpkgs/d38025438a6ee456758dc03188ca6873a415463b --json
Running: nix eval .#packages.x86_64-linux.python3Packages --override-input nixpkgs github:nixos/nixpkgs/679aa6b2219d272ae97f00adbc3092581680243 --json

Summary:
  Total packages in old: 24
  Total packages in new: 24
  Added: 0
  Removed: 0
  Changed: 24
  Unchanged: 0

Changed packages (24):
Package                            | Old Version | New Version |
-----------------------------------+-------------+-------------+--
awq-inference-engine               | 0.1.0       | 0.1.0       |
causal-conv1d                      | 1.4.0       | 1.4.0       |
compressed-tensors                 | 0.7.1       | 0.7.1       |
exllamav2                          | 0.1.8       | 0.1.8       |
flash-attn                         | 2.6.3       | 2.6.3       |
flash-attn-layer-norm              | 2.6.3       | 2.6.3       |
flash-attn-rotary                  | 2.6.3       | 2.6.3       |
flash-attn-v1                      | unknown     | unknown     |
flashinfer                         | 0.2.0.post2 | 0.2.0.post2 |
hf-transfer                        | 0.1.8       | 0.1.8       |
hf-xet                             | 1.1.5       | 1.1.5       |
kernels                            | 0.7.0       | 0.7.0       |
mamba-ssm                          | 2.2.4       | 2.2.4       |
mktestdocs                         | 0.2.5       | 0.2.5       |
moe                                | 0.3.0       | 0.3.0       |
opentelemetry-instrumentation-grpc | 0.52b1      | 0.55b0      | +
outlines                           | 0.1.13      | 0.1.13      |
paged-attention                    | 0.0.3       | 0.0.3       |
punica-sgmv                        | 0.0.1       | 0.0.1       |
quantization                       | 0.0.4       | 0.0.4       |
quantization-eetq                  | 0.0.2       | 0.0.2       |
rotary                             | 0.0.2       | 0.0.2       |
torch                              | 2.7.1       | 2.7.1       |
transformers                       | 4.53.2      | 4.55.0      | +

Note

all meaningful changes are contained in 08aa1d8

the following commits bump the lock and rerun nix fmt

Update

this PR now adds a new python module for huggingface-hub and transformers with the core goal of updating transformers (and enable a way to explicitly list and update the version in the future)

[danieldk]

I think it would be better to vendor the transformers derivation and maintain it ourselves (like we do for some other packages).

nixpkgs bumps cause a lot of rebuilds, not just the hf-nix cache, but if we need to bump up kernel-builder for a change in hf-nix (e.g. Torch), we need to rebuild all Torch variants on linux-x86_64, linux-aarch64, and darwin-aarch64. Also, often a bunch of things break (due to added checks, etc.). So, I typically only do nixpkgs builds for e.g. new CUDA versions.

By vendoring transformers, we can move versions at our own pace and quickly without mass-rebuilds.

[danieldk]

Remove.