Skip to content

Commit 877e1f4

Browse files
authored
[https://nvbugs/5451426][fix] Avoid torch compile on full eagle3 worker (#7245)
Signed-off-by: Jin Li <[email protected]>
1 parent 201fd25 commit 877e1f4

File tree

1 file changed

+0
-1
lines changed

1 file changed

+0
-1
lines changed

tensorrt_llm/_torch/speculative/eagle3.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -266,7 +266,6 @@ def __init__(self, spec_config: "EagleDecodingConfig", mapping: Mapping):
266266
self.max_draft_len = self.spec_config.max_draft_len
267267
self.mapping = mapping
268268

269-
@torch.compile(options={"max-autotune": True})
270269
def forward(self, input_ids, position_ids, hidden_states, logits,
271270
attn_metadata, spec_metadata, draft_model):
272271
batch_size = attn_metadata.num_seqs

0 commit comments

Comments
 (0)