Skip to content

[TRTLLM-5208][BREAKING CHANGE] chore: make pytorch LLM the default #5312

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jun 19, 2025
Merged

[TRTLLM-5208][BREAKING CHANGE] chore: make pytorch LLM the default #5312

merged 2 commits into from
Jun 19, 2025

Conversation

Superjomn
Copy link
Collaborator

@Superjomn Superjomn commented Jun 18, 2025 •
edited
Loading

PR Description

This PR makes the PyT LLM the default with the following API breaking change which is confirmed with @laikhtewari :

  • To use LLM with PyT backend: from tensorrt_llm import LLM
  • To use LLM with TRT backend: from tensorrt_llm._tensorrt_engine import LLM

We introduce the need for an explicit code change to existing TRT backend users due to the arglists of both PyT LLM and TRT LLM diverging, and there is no seamless way to switch the backend without modifying the code.

The usage code, including tests and examples, has been minimally updated to keep the PR concise and focused.

There will be dedicated PRs to change the doc, examples accordingly later.

@Superjomn Superjomn requested review from a team as code owners June 18, 2025 05:05
@Superjomn Superjomn requested review from lucaslie and QiJune June 18, 2025 05:05
@Superjomn Superjomn marked this pull request as draft June 18, 2025 05:05
@Superjomn Superjomn removed request for lucaslie and QiJune June 18, 2025 05:05
@Superjomn Superjomn changed the title chore[BREAKING CHANGE]: make pytorch LLM the default chore TRTLLM-5208 [BREAKING CHANGE]: make pytorch LLM the default Jun 18, 2025
@Superjomn Superjomn force-pushed the make-pyt-default branch 3 times, most recently from 5ca2c7b to 42c2cad Compare June 18, 2025 09:42
@Superjomn Superjomn marked this pull request as ready for review June 18, 2025 09:44
@juney-nvidia
Copy link
Collaborator

Since we have already branch out 0.21 branch, it is okay to land this PR onto the GH main directly.

Thanks
June

@Superjomn
Copy link
Collaborator Author

/bot run

@Superjomn Superjomn changed the title chore TRTLLM-5208 [BREAKING CHANGE]: make pytorch LLM the default [TRTLLM-5208][BREAKING CHANGE] chore: make pytorch LLM the default Jun 18, 2025
@tensorrt-cicd
Copy link
Collaborator

PR_Github #9377 [ run ] triggered by Bot

QiJune
QiJune reviewed Jun 18, 2025
View reviewed changes
@tensorrt-cicd
Copy link
Collaborator

PR_Github #9377 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #6882 completed with status: 'FAILURE'

@Superjomn
Copy link
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd
Copy link
Collaborator

PR_Github #9394 [ run ] triggered by Bot

@Superjomn
Copy link
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd
Copy link
Collaborator

PR_Github #9461 [ run ] triggered by Bot

QiJune
QiJune approved these changes Jun 19, 2025
View reviewed changes
Copy link
Collaborator

@QiJune QiJune left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tensorrt-cicd
Copy link
Collaborator

PR_Github #9461 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #6946 completed with status: 'FAILURE'

@Superjomn Superjomn requested a review from a team as a code owner June 19, 2025 10:26
@Superjomn
Copy link
Collaborator Author

/bot run --disable-fail-fast

@Superjomn Superjomn requested a review from syuoni June 19, 2025 10:27
@tensorrt-cicd
Copy link
Collaborator

PR_Github #9499 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #9499 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #6969 completed with status: 'FAILURE'

Superjomn added 2 commits June 19, 2025 14:39
@Superjomn
make PyT default
952e012 
Signed-off-by: Superjomn <[email protected]>
@Superjomn
fix
302d4b1 
Signed-off-by: Superjomn <[email protected]>
@Superjomn
Copy link
Collaborator Author

/bot run --disable-fail-fast

@Superjomn Superjomn enabled auto-merge (squash) June 19, 2025 14:41
@tensorrt-cicd
Copy link
Collaborator

PR_Github #9525 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #9525 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #6988 completed with status: 'SUCCESS'
Pipeline passed with automatic retried tests. Check the rerun report for details.

@Superjomn Superjomn merged commit 9bd42ec into NVIDIA:main Jun 19, 2025
3 checks passed
@Superjomn Superjomn deleted the make-pyt-default branch June 19, 2025 21:32
This was referenced Jun 25, 2025
Merged
Merged
dominicshanshan pushed a commit to dominicshanshan/TensorRT-LLM that referenced this pull request Jul 9, 2025
@Superjomn @dominicshanshan
[TRTLLM-5208][BREAKING CHANGE] chore: make pytorch LLM the default (N…
3b2a310 
…VIDIA#5312)

Signed-off-by: Superjomn <[email protected]>
dominicshanshan pushed a commit to dominicshanshan/TensorRT-LLM that referenced this pull request Jul 10, 2025
@Superjomn @dominicshanshan
[TRTLLM-5208][BREAKING CHANGE] chore: make pytorch LLM the default (N…
2930a6b 
…VIDIA#5312)

Signed-off-by: Superjomn <[email protected]>
dominicshanshan pushed a commit to dominicshanshan/TensorRT-LLM that referenced this pull request Jul 10, 2025
@Superjomn @dominicshanshan
[TRTLLM-5208][BREAKING CHANGE] chore: make pytorch LLM the default (N…
7e1b545 
…VIDIA#5312)

Signed-off-by: Superjomn <[email protected]>
dominicshanshan pushed a commit to dominicshanshan/TensorRT-LLM that referenced this pull request Jul 10, 2025
@Superjomn @dominicshanshan
[TRTLLM-5208][BREAKING CHANGE] chore: make pytorch LLM the default (N…
e7a6675 
…VIDIA#5312)

Signed-off-by: Superjomn <[email protected]>
dominicshanshan pushed a commit to dominicshanshan/TensorRT-LLM that referenced this pull request Jul 10, 2025
@Superjomn @dominicshanshan
[TRTLLM-5208][BREAKING CHANGE] chore: make pytorch LLM the default (N…
c52a1e2 
…VIDIA#5312)

Signed-off-by: Superjomn <[email protected]>
dominicshanshan pushed a commit to dominicshanshan/TensorRT-LLM that referenced this pull request Jul 11, 2025
@Superjomn @dominicshanshan
[TRTLLM-5208][BREAKING CHANGE] chore: make pytorch LLM the default (N…
4ce63d9 
…VIDIA#5312)

Signed-off-by: Superjomn <[email protected]>
dominicshanshan pushed a commit to dominicshanshan/TensorRT-LLM that referenced this pull request Jul 11, 2025
@Superjomn @dominicshanshan
[TRTLLM-5208][BREAKING CHANGE] chore: make pytorch LLM the default (N…
3106ee3 
…VIDIA#5312)

Signed-off-by: Superjomn <[email protected]>
dominicshanshan pushed a commit to dominicshanshan/TensorRT-LLM that referenced this pull request Jul 11, 2025
@Superjomn @dominicshanshan
[TRTLLM-5208][BREAKING CHANGE] chore: make pytorch LLM the default (N…
c9f40b8 
…VIDIA#5312)

Signed-off-by: Superjomn <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@QiJune QiJune QiJune approved these changes

@Fridah-nv Fridah-nv Fridah-nv approved these changes

@lucaslie lucaslie Awaiting requested review from lucaslie

@suyoggupta suyoggupta Awaiting requested review from suyoggupta

@nv-guomingz nv-guomingz Awaiting requested review from nv-guomingz

@syuoni syuoni Awaiting requested review from syuoni

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@Superjomn @juney-nvidia @tensorrt-cicd @QiJune @Fridah-nv