[Bug]: Docker Image

[xuuyangg]

System Info

In the Docker Image 1.2.0rc1, the file cudaDriverWrapper.h is missing from the path /app/tensorrt_llm in the Docker image. As a result, the required include header files must be manually copied from the source code.

In addition, nvidia-modelopt was installed incorrectly, which causes the export of the quantized KV cache to fail.

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

1

Expected behavior

1

actual behavior

1

additional notes

1

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.

[dc3671]

Hi @xuuyangg

• If you want to build from source, I recommend you to use devel version of images rather than release version, like nvcr.io/nvidia/tensorrt-llm/devel:1.2.0rc4. More details in README
• model-opt package is not strictly synced to the latest in our image, but you can manually update it to the latest version. The version requirement is also in our requirement.txt

[xuuyangg]

I want to link tensorrt_llm.so into my project. Which Docker image should I use? @dc3671