When loading the model I get the following error:

[JeisonJimenezA]

llm_load_tensors: ggml ctx size = 0.16 MB
llm_load_tensors: using CUDA for GPU acceleration
llm_load_tensors: mem required = 9363.40 MB
llm_load_tensors: offloading 6 repeating layers to GPU
llm_load_tensors: offloaded 6/43 layers to GPU
llm_load_tensors: VRAM used: 1637.37 MB
.................................................................................GGML_ASSERT: D:\a\llama-cpp-python-cuBLAS-wheels\llama-cpp-python-cuBLAS-wheels\vendor\llama.cpp\ggml-cuda.cu:5925: false

[jllllll]

What model are you trying to load? This error is indicative of an incompatible model.

[JeisonJimenezA]

I'm loading this model ----> TheBloke/sqlcoder-GGUF

[jllllll]

What version of llama-cpp-python are you using?

[JeisonJimenezA]

llama_cpp_python 0.2.11+cu117

[jllllll]

Yeah, just finished downloading and got the same error. There may be something wrong with the model.
Not much I can do on my end as I'm only building wheels for llama-cpp-python. As far as I can tell, the issue is with llama.cpp itself.

[jllllll]

Could simply be that StarCoder models aren't supported with CUDA? Not sure.
I do know that only some model types are supported by the cuBLAS implementation.

[jllllll]

This does seem to be the case: ggml-org/llama.cpp#3187 (comment)
Guess CUDA support for it just hasn't been done yet.

[JeisonJimenezA]

Thank you for your help. Where can I see supported models for CUDA?

[jllllll]

Only thing I can find so far is this in the source code:
https://github.com/ggerganov/llama.cpp/blob/b8fe4b5cc9cb237ca98e5bc51b5d189e3c446d13/llama.cpp#L5840-L5844

The REFACT and MPT entries are new model arch support that isn't present yet in the current version of llama-cpp-python.
That leaves current llama-cpp-python cuBLAS support at these models:

LLAMA
BAICHUAN
FALCON