Help me... #745
Description
Starting the web UI...
Warning: --gptq_bits is deprecated and will be removed. Use --wbits instead.
Warning: --gptq_pre_layer is deprecated and will be removed. Use --prelayer instead.
===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
CUDA SETUP: Loading binary C:\Users\RGWyo\AppData\Local\Programs\Python\Python310\lib\site-packages\bitsandbytes\libbitsandbytes_cpu.dll...
C:\Users\RGWyo\AppData\Local\Programs\Python\Python310\lib\site-packages\bitsandbytes\cextension.py:31: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable.
warn("The installed version of bitsandbytes was compiled without GPU support. "
The following models are available:
1. alpaca-13b
2. chatgpt4all
3. codegen-6B-multi
4. llama-13b-hf-int4
5. llama-7b-hf
6. llama-7b-hf-int4
7. rugpt3large_based_on_gpt2
Which one do you want to load? 1-7
6
Loading llama-7b-hf-int4...
CUDA extension not installed.
Loading model ...
Traceback (most recent call last):
File "F:\WBC\text-generation-webui\text-generation-webui\server.py", line 276, in <module>
shared.model, shared.tokenizer = load_model(shared.model_name)
File "F:\WBC\text-generation-webui\text-generation-webui\modules\models.py", line 102, in load_model
model = load_quantized(model_name)
File "F:\WBC\text-generation-webui\text-generation-webui\modules\GPTQ_loader.py", line 114, in load_quantized
model = load_quant(str(path_to_model), str(pt_path), shared.args.wbits, shared.args.groupsize, kernel_switch_threshold=threshold)
File "F:\WBC\text-generation-webui\text-generation-webui\modules\GPTQ_loader.py", line 45, in _load_quant
model.load_state_dict(torch.load(checkpoint))
File "C:\Users\RGWyo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\serialization.py", line 809, in load
return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
File "C:\Users\RGWyo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\serialization.py", line 1172, in _load
result = unpickler.load()
File "C:\Users\RGWyo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\serialization.py", line 1142, in persistent_load
typed_storage = load_tensor(dtype, nbytes, key, _maybe_decode_ascii(location))
File "C:\Users\RGWyo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\serialization.py", line 1116, in load_tensor
wrap_storage=restore_location(storage, location),
File "C:\Users\RGWyo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\serialization.py", line 217, in default_restore_location
result = fn(storage, location)
File "C:\Users\RGWyo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\serialization.py", line 182, in _cuda_deserialize
device = validate_cuda_device(location)
File "C:\Users\RGWyo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\serialization.py", line 166, in validate_cuda_device
raise RuntimeError('Attempting to deserialize object on a CUDA '
RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.
I just installed webui in clear folder.
Win11