Custom Diffusion retrieve.py

[rezkanas]

Describe the bug

when running retrieve.py to produce class prior photo using laion datasset,, I run through below errors.
url is bad in the code below... could be the root cause?

client = ClipClient(
        url="https://knn.laion.ai/knn-service", indice_name="laion5B-L-14", num_images=num_images, aesthetic_weight=0.1
    )

Reproduction

!python retrieve.py --class_prompt person --class_data_dir /mnt/c/Users/noobw/PycharmProjects/pythonProject/custom_diffusion_anas/class_prior --num_class_images 200

Logs

/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Traceback (most recent call last):
  File "/home/anasrezklinux/.local/lib/python3.10/site-packages/requests/models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/anasrezklinux/test_pycharm_link/diffusers/examples/custom_diffusion/retrieve.py", line 87, in <module>
    retrieve(args.class_prompt, args.class_data_dir, args.num_class_images)
  File "/home/anasrezklinux/test_pycharm_link/diffusers/examples/custom_diffusion/retrieve.py", line 37, in retrieve
    class_images = client.query(text=class_prompt)
  File "/home/anasrezklinux/.local/lib/python3.10/site-packages/clip_retrieval/clip_client.py", line 84, in query
    return self.__search_knn_api__(text=text)
  File "/home/anasrezklinux/.local/lib/python3.10/site-packages/clip_retrieval/clip_client.py", line 153, in __search_knn_api__
    ).json()
  File "/home/anasrezklinux/.local/lib/python3.10/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

System Info

• diffusers version: 0.26.1
• Platform: Linux-5.15.133.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
• Python version: 3.10.12
• PyTorch version (GPU?): 2.2.0+cu121 (True)
• Huggingface_hub version: 0.20.3
• Transformers version: 4.37.0
• Accelerate version: 0.25.0
• xFormers version: 0.0.24
• Using GPU in script?:
• Using distributed or parallel set-up in script?:

Who can help?

@sayakpaul @patrickvonplaten

[Ashutosh7982]

The website "https://knn.laion.ai/knn-service is down

[rezkanas]

I realize that but then how can retrieve.py be run?

[sayakpaul]

Meaning?

[rezkanas]

@sayakpaul, I don't understand your response. website is down, do I need to wait? or probably the url is broken?

[OchiaiNao]

You can add these images manually with the following structure:

your_class_data_dir
├── images
│   ├── 01.jpg
│   ├── 02.jpg
├── caption.txt # captions line by line
└── images.txt # absolute path line by line

Also, remember the number of images should be the same as:

--num_class_images=200

[sayakpaul]

The website incurs ratelimiting issues because it's used by many folks. So, that is a constraint.

[yiyixuxu]

It this fixed?

[sayakpaul]

@yiyixuxu it depends on the service that retrieve.py relies on. We cannot do much here.

[TruongQuocTruong]

I also has the same issue

[daeunni]

Same error. Do u guys can run retrieve.py now?

[WuTao-CS]

I have the same issue,

[sayakpaul]

The retrieve.py script relies on external services. Commenting with the same thing won’t help the issue :-)

[Sakura-hub47]

I have the same issue, are there any alternative URLs available?

[sayakpaul]

I don't think there is.