Skip to content

Commit baa64fd

Browse files
committed
Update README.md and QAnything Startup Usage
1 parent 3fc7bb6 commit baa64fd

File tree

3 files changed

+74
-5
lines changed

3 files changed

+74
-5
lines changed

README.md

Lines changed: 35 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -166,9 +166,7 @@ git clone https://github.com/netease-youdao/QAnything.git
166166
### step2: Enter the project root directory and execute the startup script.
167167
* [📖 QAnything_Startup_Usage](docs/QAnything_Startup_Usage_README.md)
168168
* Get detailed usage of LLM interface by ```bash ./run.sh -h```
169-
170169

171-
If you are in the Windows11 system: Need to enter the **WSL** environment.
172170
```shell
173171
cd QAnything
174172
bash run.sh # Start on GPU 0 by default.
@@ -186,14 +184,48 @@ huggingfase: https://huggingface.co/netease-youdao/QAnything
186184
</details>
187185

188186
<details>
189-
<summary>(Optional) Specify GPU startup</summary>
187+
<summary>(Optional) Specify GPU startup </summary>
190188

191189
```shell
192190
cd QAnything
193191
bash ./run.sh -c local -i 0 -b default # gpu id 0
194192
```
195193
</details>
196194

195+
<details>
196+
<summary>(Optional) Specify GPU startup - Recommended for Windows10/Windows11 WSL2 User</summary>
197+
198+
```shell
199+
# For Windows OS: Need to enter the **WSL2** environment.
200+
# Step 1. Download the public LLM model (e.g., Qwen-7B-QAnything) and save to "/path/to/QAnything/assets/custom_models"
201+
# (Optional) Download Qwen-7B-QAnything from ModelScope: https://www.modelscope.cn/models/netease-youdao/Qwen-7B-QAnything
202+
# (Optional) Download Qwen-7B-QAnything from Huggingface: https://huggingface.co/netease-youdao/Qwen-7B-QAnything
203+
cd QAnything/assets/custom_models
204+
git clone https://huggingface.co/netease-youdao/Qwen-7B-QAnything
205+
206+
# Step 2. Execute the service startup command. Here we use "-b hf" to specify the Huggingface transformers backend.
207+
cd ../../
208+
bash ./run.sh -c local -i 0 -b hf -m Qwen-7B-QAnything -t qwen-7b-qanything
209+
```
210+
</details>
211+
212+
<details>
213+
<summary>(Optional) Specify GPU startup - Recommended for GPU Compute Capability >= 8.6 and VRAM >= 24GB</summary>
214+
215+
```shell
216+
# GPU Compute Capability: https://developer.nvidia.com/cuda-gpus
217+
# Step 1. Download the public LLM model (e.g., Qwen-7B-QAnything) and save to "/path/to/QAnything/assets/custom_models"
218+
# (Optional) Download Qwen-7B-QAnything from ModelScope: https://www.modelscope.cn/models/netease-youdao/Qwen-7B-QAnything
219+
# (Optional) Download Qwen-7B-QAnything from Huggingface: https://huggingface.co/netease-youdao/Qwen-7B-QAnything
220+
cd QAnything/assets/custom_models
221+
git clone https://huggingface.co/netease-youdao/Qwen-7B-QAnything
222+
223+
# Step 2. Execute the service startup command. Here we use "-b vllm" to specify the vllm backend.
224+
cd ../../
225+
bash ./run.sh -c local -i 0 -b vllm -m Qwen-7B-QAnything -t qwen-7b-qanything -p 1 -r 0.85
226+
```
227+
</details>
228+
197229
<details>
198230
<summary>(Optional) Specify multi-GPU startup </summary>
199231

README_zh.md

Lines changed: 34 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -157,7 +157,6 @@ git clone https://github.com/netease-youdao/QAnything.git
157157
* [📖 QAnything_Startup_Usage](docs/QAnything_Startup_Usage_README.md)
158158
* 执行 ```bash ./run.sh -h``` 获取详细的LLM服务配置方法
159159

160-
如果在Windows系统下请先进入**WSL**环境
161160
```shell
162161
cd QAnything
163162
bash run.sh # 默认在0号GPU上启动
@@ -183,6 +182,40 @@ bash ./run.sh -c local -i 0 -b default # 指定0号GPU启动 GPU编号从0开始
183182
```
184183
</details>
185184

185+
<details>
186+
<summary>(可选)指定单GPU启动 - 推荐 Windows10/Windows11 WSL2 用户使用此方式运行 QAnything</summary>
187+
188+
```shell
189+
# 注意: Windows系统请先进入**WSL2**环境
190+
# Step 1. 下载开源 LLM 模型 (e.g., Qwen-7B-QAnything) 并保存在路径 "/path/to/QAnything/assets/custom_models"
191+
# (可选) 从 ModelScope 下载 Qwen-7B-QAnything: https://www.modelscope.cn/models/netease-youdao/Qwen-7B-QAnything
192+
# (可选) 从 Huggingface 下载 Qwen-7B-QAnything: https://huggingface.co/netease-youdao/Qwen-7B-QAnything
193+
cd QAnything/assets/custom_models
194+
git clone https://huggingface.co/netease-youdao/Qwen-7B-QAnything
195+
196+
# Step 2. 执行启动命令,其中"-b hf"表示指定使用 Huggingface transformers 后端运行 LLM.
197+
cd ../../
198+
bash ./run.sh -c local -i 0 -b hf -m Qwen-7B-QAnything -t qwen-7b-qanything
199+
```
200+
</details>
201+
202+
<details>
203+
<summary>(可选)指定单GPU启动 - 推荐 GPU Compute Capability >= 8.6 && VRAM >= 24GB 使用此方式运行 QAnything</summary>
204+
205+
```shell
206+
# 查看 GPU 算力 GPU Compute Capability: https://developer.nvidia.com/cuda-gpus
207+
# Step 1. 下载开源 LLM 模型 (e.g., Qwen-7B-QAnything) 并保存在路径 "/path/to/QAnything/assets/custom_models"
208+
# (可选) 从 ModelScope 下载 Qwen-7B-QAnything: https://www.modelscope.cn/models/netease-youdao/Qwen-7B-QAnything
209+
# (可选) 从 Huggingface 下载 Qwen-7B-QAnything: https://huggingface.co/netease-youdao/Qwen-7B-QAnything
210+
cd QAnything/assets/custom_models
211+
git clone https://huggingface.co/netease-youdao/Qwen-7B-QAnything
212+
213+
# Step 2. 执行启动命令,其中"-b vllm"表示指定使用 vllm 后端运行 LLM.
214+
cd ../../
215+
bash ./run.sh -c local -i 0 -b vllm -m Qwen-7B-QAnything -t qwen-7b-qanything -p 1 -r 0.85
216+
```
217+
</details>
218+
186219
<details>
187220
<summary>(可选)指定多GPU启动</summary>
188221

docs/QAnything_Startup_Usage_README.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,8 @@ Note: You can choose the most suitable Service Startup Command based on your own
6161
#### 1.1 Run Qwen-7B-QAnything
6262
```bash
6363
## Step 1. Download the public LLM model (e.g., Qwen-7B-QAnything) and save to "/path/to/QAnything/assets/custom_models"
64+
## (Optional) Download Qwen-7B-QAnything from ModelScope: https://www.modelscope.cn/models/netease-youdao/Qwen-7B-QAnything
65+
## (Optional) Download Qwen-7B-QAnything from Huggingface: https://huggingface.co/netease-youdao/Qwen-7B-QAnything
6466
cd /path/to/QAnything/assets/custom_models
6567
git clone https://huggingface.co/netease-youdao/Qwen-7B-QAnything
6668

@@ -88,10 +90,12 @@ bash ./run.sh -c local -i 0 -b hf -m MiniChat-2-3B -t minichat
8890
#### 2.1 Run Qwen-7B-QAnything
8991
```bash
9092
## Step 1. Download the public LLM model (e.g., Qwen-7B-QAnything) and save to "/path/to/QAnything/assets/custom_models"
93+
## (Optional) Download Qwen-7B-QAnything from ModelScope: https://www.modelscope.cn/models/netease-youdao/Qwen-7B-QAnything
94+
## (Optional) Download Qwen-7B-QAnything from Huggingface: https://huggingface.co/netease-youdao/Qwen-7B-QAnything
9195
cd /path/to/QAnything/assets/custom_models
9296
git clone https://huggingface.co/netease-youdao/Qwen-7B-QAnything
9397

94-
## Step 2. Execute the service startup command. Here we use "-b vllm" to specify the Huggingface transformers backend.
98+
## Step 2. Execute the service startup command. Here we use "-b vllm" to specify the vllm backend.
9599
## Here we use "-b vllm" to specify the vllm backend that will do bf16 inference as default.
96100
## Note you should adjust the gpu_memory_utilization yourself according to the model size to avoid out of memory (e.g., gpu_memory_utilization=0.81 is set default for 7B. Here, gpu_memory_utilization is set to 0.85 by "-r 0.85").
97101
cd /path/to/QAnything

0 commit comments

Comments
 (0)