Skip to content

Commit 190c66b

Browse files
authored
Add Qwen3 1.7, 4B, 8B, and 32B support to from-scratch nb (#709)
1 parent 2f53bf5 commit 190c66b

File tree

3 files changed

+246
-78
lines changed

3 files changed

+246
-78
lines changed

.gitignore

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,14 @@ ch05/10_llm-training-speed/loss.pdf
5555
ch05/10_llm-training-speed/model.pth
5656
ch05/11_qwen3/Qwen3-0.6B
5757
ch05/11_qwen3/Qwen3-0.6B-Base
58+
ch05/11_qwen3/Qwen3-1.7B
59+
ch05/11_qwen3/Qwen3-1.7B-Base
60+
ch05/11_qwen3/Qwen3-4B
61+
ch05/11_qwen3/Qwen3-4B-Base
62+
ch05/11_qwen3/Qwen3-8B
63+
ch05/11_qwen3/Qwen3-8B-Base
64+
ch05/11_qwen3/Qwen3-32B
65+
ch05/11_qwen3/Qwen3-32B-Base
5866

5967
ch06/01_main-chapter-code/gpt2
6068
ch06/02_bonus_additional-experiments/gpt2

ch05/11_qwen3/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
# Qwen3 From Scratch
22

3-
This [standalone-qwen3.ipynb](standalone-qwen3.ipynb) Jupyter notebook in this folder contains a from-scratch implementation of Qwen3 0.6B.
3+
This [standalone-qwen3.ipynb](standalone-qwen3.ipynb) Jupyter notebook in this folder contains a from-scratch implementation of Qwen3 0.6B, 1.7B, 4B, 8B, and 32 B.
44

55
<img src="https://sebastianraschka.com/images/LLMs-from-scratch-images/bonus/qwen/qwen-overview.webp">
66

77

88
&nbsp;
99
### Using Qwen3 0.6B via the `llms-from-scratch` package
1010

11-
For an easy way to use the Qwen3 from-scratch implementation, you can also use the `llms-from-scratch` PyPI package based on the source code in this repository at [pkg/llms_from_scratch](../../pkg/llms_from_scratch).
11+
For an easy way to use the Qwen3 0.6B from-scratch implementation, you can also use the `llms-from-scratch` PyPI package based on the source code in this repository at [pkg/llms_from_scratch](../../pkg/llms_from_scratch).
1212

1313
&nbsp;
1414
#### 1) Installation

0 commit comments

Comments
 (0)