-
Notifications
You must be signed in to change notification settings - Fork 4.5k
Open
Labels
performanceCPU and memory usage - results and comparisonsCPU and memory usage - results and comparisons
Description
Encoder
Collection of bench results for various platforms and devices.
If you want to submit info about your device, simply run the bench tool or the extra/bench-all.sh and report the results in the comments below.
Suggestions for better summary of the results are welcome
CPU | OS | Config | Model | Th | Load | Enc. | Commit |
---|---|---|---|---|---|---|---|
MacBook M1 Pro | MacOS 13.0.1 | NEON BLAS | tiny | 8 | 71 | 102 | 206fc93 |
MacBook M1 Pro | MacOS 13.0.1 | NEON BLAS | base | 8 | 96 | 220 | 206fc93 |
MacBook M1 Pro | MacOS 13.0.1 | NEON BLAS | small | 8 | 233 | 685 | 206fc93 |
MacBook M1 Pro | MacOS 13.0.1 | NEON BLAS | medium | 8 | 603 | 1928 | 206fc93 |
MacBook M1 Pro | MacOS 13.0.1 | NEON BLAS | large | 8 | 1158 | 3350 | 206fc93 |
--- | |||||||
MacBook M1 Pro | MacOS 13.0.1 | NEON BLAS | small | 1 | 251 | 2605 | 206fc93 |
MacBook M1 Pro | MacOS 13.0.1 | NEON BLAS | small | 4 | 255 | 884 | 206fc93 |
--- | |||||||
Mac Mini M1 | MacOS | NEON BLAS | tiny | 4 | 62 | 194 | fcf515d |
Mac Mini M1 | MacOS | NEON BLAS | base | 4 | 81 | 380 | fcf515d |
Mac Mini M1 | MacOS | NEON BLAS | small | 4 | 204 | 1249 | fcf515d |
Mac Mini M1 | MacOS | NEON BLAS | medium | 4 | 876 | 3980 | fcf515d |
Mac Mini M1 | MacOS | NEON BLAS | large | 4 | 1876 | 7979 | fcf515d |
--- | |||||||
Ryzen 9 3900X | Ubuntu 20.04 | AVX2 | tiny | 8 | 107 | 422 | fcf515d |
Ryzen 9 3900X | Ubuntu 20.04 | AVX2 | base | 8 | 137 | 880 | fcf515d |
Ryzen 9 3900X | Ubuntu 20.04 | AVX2 | small | 8 | 280 | 2874 | fcf515d |
Ryzen 9 3900X | Ubuntu 20.04 | AVX2 | medium | 8 | 692 | 9610 | fcf515d |
Ryzen 9 3900X | Ubuntu 20.04 | AVX2 | large | 8 | 1317 | 16917 | fcf515d |
--- | |||||||
Ryzen 9 3900X | Ubuntu 20.04 | AVX2 BLAS | tiny | 4 | 120 | 780 | fcf515d |
Ryzen 9 3900X | Ubuntu 20.04 | AVX2 BLAS | base | 4 | 151 | 1173 | fcf515d |
Ryzen 9 3900X | Ubuntu 20.04 | AVX2 BLAS | small | 4 | 289 | 3062 | fcf515d |
Ryzen 9 3900X | Ubuntu 20.04 | AVX2 BLAS | medium | 4 | 711 | 9175 | fcf515d |
Ryzen 9 3900X | Ubuntu 20.04 | AVX2 BLAS | large | 4 | 1282 | 16050 | fcf515d |
--- | |||||||
Ryzen 9 5950X | Ubuntu 22.04 | AVX2 | tiny | 8 | 135 | 197 | fcf515d |
Ryzen 9 5950X | Ubuntu 22.04 | AVX2 | base | 8 | 176 | 421 | fcf515d |
Ryzen 9 5950X | Ubuntu 22.04 | AVX2 | small | 8 | 357 | 1393 | fcf515d |
Ryzen 9 5950X | Ubuntu 22.04 | AVX2 | medium | 8 | 855 | 4404 | fcf515d |
Ryzen 9 5950X | Ubuntu 22.04 | AVX2 | large | 8 | 1576 | 8118 | fcf515d |
--- | |||||||
Raspberry Pi 4 | NEON | tiny | 4 | 1436 | 13839 | fcf515d | |
Raspberry Pi 4 | NEON | base | 4 | 1894 | 30552 | fcf515d | |
--- | |||||||
iPhone 13 Mini | iOS 16.0 | NEON BLAS | base | 4 | 97 | 1091 | fcf515d |
--- | |||||||
MacBook M1 Pro | Vivaldi | WASM | tiny | 8 | 133 | 3785 | fcf515d |
MacBook M1 Pro | Vivaldi | WASM | base | 8 | 172 | 8253 | fcf515d |
--- | |||||||
MacBook M1 Pro | Chrome | WASM | tiny | 8 | 134 | 3776 | fcf515d |
MacBook M1 Pro | Chrome | WASM | base | 8 | 168 | 8200 | fcf515d |
--- | |||||||
MacBook M1 Pro | Firefox | WASM | tiny | 8 | 137 | 2626 | fcf515d |
MacBook M1 Pro | Firefox | WASM | base | 8 | 183 | 6226 | fcf515d |
memcpy
MacBook M1 Pro
./bench -w 1 -t 1
memcpy: 37.59 GB/s
Ryzen 9 5950X
./bench -w 1 -t 1
memcpy: 16.74 GB/s
ggml_mul_mat
MacBook M1 Pro
./bench -w 2 -t 1
ggml_mul_mat: 64 x 64: F16 330.6 GFLOPS (128 runs) / F32 466.0 GFLOPS (128 runs)
ggml_mul_mat: 128 x 128: F16 737.5 GFLOPS (128 runs) / F32 838.9 GFLOPS (128 runs)
ggml_mul_mat: 256 x 256: F16 938.6 GFLOPS (128 runs) / F32 1062.3 GFLOPS (128 runs)
ggml_mul_mat: 512 x 512: F16 1312.5 GFLOPS (128 runs) / F32 1835.5 GFLOPS (128 runs)
ggml_mul_mat: 1024 x 1024: F16 1765.1 GFLOPS (128 runs) / F32 2041.4 GFLOPS (128 runs)
ggml_mul_mat: 2048 x 2048: F16 1784.3 GFLOPS (104 runs) / F32 1859.2 GFLOPS (109 runs)
ggml_mul_mat: 4096 x 4096: F16 1855.1 GFLOPS ( 14 runs) / F32 1873.3 GFLOPS ( 14 runs)
Ryzen 9 5950X
WHISPER_OPENBLAS=1 make -j bench && ./bench -w 2 -t 1
ggml_mul_mat: 64 x 64: F16 56.3 GFLOPS (128 runs) / F32 70.2 GFLOPS (128 runs)
ggml_mul_mat: 128 x 128: F16 47.8 GFLOPS (128 runs) / F32 67.0 GFLOPS (128 runs)
ggml_mul_mat: 256 x 256: F16 185.1 GFLOPS (128 runs) / F32 332.7 GFLOPS (128 runs)
ggml_mul_mat: 512 x 512: F16 386.4 GFLOPS (128 runs) / F32 658.6 GFLOPS (128 runs)
ggml_mul_mat: 1024 x 1024: F16 636.2 GFLOPS (128 runs) / F32 1012.0 GFLOPS (128 runs)
ggml_mul_mat: 2048 x 2048: F16 950.9 GFLOPS ( 56 runs) / F32 1296.8 GFLOPS ( 76 runs)
ggml_mul_mat: 4096 x 4096: F16 1168.6 GFLOPS ( 9 runs) / F32 1403.1 GFLOPS ( 11 runs)
matijagrcic, swoook, dohyung-kim-moreh and jeffzhou2000
Metadata
Metadata
Assignees
Labels
performanceCPU and memory usage - results and comparisonsCPU and memory usage - results and comparisons