You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+17-23Lines changed: 17 additions & 23 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,8 +1,8 @@
1
1
# cuMLPrims
2
2
3
-
This repository contains C++ and CUDA code of muti-node multi-GPU (MNMG) ML mathematical primitives and some algorithms, that are used by [the main cuML project](https://github.com/rapidsai/cuml). The build system uses CMake for build configuration, and an out-of-source build is recommended.
3
+
This repository contains C++ and CUDA code of muti-node multi-GPU (MNMG) ML mathematical primitives and some algorithms, that are used by [the main cuML project](https://github.com/rapidsai/cuml). The build system uses CMake for build configuration, and an out-of-source build is recommended.
4
4
5
-
As of version 0.13, the MNMG code included in cuMLPrims follows the model one-process-per-GPU (OPG), where the code uses a communication library (based on cuML's comms) and each process has one GPU assigned to it. This is in contrast to single-process-multi-GPU (SPMG) approaches, which are no longer part of the code base.
5
+
The MNMG code included in cuMLPrims follows the model one-process-per-GPU (OPG), where the code uses a communication library (based on cuML's comms) and each process has one GPU assigned to it. This is in contrast to single-process-multi-GPU (SPMG) approaches, which are no longer part of the code base.
6
6
7
7
## Folder Structure
8
8
@@ -14,30 +14,28 @@ The folder structure mirrors closely the structure of github cuML. The folders a
14
14
-`cpp/cmake`: CMake related scripts.
15
15
-`cpp/include`: The include folder for headers that are necessary to be installed/distributed to use the libcumlprims.so artifact by users of the library.
16
16
-`cpp/src_prims_opg`: Contains source code for MNMG ML primitives. It also contains source code for algorithms that use the primitives that are still included in cuMLPrims as opposed to cuML.
17
-
-`cpp/test`: Googletest based unit tests (work in progress).
17
+
-`cpp/test`: Googletest based unit tests.
18
18
19
19
## Building cuMLPrims:
20
20
21
21
### Requirements
22
22
23
23
The main artifact produced by the build system is the shared library libcumlprims. Ensure the following dependencies are satisfied:
24
24
25
-
1. CMake (>= 3.14.5)
26
-
2. CUDA (>= 10.0)
27
-
3. GCC (>= 5.4.0)
28
-
4. NCCL (>= 2.4.6.1)
29
-
5. UCX with CUDA support (optional)(>=1.7) - enables point-to-point messaging in the cuML communicator.
30
-
6. zlib
25
+
1. CMake (>= 3.30.4)
26
+
2. CUDA (>= 12.0)
27
+
3. GCC (>= 14)
28
+
4. NCCL (>= 2.5)
31
29
32
-
It is recommended to use conda for environment/package management. If doing so, it is recommended to use the convenience environment .yml file located in [**the cuML repository**, in `conda/environments/cuml_dev_cudax.y.yml` (replace x.y for your CUDA version)](https://github.com/rapidsai/cuml/tree/branch-0.14/conda/environments). This file contains most of the dependencies ment1ioned above (notable exceptions are gcc and zlib). To use it, for example to create an environment named cuml_dev for CUDA 10.0 and Python 3.7, you can use the follow command:
30
+
It is recommended to use conda for environment/package management. See `conda/environments/`for available environment files.
As a convenience, a `build.sh` script is provided which can be used to execute the build commands in an automated manner. Note that the libraries will be installed to the location set in `$INSTALL_PREFIX` if set (i.e. `export INSTALL_PREFIX=/install/path`), otherwise to `$CONDA_PREFIX`.
38
+
As a convenience, a `build.sh` script is provided which can be used to execute the build commands in an automated manner. Note that the libraries will be installed to the location set in `$INSTALL_PREFIX` if set (i.e. `export INSTALL_PREFIX=/install/path`), otherwise to `$CONDA_PREFIX`.
41
39
```bash
42
40
$ ./build.sh # build the libcuml library and tests
43
41
# install them to $INSTALL_PREFIX if set, otherwise $CONDA_PREFIX
@@ -46,12 +44,12 @@ $ ./build.sh # build the libcuml library and tests
46
44
Other `build.sh` options:
47
45
48
46
```bash
49
-
$ ./build.sh clean # remove any prior build artifacts and configuration (start over)
47
+
$ ./build.sh clean # remove any prior build artifacts and configuration (start over)
50
48
$ ./build.sh libcumlprims -v # build and install libcumlprims with verbose output
51
49
$ ./build.sh libcumlprims -g # build and install libcumlprims for debug
52
50
$ PARALLEL_LEVEL=4 ./build.sh libcumlprims # build and install libcumlprims limiting parallel build jobs to 4 (make -j4)
53
-
$ ./build.sh libcuml -n # build libcuml but do not install
54
-
$ ./build.sh libcumlprims --allgpuarch # build the tests for all supported GPU architectures
51
+
$ ./build.sh libcuml -n # build libcuml but do not install
52
+
$ ./build.sh libcumlprims --allgpuarch # build the tests for all supported GPU architectures
55
53
56
54
```
57
55
@@ -107,11 +105,7 @@ cuMLPrims CMake has the following configurable flags available:
107
105
108
106
| Flag | Possible Values | Default Value | Behavior |
109
107
| --- | --- | --- | --- |
110
-
| BUILD_OPG_TESTS |[ON, OFF]| OFF | Build MPI cumlcomms based C++ unit tests (in progress, refer to step 4. of the build steps). |
111
-
| BUILD_CUMLPRIMS_LIBRARY |[ON, OFF]| ON | Enable/disable building libcumprims shared library. |
112
-
| CMAKE_CXX11_ABI |[ON, OFF]| ON | Enable/disable the GLIBCXX11 ABI |
113
-
| DISABLE_OPENMP |[ON, OFF]| OFF | Set to `ON` to disable OpenMP |
114
-
| KERNEL_INFO |[ON, OFF]| OFF | Enable/disable kernel resource usage info in nvcc. |
115
-
| LINE_INFO |[ON, OFF]| OFF | Enable/disable lineinfo in nvcc. |
116
-
| NVTX |[ON, OFF]| OFF | Enable/disable nvtx markers in libcumlprims. |
117
-
| GPU_ARCHS | List of GPU architectures, semicolon-separated | 60;70;75 | List of GPU architectures that all artifacts are compiled for. |
108
+
| BUILD_OPG_TESTS |[ON, OFF]| OFF | Build MPI cumlcomms based C++ unit tests (in progress, refer to step 4. of the build steps). |
109
+
| BUILD_CUMLPRIMS_LIBRARY |[ON, OFF]| ON | Enable/disable building libcumlprims shared library. |
110
+
| DISABLE_OPENMP |[ON, OFF]| OFF | Set to `ON` to disable OpenMP |
111
+
| NVTX |[ON, OFF]| OFF | Enable/disable nvtx markers in libcumlprims.|
0 commit comments