[cmake] use wfn91's linear algebra discovery modules #254

evaleev · 2021-02-11T14:40:04Z

No description provided.

…p version to fix problems in BLACS/ScaLAPACK discovery (#256) + fixed associated API changes

wavefunction91 · 2021-05-05T22:04:02Z

@evaleev This passes all CI except for Travis + clang (which is not a problem in the build, it's a problem in the Ubuntu packages). Do we want to fix the Travis build, or take Gitlab as being indicative of it working?

…aleevGroup/tiledarray into evaleev/feature/wfn91-linalg-discovery

…ed, if needed

@wavefunction91

@wavefunction91 would be nice to have some docs for the discovery kit to avoid duplication of effort ... BTW is `BLA_STATIC` respected by your kit?

evaleev · 2021-05-16T14:44:22Z

@wavefunction91 I think this is ready. I'm pretty sure my updates of the dox (INSTALL.md) are not 100% correct (does your kit respect BLA_STATIC?), but are mostly there

wavefunction91 · 2021-05-17T18:01:56Z

We have BLAS_PREFERS_STATIC, but I can add a BLA_STATIC compatibility if you'd prefer

wavefunction91 · 2021-05-17T18:28:37Z

INSTALL.md

+  - standard CMake BLAS/LAPACK modules.
+
+  The latter is used if CMake cache variable `BLA_VENDOR` is specified:
+  - `BLA_VENDOR` -- controls which vendor BLAS/LAPACK library will be sought


We don't use BLA_VENDOR, we set a priority list BLAS_PREFERENCE_LIST which will have the same effect as BLA_VENDOR when set to a single item. We do this to decouple LAPACK/BLAS (e.g. in the case of BLIS + NETLIB / BLIS + FLAME), but due to the hierarchy discovery, allowing full discovery (e.g. setting BLAS_PREFERENCE_LIST="IntelMKL" and letting FindLAPACK satisfy the FindBLAS dependency and recognize that it contains a LAPACK linker) will do the right thing.

We could add a BLA_VENDOR <-> BLAS_PREFERENCE_LIST compatibility, or you propagate in the VG CMake kit, which ever you prefer

I think it is least surprising to emulate Find{BLAS,LAPACK} as much as reasonable, no?

i.e. I think it's reasonable to read same input vars as those, i.e. read BLA_{VENDOR,STATIC}

I suppose that is the least surprising behaviour, I'll patch that today

INSTALL.md

wavefunction91 · 2021-05-17T19:25:41Z

INSTALL.md

-Fortunately, Intel MKL can be discovered by BLAS++/LAPACK++ automatically in most instances; if needed, specifying `BLA_VENDOR` with [appropriate argument](https://cmake.org/cmake/help/latest/module/FindBLAS.html#input-variables) can be used to force TiledArray to use MKL. Unfortunately it is not possible to specify the use of TBB-based backend for MKL without the use of a toolchain file. All MKL-enabled toolchains in [The Valeev Group CMake kit](https://github.com/ValeevGroup/kit-cmake/tree/master/toolchains) can be used to configure TiledArray to use sequential, OpenMP, or TBB backend by setting the `MKL_THREADING` CMake cache variable to `SEQ`, `OMP`, or `TBB`, respectively. The toolchains also respect the user-provided choice of `BLA_STATIC`. If multiple MKL versions are present on your system, specify the apropriate variant of the library by loading the corresponding `mklvars.sh` script to set environment variables `MKLROOT` and, if necessary, `LD_LIBRARY_PATH`/`DYLD_LIBRARY_PATH`.
+To discover and configure the use of Intel MKL consider these suggestions:
+- The use of NWChemEx discovery kit is strongly recommended for discovering Intel MKL. The following CMake cache variables can be used to specify the desired Intel MKL configuration:
+  - `intelmkl_PREFERS_STATIC`: whether to look for static or shared/dynamic libraries (default = `OFF`)


This would be covered by BLAS_PREFERS_STATIC

wavefunction91 · 2021-05-20T17:27:27Z

@evaleev Kitware compatibility is added in wavefunction91/linalg-cmake-modules@510b928 , we should just need to bump the version in vg-kit. This will allow for BLA_STATIC -> {BLAS,LAPACK,ScaLAPACK}_PREFERS_STATIC emulation. Same goes for BLA_VENDOR, but as we've discussed offline, this generally leads to ambiguity in the cases where BLAS/LAPACK interoperability is desired

The following are the allowed specs for XXX_PREFERENCE_LIST:

XXX=BLAS

ReferenceBLAS (NETLIB)
IntelMKL
IBMESSL
BLIS
OpenBLAS
Accelerate

XXX=LAPACK

ReferenceLAPACK (NETLIB)
FLAME

evaleev · 2021-05-20T19:30:27Z

@wavefunction91 OK, thanks, will do

per discussion here ValeevGroup/tiledarray#254

evaleev · 2021-05-25T13:15:28Z

@evaleev Kitware compatibility is added in wavefunction91/linalg-cmake-modules@510b928 , we should just need to bump the version in vg-kit.

done

The following are the allowed specs for XXX_PREFERENCE_LIST:

XXX=BLAS

ReferenceBLAS (NETLIB)

IntelMKL

IBMESSL

BLIS

OpenBLAS

Accelerate

XXX=LAPACK

ReferenceLAPACK (NETLIB)

FLAME

Does the user need to specify {LAPACK,ScaLAPACK}_PREFERENCE_LIST in the case BLAS_PREFERENCE_LIST= IntelMKL? I.e. does the latter imply that only IntelMKL will be considered when searching for LAPACK and ScaLAPACK?

wavefunction91 · 2021-05-25T16:20:27Z

@evaleev The user should not have to specify {Sca,}LAPACK_PREFERENCE_LIST for MKL. Take ScaLAPACK for example (since its the highest up on the stack) - if unguided discovery is used, then it will go like

FindScaLAPACK -> no ScaLAPACK_LIBRARIES are set, so it looks for LAPACK
FindLAPACK -> no LAPACK_LIBRARIES are set, looks for BLAS
FindBLAS -> no BLAS_LIBRARIES are set, goes through BLAS_PREFERENCE_LIST, finds IntelMKL, propagates IntelMKL_LIBRARIES to BLAS_LIBRARIES. Because ScaLAPACK was requested, it also tacks on the BLACS/ScaLAPACK MKL linker (+ MPI), which is non-trivial (which is why it's important to ask for the highest level of the stack desired so it can propagate down to Find<VENDOR> if need be)
FindLAPACK -> recognizes that it found BLAS, checks for LAPACK linker in BLAS_LIBRARIES, this will be successful for MKL and will propagate BLAS_LIBRARIES (and associated variables) to LAPACK_LIBRARIES
FindScaLAPACK -> recognizes that it found LAPACK, checks for ScaLAPACK linker in LAPACK_LIBRARIES (which will work for MKL) and propagates to ScaLAPACK_LIBRARIES, etc.

evaleev · 2021-05-25T17:03:11Z

@wavefunction91 sorry for wasting your time ... since the proper thing will be to interpret BLA_VENDOR rather than just passing it on as BLAS_PREFERENCE_LIST it's probably best to just use your kit's vars directly ... only realized this after seeing what happens when BLA_VENDOR used with standard CMake module is used with your kit: https://gitlab.com/ValeevGroup/tiledarray/-/jobs/1291849639#L538 ... I propose we avoid the use of BLA_VENDOR to avoid these kinds of issues.

evaleev and others added 7 commits February 9, 2021 13:24

[cmake] use wfn91's linear algebra discovery modules

07e50e3

Merge branch 'master' into evaleev/feature/wfn91-linalg-discovery

b6478d0

Fixes to invocation of DBWY's linalg CMake modules, bumped scalapackp…

f4da362

…p version to fix problems in BLACS/ScaLAPACK discovery (#256) + fixed associated API changes

Update Gitlab CI with ScaLAPACK conventions in DBWY's linalg modules

e42558a

Add BLAS/LAPACK linker to ScaLAPACK spec in Gitlab CI

bc24349

Tell ScaLAPACK to link to MPI in Gitlab CI

ea0ccb7

Bump scalapack++ version

ca40628

evaleev added 6 commits May 15, 2021 09:51

missing lapack.hh include in rank-local.h

620f6a4

Merge branch 'master' into evaleev/feature/wfn91-linalg-discovery

1732b1b

Merge branch 'evaleev/feature/wfn91-linalg-discovery' of github.com:V…

ff74faf

…aleevGroup/tiledarray into evaleev/feature/wfn91-linalg-discovery

typo

63e1371

[cmake] the use of nwx discovery kit for linear algebra can be disabl…

80e3bad

…ed, if needed

updated dox to discuss linear algebra discovery kits

8a83148

@wavefunction91 would be nice to have some docs for the discovery kit to avoid duplication of effort ... BTW is `BLA_STATIC` respected by your kit?

evaleev requested a review from wavefunction91 May 16, 2021 14:44

point vg cmake kit back to master which contains the module

763c5a8

wavefunction91 reviewed May 17, 2021

View reviewed changes

evaleev added a commit to ValeevGroup/kit-cmake that referenced this pull request May 25, 2021

point to wfn91's fork of the nwx discovery kit

a09f790

per discussion here ValeevGroup/tiledarray#254

evaleev added 2 commits May 25, 2021 09:39

[cmake] point to vg cmake kit version using nwx's linalg discovery kit

48a4412

updated dox related to linear algebra discovery

3f8a4d1

evaleev force-pushed the evaleev/feature/wfn91-linalg-discovery branch from 3989d45 to 3f8a4d1 Compare May 25, 2021 13:39

no longer support the use of BLA_VENDOR with NWX's linalg discovery kit

2e02078

evaleev force-pushed the evaleev/feature/wfn91-linalg-discovery branch from e5b8bcc to 2e02078 Compare May 25, 2021 18:04

evaleev merged commit e7f8f90 into master May 25, 2021

evaleev deleted the evaleev/feature/wfn91-linalg-discovery branch May 25, 2021 20:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[cmake] use wfn91's linear algebra discovery modules #254

[cmake] use wfn91's linear algebra discovery modules #254

Uh oh!

evaleev commented Feb 11, 2021

Uh oh!

wavefunction91 commented May 5, 2021

Uh oh!

evaleev commented May 16, 2021

Uh oh!

wavefunction91 commented May 17, 2021

Uh oh!

wavefunction91 May 17, 2021

Uh oh!

evaleev May 17, 2021

Uh oh!

evaleev May 17, 2021

Uh oh!

wavefunction91 May 18, 2021

Uh oh!

Uh oh!

wavefunction91 May 17, 2021

Uh oh!

wavefunction91 commented May 20, 2021

Uh oh!

evaleev commented May 20, 2021

Uh oh!

evaleev commented May 25, 2021

Uh oh!

wavefunction91 commented May 25, 2021

Uh oh!

evaleev commented May 25, 2021

Uh oh!

Uh oh!

[cmake] use wfn91's linear algebra discovery modules #254

[cmake] use wfn91's linear algebra discovery modules #254

Uh oh!

Conversation

evaleev commented Feb 11, 2021

Uh oh!

wavefunction91 commented May 5, 2021

Uh oh!

evaleev commented May 16, 2021

Uh oh!

wavefunction91 commented May 17, 2021

Uh oh!

wavefunction91 May 17, 2021

Choose a reason for hiding this comment

Uh oh!

evaleev May 17, 2021

Choose a reason for hiding this comment

Uh oh!

evaleev May 17, 2021

Choose a reason for hiding this comment

Uh oh!

wavefunction91 May 18, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wavefunction91 May 17, 2021

Choose a reason for hiding this comment

Uh oh!

wavefunction91 commented May 20, 2021

Uh oh!

evaleev commented May 20, 2021

Uh oh!

evaleev commented May 25, 2021

Uh oh!

wavefunction91 commented May 25, 2021

Uh oh!

evaleev commented May 25, 2021

Uh oh!

Uh oh!