[BUG] Error when building MatX on Linux.

[HugoPhibbs]

Describe the Bug
I followed the given instructions to install MatX

To Reproduce
Steps to reproduce the behavior:

I ran (it fails on the make line):

cmake -DMATX_BUILD_TESTS=OFF -DMATX_BUILD_BENCHMARKS=ON -DMATX_BUILD_EXAMPLES=ON -DMATX_BUILD_DOCS=OFF ..
make -j 

Expected Behavior
Build with no errors

Code Snippets
I get the error:`

/home/hphi344/Documents/MatX/include/matx/kernels/channelize_poly.cuh(349): error: initializer not allowed for __shared__ variable 

/home/hphi344/Documents/MatX/include/matx/kernels/channelize_poly.cuh(349): error: initializer not allowed for __shared__ variable 

/home/hphi344/Documents/MatX/include/matx/kernels/channelize_poly.cuh(349): error: initializer not allowed for __shared__ variable 

/home/hphi344/Documents/MatX/include/matx/kernels/channelize_poly.cuh(349): error: initializer not allowed for __shared__ variable 

/home/hphi344/Documents/MatX/include/matx/kernels/channelize_poly.cuh(349): error: initializer not allowed for __shared__ variable 

/home/hphi344/Documents/MatX/include/matx/kernels/channelize_poly.cuh(64): error: initializer not allowed for __shared__ variable 

6 errors detected in the compilation of "/home/hphi344/Documents/MatX/examples/channelize_poly_bench.cu".
make[2]: *** [examples/CMakeFiles/channelize_poly_bench.dir/build.make:77: examples/CMakeFiles/channelize_poly_bench.dir/channelize_poly_bench.cu.o] Error 255
make[1]: *** [CMakeFiles/Makefile2:333: examples/CMakeFiles/channelize_poly_bench.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
^Cnvcc: nvcc: InterruptInterrupt

nvcc: Interrupt
nvcc: Interrupt
nvcc: Interrupt
make[2]: *** Deleting file 'bench/CMakeFiles/matx_bench.dir/00_transform/qr.cu.o'
make[2]: *** [bench/CMakeFiles/matx_bench.dir/build.make:122: bench/CMakeFiles/matx_bench.dir/00_transform/cub.cu.o] Interrupt
make[2]: *** [bench/CMakeFiles/matx_bench.dir/build.make:197: bench/CMakeFiles/matx_bench.dir/00_operators/reduction.cu.o] Interrupt
make[2]: *** [bench/CMakeFiles/matx_bench.dir/build.make:167: bench/CMakeFiles/matx_bench.dir/00_transform/qr.cu.o] Interrupt
make[2]: *** [bench/CMakeFiles/matx_bench.dir/build.make:152: bench/CMakeFiles/matx_bench.dir/00_transform/svd_power.cu.o] Interrupt
make[2]: *** [bench/CMakeFiles/matx_bench.dir/build.make:107: bench/CMakeFiles/matx_bench.dir/00_transform/conv.cu.o] Interrupt
make[1]: *** [CMakeFiles/Makefile2:935: bench/CMakeFiles/matx_bench.dir/all] Interrupt
make: *** [Makefile:136: all] Interrupt

System Details (please complete the following information):

• OS: Ubuntu 20.04
• CUDA version: Cuda 11.6
• g++ version: 9.4.0

Additional Context
None

[cliffburdick]

@HugoPhibbs, @tbensonatl looked into the problem and found that this was a bug in version of CUDA 11.7 and below. Given that we're starting to rely more and more on newer CUDA features, would it be a problem for you to update to CUDA 11.8?

[rcrovella]

cliffbudick Is it possible to get the requirements section updated to reflect the suggested build config? Right now it seems to suggest: "Please use CUDA 11.5-11.8 or CUDA 12.2.1+ with MatX."

[cliffburdick]

cliffbudick Is it possible to get the requirements section updated to reflect the suggested build config? Right now it seems to suggest: "Please use CUDA 11.5-11.8 or CUDA 12.2.1+ with MatX."

Hi @rcrovella yes, we were about to update that. I noticed 11.8 was released almost two years ago, and given our testing resources we have struggled to test down to 11.5 recently. I will update the docs to reflect 11.8 or above (except 12.1 and 12.2 since there are compiler bugs mentioned in the docs).

[cliffburdick]

The docs have been updated to reflect CUDA 11.8 or above. That being said, @tbensonatl may have a workaround for the issue you were seeing @HugoPhibbs. If he pushes it then it may get you by on 11.6 temporarily.

[HugoPhibbs]

Ok can confirm that I was able to get past this error using CUDA 11.8. I'm now getting an error that looks like:

[ 91%] Linking CUDA executable spectrogram_graph
/home/hphi344/Documents/MatX/include/matx/transforms/conv.h(67): error: class "matx::HostExecutor<matx::ThreadsMode::SINGLE>" has no member "getStream"
          detected during:
            instantiation of "void matx::detail::matxFFTConv1DInternal(OutputType &, const InType &, const FilterType &, matx::matxConvCorrMode_t, const Executor &) [with OutputType=matx::detail::tensor_impl_t<float, 1, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, InType=matx::tensor_t<float, 1, matx::basic_storage<matx::raw_pointer_buffer<float, matx::matx_allocator<float>>>, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, FilterType=matx::tensor_t<float, 1, matx::basic_storage<matx::raw_pointer_buffer<float, matx::matx_allocator<float>>>, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, Executor=matx::SingleThreadedHostExecutor]" 
(276): here
            instantiation of "void matx::conv1d_impl_internal(OutputType &, const In1Type &, const In2Type &, matx::matxConvCorrMode_t, matx::matxConvCorrMethod_t, const Executor &) [with OutputType=matx::tensor_t<float, 1, matx::basic_storage<matx::raw_pointer_buffer<float, matx::matx_allocator<float>>>, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, In1Type=matx::tensor_t<float, 1, matx::basic_storage<matx::raw_pointer_buffer<float, matx::matx_allocator<float>>>, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, In2Type=matx::tensor_t<float, 1, matx::basic_storage<matx::raw_pointer_buffer<float, matx::matx_allocator<float>>>, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, Executor=matx::SingleThreadedHostExecutor]" 
(370): here
            instantiation of "void matx::conv1d_impl(OutputType, const In1Type &, const In2Type &, matx::matxConvCorrMode_t, matx::matxConvCorrMethod_t, const Executor &) [with OutputType=matx::tensor_t<float, 1, matx::basic_storage<matx::raw_pointer_buffer<float, matx::matx_allocator<float>>>, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, In1Type=matx::tensor_t<float, 1, matx::basic_storage<matx::raw_pointer_buffer<float, matx::matx_allocator<float>>>, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, In2Type=matx::tensor_t<float, 1, matx::basic_storage<matx::raw_pointer_buffer<float, matx::matx_allocator<float>>>, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, Executor=matx::SingleThreadedHostExecutor]" 
/home/hphi344/Documents/MatX/include/matx/operators/conv.h(159): here
            instantiation of "void matx::detail::Conv1DOp<OpA, OpB, PermDims>::Exec(Out &&, Executor &&) const [with OpA=matx::tensor_t<float, 1, matx::basic_storage<matx::raw_pointer_buffer<float, matx::matx_allocator<float>>>, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, OpB=matx::tensor_t<float, 1, matx::basic_storage<matx::raw_pointer_buffer<float, matx::matx_allocator<float>>>, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, PermDims=matx::detail::no_permute_t, Out=cuda::std::__4::tuple<matx::tensor_t<float, 1, matx::basic_storage<matx::raw_pointer_buffer<float, matx::matx_allocator<float>>>, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, matx::detail::Conv1DOp<matx::tensor_t<float, 1, matx::basic_storage<matx::raw_pointer_buffer<float, matx::matx_allocator<float>>>, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, matx::tensor_t<float, 1, matx::basic_storage<matx::raw_pointer_buffer<float, matx::matx_allocator<float>>>, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, matx::detail::no_permute_t>> &, Executor=matx::SingleThreadedHostExecutor &]" 
/home/hphi344/Documents/MatX/include/matx/core/tie.h(102): here
            instantiation of "void matx::mtie<Ts...>::Exec(Executor &&) [with Ts=<matx::tensor_t<float, 1, matx::basic_storage<matx::raw_pointer_buffer<float, matx::matx_allocator<float>>>, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, matx::detail::Conv1DOp<matx::tensor_t<float, 1, matx::basic_storage<matx::raw_pointer_buffer<float, matx::matx_allocator<float>>>, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, matx::tensor_t<float, 1, matx::basic_storage<matx::raw_pointer_buffer<float, matx::matx_allocator<float>>>, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, matx::detail::no_permute_t>>, Executor=matx::SingleThreadedHostExecutor &]" 
/home/hphi344/Documents/MatX/include/matx/operators/base_operator.h(159): here
            instantiation of "void matx::BaseOp<T>::run(Ex &&) [with T=matx::mtie<matx::tensor_t<float, 1, matx::basic_storage<matx::raw_pointer_buffer<float, matx::matx_allocator<float>>>, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, matx::detail::Conv1DOp<matx::tensor_t<float, 1, matx::basic_storage<matx::raw_pointer_buffer<float, matx::matx_allocator<float>>>, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, matx::tensor_t<float, 1, matx::basic_storage<matx::raw_pointer_buffer<float, matx::matx_allocator<float>>>, matx::tensor_desc_t<cuda::std::__4::array<long long, 1UL>, cuda::std::__4::array<long long, 1UL>, 1>>, matx::detail::no_permute_t>>, Ex=matx::SingleThreadedHostExecutor &]" 
/home/hphi344/Documents/MatX/test/00_transform/ConvCorr.cu(216): here
            instantiation of "void CorrelationConvolutionLargeTestFloatTypes_FFT1DConvolutionLarge_Test<gtest_TypeParam_>::TestBody() [with gtest_TypeParam_=cuda::std::__4::tuple<float, matx::SingleThreadedHostExecutor>]" 
/home/hphi344/Documents/MatX/build/_deps/gtest-src/googletest/include/gtest/internal/gtest-internal.h(472): here
            implicit generation of "CorrelationConvolutionLargeTestFloatTypes_FFT1DConvolutionLarge_Test<gtest_TypeParam_>::~CorrelationConvolutionLargeTestFloatTypes_FFT1DConvolutionLarge_Test() noexcept [with gtest_TypeParam_=cuda::std::__4::tuple<float, matx::SingleThreadedHostExecutor>]" 
/home/hphi344/Documents/MatX/build/_deps/gtest-src/googletest/include/gtest/internal/gtest-internal.h(472): here
            instantiation of class "CorrelationConvolutionLargeTestFloatTypes_FFT1DConvolutionLarge_Test<gtest_TypeParam_> [with gtest_TypeParam_=cuda::std::__4::tuple<float, matx::SingleThreadedHostExecutor>]" 
/home/hphi344/Documents/MatX/build/_deps/gtest-src/googletest/include/gtest/internal/gtest-internal.h(472): here
            implicit generation of "CorrelationConvolutionLargeTestFloatTypes_FFT1DConvolutionLarge_Test<gtest_TypeParam_>::CorrelationConvolutionLargeTestFloatTypes_FFT1DConvolutionLarge_Test() [with gtest_TypeParam_=cuda::std::__4::tuple<float, matx::SingleThreadedHostExecutor>]" 
/home/hphi344/Documents/MatX/build/_deps/gtest-src/googletest/include/gtest/internal/gtest-internal.h(472): here
            instantiation of class "CorrelationConvolutionLargeTestFloatTypes_FFT1DConvolutionLarge_Test<gtest_TypeParam_> [with gtest_TypeParam_=cuda::std::__4::tuple<float, matx::SingleThreadedHostExecutor>]" 
/home/hphi344/Documents/MatX/build/_deps/gtest-src/googletest/include/gtest/internal/gtest-internal.h(472): here
            instantiation of "testing::Test *testing::internal::TestFactoryImpl<TestClass>::CreateTest() [with TestClass=CorrelationConvolutionLargeTestFloatTypes_FFT1DConvolutionLarge_Test<cuda::std::__4::tuple<float, matx::SingleThreadedHostExecutor>>]" 
/home/hphi344/Documents/MatX/build/_deps/gtest-src/googletest/include/gtest/internal/gtest-internal.h(740): here
            implicit generation of "testing::internal::TestFactoryImpl<TestClass>::~TestFactoryImpl() noexcept [with TestClass=CorrelationConvolutionLargeTestFloatTypes_FFT1DConvolutionLarge_Test<cuda::std::__4::tuple<float, matx::SingleThreadedHostExecutor>>]" 
/home/hphi344/Documents/MatX/build/_deps/gtest-src/googletest/include/gtest/internal/gtest-internal.h(740): here
            instantiation of class "testing::internal::TestFactoryImpl<TestClass> [with TestClass=CorrelationConvolutionLargeTestFloatTypes_FFT1DConvolutionLarge_Test<cuda::std::__4::tuple<float, matx::SingleThreadedHostExecutor>>]" 
/home/hphi344/Documents/MatX/build/_deps/gtest-src/googletest/include/gtest/internal/gtest-internal.h(740): here
            implicit generation of "testing::internal::TestFactoryImpl<TestClass>::TestFactoryImpl() [with TestClass=CorrelationConvolutionLargeTestFloatTypes_FFT1DConvolutionLarge_Test<cuda::std::__4::tuple<float, matx::SingleThreadedHostExecutor>>]" 
/home/hphi344/Documents/MatX/build/_deps/gtest-src/googletest/include/gtest/internal/gtest-internal.h(740): here
            instantiation of class "testing::internal::TestFactoryImpl<TestClass> [with TestClass=CorrelationConvolutionLargeTestFloatTypes_FFT1DConvolutionLarge_Test<cuda::std::__4::tuple<float, matx::SingleThreadedHostExecutor>>]" 
/home/hphi344/Documents/MatX/build/_deps/gtest-src/googletest/include/gtest/internal/gtest-internal.h(740): here
            instantiation of "__nv_bool testing::internal::TypeParameterizedTest<Fixture, TestSel, Types>::Register(const char *, const testing::internal::CodeLocation &, const char *, const char *, int, const std::vector<std::string, std::allocator<std::string>> &) [with Fixture=CorrelationConvolutionLargeTestFloatTypes, TestSel=testing::internal::TemplateSel<CorrelationConvolutionLargeTestFloatTypes_FFT1DConvolutionLarge_Test>, Types=testing::internal::Types<cuda::std::__4::tuple<float, matx::SingleThreadedHostExecutor>, cuda::std::__4::tuple<float, matx::AllThreadsHostExecutor>, cuda::std::__4::tuple<float, matx::SelectThreadsHostExecutor>, cuda::std::__4::tuple<double, matx::cudaExecutor>, cuda::std::__4::tuple<double, matx::SingleThreadedHostExecutor>, cuda::std::__4::tuple<double, matx::AllThreadsHostExecutor>, cuda::std::__4::tuple<double, matx::SelectThreadsHostExecutor>, cuda::std::__4::tuple<cuda::std::__4::complex<float>, matx::cudaExecutor>, cuda::std::__4::tuple<cuda::std::__4::complex<float>, matx::SingleThreadedHostExecutor>, cuda::std::__4::tuple<cuda::std::__4::complex<float>, matx::AllThreadsHostExecutor>, cuda::std::__4::tuple<cuda::std::__4::complex<float>, matx::SelectThreadsHostExecutor>, cuda::std::__4::tuple<cuda::std::__4::complex<double>, matx::cudaExecutor>, cuda::std::__4::tuple<cuda::std::__4::complex<double>, matx::SingleThreadedHostExecutor>, cuda::std::__4::tuple<cuda::std::__4::complex<double>, matx::AllThreadsHostExecutor>, cuda::std::__4::tuple<cuda::std::__4::complex<double>, matx::SelectThreadsHostExecutor>>]" 
/home/hphi344/Documents/MatX/build/_deps/gtest-src/googletest/include/gtest/internal/gtest-internal.h(743): here
            instantiation of "__nv_bool testing::internal::TypeParameterizedTest<Fixture, TestSel, Types>::Register(const char *, const testing::internal::CodeLocation &, const char *, const char *, int, const std::vector<std::string, std::allocator<std::string>> &) [with Fixture=CorrelationConvolutionLargeTestFloatTypes, TestSel=testing::internal::TemplateSel<CorrelationConvolutionLargeTestFloatTypes_FFT1DConvolutionLarge_Test>, Types=gtest_type_params_CorrelationConvolutionLargeTestFloatTypes_]" 
/home/hphi344/Documents/MatX/test/00_transform/ConvCorr.cu(206): here

1 error detected in the compilation of "/home/hphi344/Documents/MatX/test/00_transform/ConvCorr.cu".
make[2]: *** [test/CMakeFiles/matx_test.dir/build.make:242: test/CMakeFiles/matx_test.dir/00_transform/ConvCorr.cu.o] Error 255
make[2]: *** Waiting for unfinished jobs....

[cliffburdick]

@HugoPhibbs you sent that as we were debugging this issue. It has to do with the new host FFT support and we should have a fix soon.

[HugoPhibbs]

@cliffburdick thx. Is there a work around for this atm?

[cliffburdick]

A very quick and dirty workaround would be to change in operators/conv.h:

          if constexpr (!std::is_same_v<PermDims, no_permute_t>) {
            conv1d_impl(permute(cuda::std::get<0>(out), perm_), a_, b_, mode_, method_, ex);
          }
          else {
            conv1d_impl(cuda::std::get<0>(out), a_, b_, mode_, method_, ex);
          }

to

  if constexpr (is_cuda_executor_v<Executor>) {
          if constexpr (!std::is_same_v<PermDims, no_permute_t>) {
            conv1d_impl(permute(cuda::std::get<0>(out), perm_), a_, b_, mode_, method_, ex);
          }
          else {
            conv1d_impl(cuda::std::get<0>(out), a_, b_, mode_, method_, ex);
          }
  }

I have not tested this

[HugoPhibbs]

But if I'm not using convolutions should be ok?

[cliffburdick]

Yes, the goal right now is to just get it to compile while we debug

[HugoPhibbs]

Ok, I applied the given fix to conv.h. I now get a couple more errors, so I'll just wait for a fix for conv.h

[cliffburdick]

@HugoPhibbs this one appears to be a bug using if constexpr inside of a lambda in CUDA 11.8. It does not fail on CUDA 12.x. We're discussing what the best option is from here.

[cliffburdick]

@HugoPhibbs there's a fix here I'm testing: https://github.com/NVIDIA/MatX/pull/671/files

You're welcome to test on your end in the meantime too.

[HugoPhibbs]

@cliffburdick Hi, I think your PR may introduce another bug (you may already be aware of it?).

The stack trace is shown below (it might be a bug within cpp itself?).

[ 82%] Built target spectrogram_graph
/usr/include/c++/9/bits/specfun.h(342): error: expected a ">"

/usr/include/c++/9/bits/specfun.h(346): error: expected a declaration

/usr/include/c++/9/bits/specfun.h(359): error: name followed by "::" must be a class or namespace name

/usr/include/c++/9/bits/specfun.h(359): error: expected an identifier

/usr/include/c++/9/bits/specfun.h(359): error: expected a ";"

... A bunch more errors just like the above ...

/usr/include/c++/9/bits/specfun.h(988): error: expected an identifier

/usr/include/c++/9/bits/specfun.h(988): error: expected a ";"

/usr/include/c++/9/bits/specfun.h(1010): error: name followed by "::" must be a class or namespace name

Error limit reached.
100 errors detected in the compilation of "/home/hphi344/Documents/MatX/test/00_operators/PWelch.cu".
Compilation terminated.
make[2]: *** [test/CMakeFiles/matx_test.dir/build.make:212: test/CMakeFiles/matx_test.dir/00_operators/PWelch.cu.o] Error 255
make[2]: *** Waiting for unfinished jobs....