Skip to content
This repository was archived by the owner on Mar 12, 2021. It is now read-only.
This repository was archived by the owner on Mar 12, 2021. It is now read-only.

Linear indexing into arrays with two or more dimensions throws an obscure error during gradient calculation with Zygote #590

@AzamatB

Description

@AzamatB

MRE:

julia> using CuArrays, Zygote

julia> gradient(x -> sum(x[[1,2]]), CuArrays.rand(2,1))
ERROR: ArgumentError: an array of type `CuArray` shares memory with another argument and must
make a preventative copy of itself in order to maintain consistent semantics,
but `copy(A)` returns a new array of type `CuArray{Float32,1,Nothing}`. To fix, implement:
    `Base.unaliascopy(A::CuArray)::typeof(A)`
Stacktrace:
 [1] _unaliascopy(::CuArray{Float32,1,CuArray{Float32,2,Nothing}}, ::CuArray{Float32,1,Nothing}) at ./abstractarray.jl:1174
 [2] unaliascopy(::CuArray{Float32,1,CuArray{Float32,2,Nothing}}) at ./abstractarray.jl:1172
 [3] unaliascopy(::SubArray{Float32,1,CuArray{Float32,1,CuArray{Float32,2,Nothing}},Tuple{Array{Int64,1}},false}) at ./subarray.jl:100
 [4] materialize! at ./abstractarray.jl:1155 [inlined]
 [5] (::Zygote.var"#1003#1005"{CuArray{Float32,2,Nothing},Tuple{Array{Int64,1}}})(::CuArray{Float32,1,Nothing}) at /home/azamat/.julia/packages/Zygote/tJj2w/src/lib/array.jl:43
 [6] #5 at /home/azamat/.julia/packages/ZygoteRules/6nssF/src/adjoint.jl:49 [inlined]
 [7] (::Zygote.var"#36#37"{typeof((#5))})(::Float32) at /home/azamat/.julia/packages/Zygote/tJj2w/src/compiler/interface.jl:38
 [8] gradient(::Function, ::CuArray{Float32,2,Nothing}) at /home/azamat/.julia/packages/Zygote/tJj2w/src/compiler/interface.jl:47
 [9] top-level scope at REPL[2]:1

yet works as expected with Base.Array

julia> gradient(x -> sum(x[[1,2]]), rand(2,1))
([1.0; 1.0],)

I've noticed that this only happens when you do linear indexing with vector into array with two or more dimensions.

julia> Pkg.build()
  Building CMake ────────── `~/.julia/packages/CMake/ULbyn/deps/build.log`
  Building Blosc ────────── `~/.julia/packages/Blosc/lzFr0/deps/build.log`
  Building HDF5 ─────────── `~/.julia/packages/HDF5/Zh9on/deps/build.log`
  Building MbedTLS ──────── `~/.julia/packages/MbedTLS/a1JFn/deps/build.log`
  Building LibCURL ──────── `~/.julia/packages/LibCURL/lWJxD/deps/build.log`
  Building WinRPM ───────── `~/.julia/packages/WinRPM/BfpQI/deps/build.log`
  Building PackageCompiler  `~/.julia/packages/PackageCompiler/4yNnV/deps/build.log`
  Building CodecBzip2 ───── `~/.julia/packages/CodecBzip2/T5yr7/deps/build.log`
  Building PkgBenchmark ─── `~/.julia/packages/PkgBenchmark/fIDIa/deps/build.log`
  Building CodecZlib ────── `~/.julia/packages/CodecZlib/5t9zO/deps/build.log`
  Building NNlib ────────── `~/.julia/packages/NNlib/3krvM/deps/build.log`
  Building FFTW ─────────── `~/.julia/packages/FFTW/qqcBj/deps/build.log`
  Building WebIO ────────── `~/.julia/packages/WebIO/2nnB1/deps/build.log`
  Building Conda ────────── `~/.julia/packages/Conda/3rPhK/deps/build.log`
  Building PyCall ───────── `~/.julia/packages/PyCall/kAhnQ/deps/build.log`
  Building MKL ──────────── `~/.julia/packages/MKL/9dWbC/deps/build.log`
false
julia> versioninfo()
Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
(v1.3) pkg> status CuArrays Zygote
    Status `~/.julia/environments/v1.3/Project.toml`
  [c5f51814] CUDAdrv v6.0.0
  [be33ccc6] CUDAnative v2.10.2
  [3a865a2d] CuArrays v1.7.2
  [0c68f7d7] GPUArrays v2.0.1
  [7869d1d1] IRTools v0.3.1
  [1914dd2f] MacroTools v0.5.4
  [e88e6eb3] Zygote v0.4.7
  [700de1a5] ZygoteRules v0.2.0
shell> nvidia-smi
Mon Feb 17 04:36:11 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.33.01    Driver Version: 440.33.01    CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 108...  On   | 00000000:01:00.0  On |                  N/A |
| 24%   43C    P2    56W / 250W |   1635MiB / 11177MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      1678      G   /usr/lib/xorg/Xorg                            40MiB |
|    0      1780      G   /usr/bin/gnome-shell                          50MiB |
|    0      2207      G   /usr/lib/xorg/Xorg                           525MiB |
|    0      2354      G   /usr/bin/gnome-shell                         224MiB |
|    0      3002      G   ...AAAAAAAAAAAAAAgAAAAAAAAA --shared-files   294MiB |
|    0     12443      G   ...equest-channel-token=730461811546582272   208MiB |
|    0     15918      C   /home/azamat/julia-1.3.1/bin//julia          141MiB |
|    0     17036      C   julia                                        141MiB |
+-----------------------------------------------------------------------------+

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions