Should reshaping a SubArray produce another SubArray?

[johnmyleswhite]

Right now, calling reshape on a SubArray allocates memory, which means that reshaping a slice causes you to lose your view into the original data. See the example below for the kinds of situations you might find yourself in.

julia> a = ones(9)
9-element Array{Float64,1}:
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0

julia> b = reshape(a, 3, 3)
3x3 Array{Float64,2}:
 1.0  1.0  1.0
 1.0  1.0  1.0
 1.0  1.0  1.0

julia> b[3, 3] = 9.0
9.0

julia> a
9-element Array{Float64,1}:
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 9.0

julia> c = slice(a, 4:9)
6-element SubArray{Float64,1,Array{Float64,1},(UnitRange{Int64},),1}:
 1.0
 1.0
 1.0
 1.0
 1.0
 9.0

julia> c[1] = 4.0
4.0

julia> a
9-element Array{Float64,1}:
 1.0
 1.0
 1.0
 4.0
 1.0
 1.0
 1.0
 1.0
 9.0

julia> d = reshape(c, 3, 2)
3x2 Array{Float64,2}:
 4.0  1.0
 1.0  1.0
 1.0  9.0

julia> d[2, 1] = 5.0
5.0

julia> a
9-element Array{Float64,1}:
 1.0
 1.0
 1.0
 4.0
 1.0
 1.0
 1.0
 1.0
 9.0

[timholy]

If that's how we want them to work, a rudimentary version is basically all ready-to-go, because our new SubArrays support a Vector{Int} as an index. Here's why that fact is relevant:

julia> A = reshape(1:16, 4, 4)
4x4 Array{Int64,2}:
 1  5   9  13
 2  6  10  14
 3  7  11  15
 4  8  12  16

julia> B = sub(A, 2:4, 3:4)
3x2 SubArray{Int64,2,Array{Int64,2},(UnitRange{Int64},UnitRange{Int64}),1}:
 10  14
 11  15
 12  16

julia> b = sub(A, [10,11,12,14,15,16])
6-element SubArray{Int64,1,Array{Int64,2},(Array{Int64,1},),0}:
 10
 11
 12
 14
 15
 16

julia> b[4] = 0
0

julia> A
4x4 Array{Int64,2}:
 1  5   9  13
 2  6  10   0
 3  7  11  15
 4  8  12  16

One big downside, though, is that you have to allocate a full-length vector just to express the "unwrapped" 2d indexing. It seems possible one could do better with some kind of smart multidimensional Range object, but I haven't given this any serious design thought yet.

[johnmyleswhite]

That is a really helpful start, @timholy. But often we might want to reshape upwards by adding dimensions. So we want to go from a 1D structure to a 2D structure. That seems to require a call to reshape as far as I can tell with my limited understanding of sub.

[johnmyleswhite]

Actually, I'm pretty confused by the b = sub(A, [10,11,12,14,15,16]) line. It seems like you can get the same effect just by using b = sub(A, 10:16) without allocating a full length vector.

[timholy]

But often we might want to reshape upwards by adding dimensions

You can currently only do that in a very limited way: a = 1:10; s = sub(a, 2:4, 1). This makes a (trivial) 2d array out of a 1d array.

Actually, I'm pretty confused by the b = sub(A, [10,11,12,14,15,16]) line. It seems like you can get the same effect just by using b = sub(A, 10:16) without allocating a full length vector.

Notice I was deliberately omitting 13, which you can't do with the range.

[timholy]

But adding dimensions could be implemented more generally with smart multidimensional range objects.

[johnmyleswhite]

Thanks for clarifying the index point.

When you say multidimensional range objects, are you thinking of something like the punned meaning of 1:3:1:5 to indicate a range in two dimensions at once?

[timholy]

Ideally, in terms of my example above we'd have an indexing object ind that returns 2,4 when accessed as ind[4]: b[4] -> A[2,4]. (In the 2d space spanned by (2:4,3:4)---the indexes used to create B---the 4th entry is (2,4) if you're incrementing along the first dimension.) So b would be effectively created as b = sub(A, ind) where ind is a magical 1d to 2d indexing object mapping 1:6 -> (2:4,3:4).

The problem is, I don't really know how to realistically make such objects work without calling div, and div is absolutely deadly for performance.

Going the opposite direction in a performant manner is much easier. Suppose you have a vector and you want to reshape it into a matrix:

a = 1:15
B = sub(a, Index(1:5,1:3))  # this would become the implementation of `reshape(a, 5, 3)`

Then B[3,2] could be trivially expanded to a[(2-1)*5+3] = 8.

More details at http://docs.julialang.org/en/latest/devdocs/subarrays/, especially the first two sections.

[simonster]

The reshape docs say "an implementation for a particular type of array may choose whether the data is copied or shared." To me this implies that reshape of SubArrays should share data for SubArrays when that will be faster than constructing a new array (i.e., when we have fast linear indexing), but shouldn't when it won't be.

We could also possibly have a reshape_shared function (or something with a better name) that always shares data for any AbstractArray but possibly returns a view into the original array that uses linear indexing and thus may incur a performance penalty. reshape would be good for operations that only need to read from the array, whereas reshape_shared would be useful for in-place modification as described above.

[johnmyleswhite]

"an implementation for a particular type of array may choose whether the data is copied or shared."

I think this makes reshape a little too difficult to reason about. Having to look up information about every specific method for a generic function to understand whether it introduces shared data seems problematic, especially when we go to good effort to make mutation so easy to predict from the name of a function alone.

For context on how these issues are coming up, I've been working with some folks that are new to Julia and are accustomed to building their entire infrastructure around higher-order tensors. They routinely takes slices (possibly with subsequent reshaping) of a linear packing of many distinct arrays. The linear packing is used because they send the entire packed data store into a black-box optimization suite that operates on vectors, but they simultaneously want to articulate the objective function they're optimizing in terms of many different objects -- all of which point, via SubArray-like objects, into the memory store of the grand linear packing. So they need to make sure that every operation they might perform can be done without memory copying.

[nalimilan]

I agree with John. I'd rather people choose do an explicit action (like copying, or calling different function that copies) when they need performance, rather than forcing them to reason about whether the data is going to be copied or not. The fact that in Julia arrays sometimes share data is already relatively complex to handle (though incredibly powerful of course).

[StefanKarpinski]

I'm in agreement on this too – "Schrödinger's copy" is no good.

[milktrader]

I'd like to keep tabs on this discussion but have nothing to add. And I don't know how to do so in github without posting a comment. Sighs. Sorry for the noise.

[johnmyleswhite]

@milktrader, there's a notifications button the right-hand side that will let you do that. It's not very conspicuous, but it's near the bottom of the right-hand side panel.

[timholy]

Relevant discussion of faster ways to do div: #8188 (comment). In the case of reshape the denominator is fixed, so the smart ind object could precompute the magic numbers so that usage is fast.