ERROR: BoundsError: attempt to access 1-element Vector{Core.LineInfoNode} at index [0]

[chengchingwen]

MWE:

using Flux, Cthulhu
@descend gradient((m, x)->sum(sin.(m(x))), d, x)
# [...]
Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
Toggles: [w]arn, [h]ide type-stable statements, [t]ype annotations, [s]yntax highlight for Source/LLVM/Native.
Show: [S]ource code, [A]ST, [T]yped code, [L]LVM IR, [N]ative code
Actions: [E]dit source code, [R]evise and redisplay
 • 96 pullback(f, args...)
   96 pullback(f, args...)
   96 pullback(f::var"#27#28", args::Tuple{Dense{typeof(identity), Matrix{Float32}, Vector{Float3…
   97 sensitivity(y::Float32)
   97 back::Zygote.var"#60#61"{Zygote.Pullback{Tuple{var"#27#28", Dense{typeof(identity), Matrix{Float32}…………………………
   %11 = isnothing(::Tuple{…})::Core.Const(false)
    98 map(_project, args::Tuple{Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}, Matrix{Float3…………
   ↩
# [...]
42  pullback(f::var"#27#28", args::Tuple{Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}, Matrix{Float32}}...) = pullback(f::var"#27#28", Context::Type{Zygote.Context}()::Zygote.Context{false}, args::Tuple{Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}, Matrix{Float32}}...)
Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
Toggles: [w]arn, [h]ide type-stable statements, [t]ype annotations, [s]yntax highlight for Source/LLVM/Native.
Show: [S]ource code, [A]ST, [T]yped code, [L]LVM IR, [N]ative code
Actions: [E]dit source code, [R]evise and redisplay
   42 Context::Type{Zygote.Context}()
 • 42 pullback(f::var"#27#28", Context::Type{Zygote.Context}()::Zygote.Context{false}…
# [...]
Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
Toggles: [w]arn, [h]ide type-stable statements, [t]ype annotations, [s]yntax highlight for Source/LLVM/Native.
Show: [S]ource code, [A]ST, [T]yped code, [L]LVM IR, [N]ative code
Actions: [E]dit source code, [R]evise and redisplay
 • 44 _pullback(cx, f, args...)
   44 _pullback(cx, f, args...)
   44 _pullback(cx::Zygote.Context{false}, f::var"#27#28", args::Tuple{Dense{typeof(ident…
   ↩
ERROR: BoundsError: attempt to access 1-element Vector{Core.LineInfoNode} at index [0]
Stacktrace:                                                                                                               
  [1] getindex
    @ ./essentials.jl:13 [inlined]
  [2] getindex
    @ ./abstractarray.jl:1297 [inlined]
  [3] getline(lt::Vector{Core.LineInfoNode}, j::Int32)
    @ TypedSyntax ~/.julia/packages/TypedSyntax/xKbe0/src/node.jl:672
  [4] (::TypedSyntax.var"#append_targets_for_line!#17"{Core.CodeInfo, Int64})(mapped::Vector{JuliaSyntax.TreeNode{JuliaSyn
tax.SyntaxData}}, i::Int64, targets::Vector{JuliaSyntax.TreeNode{JuliaSyntax.SyntaxData}})
    @ TypedSyntax ~/.julia/packages/TypedSyntax/xKbe0/src/node.jl:307
  [5] (::TypedSyntax.var"#append_targets_for_arg!#18"{TypedSyntax.var"#append_targets_for_line!#17"{Core.CodeInfo, Int64}}
)(mapped::Vector{JuliaSyntax.TreeNode{JuliaSyntax.SyntaxData}}, i::Int64, arg::Any)
    @ TypedSyntax ~/.julia/packages/TypedSyntax/xKbe0/src/node.jl:319
  [6] map_ssas_to_source(src::Core.CodeInfo, rootnode::JuliaSyntax.TreeNode{JuliaSyntax.SyntaxData}, Δline::Int64)
    @ TypedSyntax ~/.julia/packages/TypedSyntax/xKbe0/src/node.jl:430
  [7] tsn_and_mappings(m::Method, src::Core.CodeInfo, rt::Any; warn::Bool, strip_macros::Bool, kwargs::Base.Pairs{Symbol, 
Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ TypedSyntax ~/.julia/packages/TypedSyntax/xKbe0/src/node.jl:44
  [8] get_typed_sourcetext(mi::Core.MethodInstance, src::Core.CodeInfo, rt::Any; warn::Bool)
    @ Cthulhu ~/.julia/packages/Cthulhu/e6txW/src/reflection.jl:350
  [9] find_callsites(interp::Cthulhu.CthulhuInterpreter, CI::Core.CodeInfo, stmt_infos::Vector{Core.Compiler.CallInfo}, mi
::Core.MethodInstance, slottypes::Vector{Any}, optimize::Bool, annotate_source::Bool)
    @ Cthulhu ~/.julia/packages/Cthulhu/e6txW/src/reflection.jl:28
 [10] _descend(term::REPL.Terminals.TTYTerminal, interp::Cthulhu.CthulhuInterpreter, curs::Cthulhu.CthulhuCursor; override
::Nothing, debuginfo::Cthulhu.DInfo.DebugInfo, optimize::Bool, interruptexc::Bool, iswarn::Bool, hide_type_stable::Bool, v
erbose::Nothing, remarks::Bool, with_effects::Bool, inline_cost::Bool, type_annotations::Bool, annotate_source::Bool)
    @ Cthulhu ~/.julia/packages/Cthulhu/e6txW/src/Cthulhu.jl:496
 [11] _descend(term::REPL.Terminals.TTYTerminal, interp::Cthulhu.CthulhuInterpreter, curs::Cthulhu.CthulhuCursor; override
::Nothing, debuginfo::Cthulhu.DInfo.DebugInfo, optimize::Bool, interruptexc::Bool, iswarn::Bool, hide_type_stable::Bool, v
erbose::Nothing, remarks::Bool, with_effects::Bool, inline_cost::Bool, type_annotations::Bool, annotate_source::Bool) (rep
eats 2 times)
    @ Cthulhu ~/.julia/packages/Cthulhu/e6txW/src/Cthulhu.jl:629
 [12] _descend(term::REPL.Terminals.TTYTerminal, interp::Cthulhu.CthulhuInterpreter, curs::Cthulhu.CthulhuCursor; override
::Nothing, debuginfo::Symbol, optimize::Bool, interruptexc::Bool, iswarn::Bool, hide_type_stable::Bool, verbose::Nothing,
remarks::Bool, with_effects::Bool, inline_cost::Bool, type_annotations::Bool, annotate_source::Bool)
    @ Cthulhu ~/.julia/packages/Cthulhu/e6txW/src/Cthulhu.jl:629
 [13] _descend(term::REPL.Terminals.TTYTerminal, interp::Cthulhu.CthulhuInterpreter, mi::Core.MethodInstance; kwargs::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:iswarn,), Tuple{Bool}}})
    @ Cthulhu ~/.julia/packages/Cthulhu/e6txW/src/Cthulhu.jl:765
 [14] _descend(term::REPL.Terminals.TTYTerminal, args::Any; interp::Core.Compiler.NativeInterpreter, kwargs::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:iswarn,), Tuple{Bool}}})
    @ Cthulhu ~/.julia/packages/Cthulhu/e6txW/src/Cthulhu.jl:781
 [15] __descend_with_error_handling(args::Any; terminal::Any, kwargs::Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where {V, N, names, T<:Tuple{Vararg{Any, N}}})
    @ Cthulhu ~/.julia/packages/Cthulhu/e6txW/src/Cthulhu.jl:239
 [16] _descend_with_error_handling(f::Any, argtypes::Any; kwargs::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:iswarn,), Tuple{Bool}}})
    @ Cthulhu ~/.julia/packages/Cthulhu/e6txW/src/Cthulhu.jl:228
 [17] descend_code_typed(::Any, ::Vararg{Any}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Cthulhu ~/.julia/packages/Cthulhu/e6txW/src/Cthulhu.jl:173
 [18] descend_code_typed(::Any, ::Any)
    @ Cthulhu ~/.julia/packages/Cthulhu/e6txW/src/Cthulhu.jl:173
 [19] top-level scope
    @ REPL[21]:1
 [20] top-level scope
    @ ~/.julia/packages/Infiltrator/r3Hf5/src/Infiltrator.jl:710
 [21] top-level scope
    @ ~/.julia/packages/CUDA/ZdCxS/src/initialization.jl:155

[timholy]

I need more information to replicate this:

julia> @descend gradient((m, x)->sum(sin.(m(x))), d, x)
ERROR: UndefVarError: `d` not defined
Stacktrace:
 [1] top-level scope
   @ REPL[5]:1

[chengchingwen]

Ah sorry, I forgot to paste it. The complete code should be

using Flux, Cthulhu
d = Dense(10, 10)
x = randn(Float32, 10, 5)
@descend gradient((m, x)->sum(sin.(m(x))), d, x)

tested with julia-1.9-rc1

[timholy]

If you pkg> update TypedSyntax Cthulhu in 15ish minutes it should be fixed.

Keep in mind that type-annotations will be missing in the cases where this bug was triggered. The root cause is that there's code that's missing any kind of line number annotation, and having good line numbers is crucial for this new Cthulhu mode to work. I'd guess that something in Zygote, perhaps, is generating code and not adding line info? Fixing that might help both the quality of backtraces and Cthulhu.

[chengchingwen]

I'd guess that something in Zygote, perhaps, is generating code and not adding line info? Fixing that might help both the quality of backtraces and Cthulhu.

The gradient code is generated with IRTools, that's probably the reason why it miss the line info. But I'm not sure if it's possible to add any kind of line info. Maybe @ToucheSir have any thought?

[ToucheSir]

Line info tracking in IRTools is a mess and trips up other tools as well (Debugger.jl notably). Since nobody around understands the logic it uses, I'm not sure what we can do short of getting rid of IRTools itself.

[timholy]

If it's generating code by manually creating new Exprs, "all" you have to do is insert LineNumberNodes in the right places. For example, let's manually define f() = 1 (if you don't already know, you can learn how this works with dump(:(f() = 1))) in two ways:

julia> eval(Expr(:(=), Expr(:call, :f), Expr(:block, 1)))
f (generic function with 1 method)

julia> f()
1

julia> methods(f)
# 1 method for generic function "f" from Main:
 [1] f()
     @ none:0

which lacks line/file info, vs

julia> eval(Expr(:(=), Expr(:call, :f), Expr(:block, LineNumberNode(55, Symbol("somefile.jl")), 1)))
f (generic function with 1 method)

julia> f()
1

julia> methods(f)
# 1 method for generic function "f" from Main:
 [1] f()
     @ somefile.jl:55

which has the line/file info I supplied. You need one LineNumberNode for each new line in the "source" text.

For things that are essentially macros, it can get a little more involved (is it the location of the macro or its "caller"?). mauro3/SimpleTraits.jl#6 has an example of how you can pull this off.

[ToucheSir]

I'm afraid the answer is that it does all of the above and none of the above simultaneously! That's how we get the classic stacktraces with alternating line #0 and line numbers from the source (i.e. non transformed) function. The IRTools line handling logic is scattered all over https://github.com/FluxML/IRTools.jl/blob/8a6971e4a594d540a491cafa059942472dc86a26/src/ir/ir.jl and https://github.com/FluxML/IRTools.jl/blob/8a6971e4a594d540a491cafa059942472dc86a26/src/ir/wrap.jl, so tracking it all down would be a pain. What's more, troubleshooting line number placement or tracking provenance of LineNumberNodes is made much harder because IRTools converts to and from its own, proprietary IR format! I'm almost certain this loses source line or file information at some point.

[timholy]

Wow, that is a bit of a mess. I'll wish for a miracle for you, then 😉 .