Tape instruction

[KDr2]

Try to implement #99.

[KDr2]

I test AdvancedPS against this change: TuringLang/AdvancedPS.jl@9d7debf, it got a numeric error.

UPDATE:
Fixed.

[yebai]

@KDr2 can you update the Turing PR (TuringLang/Turing.jl#1744), and see whether this works there?

[KDr2]

@KDr2 can you update the Turing PR (TuringLang/Turing.jl#1744), and see whether this works there?

I am on it now, it has some deeper issues, I haven't find the root cause yet.

[KDr2]

One of them seems to be an issue of IRTools, but I am not very sure yet:

mh inference: Error During Test at /data/zhuoql/Work/julia/Turing.jl/test/test_utils/staging.jl:42
  Got exception outside of a @test
  MethodError: no method matching var"##1226"(::DynamicPPL.Model{typeof(MoGtest), (:D,), (), (), Tuple{Matrix{Float64}}, Tuple{}, DynamicPPL.DefaultContext}, ::Libtask.Box{
DynamicPPL.TypedVarInfo{NamedTuple{(:mu1, :mu2, :z1, :z2, :z3, :z4), Tuple{DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:mu1, Setfield.IdentityLens}, Int64}, Vector{Normal{
Float64}}, Vector{AbstractPPL.VarName{:mu1, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}, DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:mu2,
Setfield.IdentityLens}, Int64}, Vector{Normal{Float64}}, Vector{AbstractPPL.VarName{:mu2, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}, Dynam
icPPL.Metadata{Dict{AbstractPPL.VarName{:z1, Setfield.IdentityLens}, Int64}, Vector{Categorical{Float64, Vector{Float64}}}, Vector{AbstractPPL.VarName{:z1, Setfield.Identit
yLens}}, Vector{Int64}, Vector{Set{DynamicPPL.Selector}}}, DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:z2, Setfield.IdentityLens}, Int64}, Vector{Categorical{Float64, Vec
tor{Float64}}}, Vector{AbstractPPL.VarName{:z2, Setfield.IdentityLens}}, Vector{Int64}, Vector{Set{DynamicPPL.Selector}}}, DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:z3,
 Setfield.IdentityLens}, Int64}, Vector{Categorical{Float64, Vector{Float64}}}, Vector{AbstractPPL.VarName{:z3, Setfield.IdentityLens}}, Vector{Int64}, Vector{Set{DynamicPP
L.Selector}}}, DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:z4, Setfield.IdentityLens}, Int64}, Vector{Categorical{Float64, Vector{Float64}}}, Vector{AbstractPPL.VarName{:
z4, Setfield.IdentityLens}}, Vector{Int64}, Vector{Set{DynamicPPL.Selector}}}}}, Float64}}, ::Libtask.Box{Sampler{PG{(:z1, :z2, :z3, :z4), ResampleWithESSThreshold{typeof(r
esample_systematic), Float64}}}})
  Closest candidates are:
    var"##1226"(::Any, ::Any) at /data/zhuoql/.julia/packages/IRTools/isLV2/src/eval.jl:18
  Stacktrace:
    [1] invokelatest(::Any, ::Any, ::Vararg{Any}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
      @ Base ./essentials.jl:716
    [2] invokelatest(::Any, ::Any, ::Vararg{Any})
      @ Base ./essentials.jl:714
    [3] evalir(::Module, ::IRTools.Inner.IR, ::Any, ::Vararg{Any})
      @ IRTools.Inner ~/.julia/packages/IRTools/isLV2/src/eval.jl:25
    [4] evalir(::IRTools.Inner.IR, ::Any, ::Any, ::Vararg{Any})
      @ IRTools.Inner ~/.julia/packages/IRTools/isLV2/src/eval.jl:26
    [5] run_and_record!(::Libtask.Tape, ::Libtask.Box{DynamicPPL.Model{typeof(MoGtest), (:D,), (), (), Tuple{Matrix{Float64}}, Tuple{}, DynamicPPL.DefaultContext}}, ::Libta
sk.Box{DynamicPPL.TypedVarInfo{NamedTuple{(:mu1, :mu2, :z1, :z2, :z3, :z4), Tuple{DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:mu1, Setfield.IdentityLens}, Int64}, Vector{
Normal{Float64}}, Vector{AbstractPPL.VarName{:mu1, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}, DynamicPPL.Metadata{Dict{AbstractPPL.VarName
{:mu2, Setfield.IdentityLens}, Int64}, Vector{Normal{Float64}}, Vector{AbstractPPL.VarName{:mu2, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}
, DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:z1, Setfield.IdentityLens}, Int64}, Vector{Categorical{Float64, Vector{Float64}}}, Vector{AbstractPPL.VarName{:z1, Setfield.
IdentityLens}}, Vector{Int64}, Vector{Set{DynamicPPL.Selector}}}, DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:z2, Setfield.IdentityLens}, Int64}, Vector{Categorical{Float
64, Vector{Float64}}}, Vector{AbstractPPL.VarName{:z2, Setfield.IdentityLens}}, Vector{Int64}, Vector{Set{DynamicPPL.Selector}}}, DynamicPPL.Metadata{Dict{AbstractPPL.VarNa
me{:z3, Setfield.IdentityLens}, Int64}, Vector{Categorical{Float64, Vector{Float64}}}, Vector{AbstractPPL.VarName{:z3, Setfield.IdentityLens}}, Vector{Int64}, Vector{Set{Dy
namicPPL.Selector}}}, DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:z4, Setfield.IdentityLens}, Int64}, Vector{Categorical{Float64, Vector{Float64}}}, Vector{AbstractPPL.Va
rName{:z4, Setfield.IdentityLens}}, Vector{Int64}, Vector{Set{DynamicPPL.Selector}}}}}, Float64}}, ::Vararg{Any})
      @ Libtask ~/Work/julia/Libtask.jl/src/tapedfunction.jl:124
    [6] ##1225
      @ ~/Work/julia/Turing.jl/src/core/container.jl:16 [inlined]
...

[KDr2]

Varargs needs some special manipulation, I (hopefully suppose that) had fixed that. But, there're still issues, one of which is dynamic branches, e.g., https://github.com/TuringLang/DynamicPPL.jl/blob/v0.16.2/src/model.jl#L387

[KDr2]

There are also some dynamically generated (by macro expanding or other facilities) functions, which we can't trace into.

[KDr2]

And, there are too many branches on the way by which we trace down into a deep produce call.
like this, https://github.com/TuringLang/Turing.jl/blob/master/test/test_utils/models.jl#L20.

So maybe we should support braches before we use it in Turing.

[yebai]

No longer needed after #107 - in the future, we can treat TapedFunction as special instructions with help from #97