Trying some other optimizations beside type-infer

[KDr2]

No description provided.

[KDr2]

@testset "Turing" begin
    @time include("inference/AdvancedSMC.jl")
    @time include("inference/gibbs.jl")
    @time include("contrib/inference/sghmc.jl")
    @time include("stdlib/RandomMeasures.jl")
end

Branch master:

 34.395940 seconds (82.78 M allocations: 5.174 GiB, 4.03% gc time, 95.12% compilation time)
285.165249 seconds (834.03 M allocations: 67.579 GiB, 3.62% gc time, 25.78% compilation time)
 11.141108 seconds (38.60 M allocations: 2.341 GiB, 4.64% gc time, 98.42% compilation time)
  7.159114 seconds (16.94 M allocations: 1.029 GiB, 4.85% gc time, 70.14% compilation time)

Branch no-type-opt:

 39.743907 seconds (89.76 M allocations: 5.518 GiB, 3.81% gc time, 95.43% compilation time)
299.946079 seconds (831.72 M allocations: 67.493 GiB, 3.45% gc time, 24.85% compilation time)
 11.235855 seconds (38.59 M allocations: 2.341 GiB, 4.98% gc time, 97.71% compilation time)
  9.119536 seconds (26.03 M allocations: 1.275 GiB, 4.52% gc time, 75.07% compilation time)

It becomes slower... 😢

[KDr2]

First commit(5ac43d4) on Branch no-type-opt:

 34.459184 seconds (81.37 M allocations: 5.088 GiB, 3.99% gc time, 94.95% compilation time)
302.663785 seconds (553.19 M allocations: 59.879 GiB, 3.18% gc time, 23.32% compilation time)
 10.996351 seconds (38.60 M allocations: 2.341 GiB, 5.07% gc time, 98.22% compilation time)
  7.437808 seconds (15.85 M allocations: 1012.160 MiB, 4.32% gc time, 69.07% compilation time)

[KDr2]

Second commit(cce6cb3) on Branch no-type-opt:

 34.994963 seconds (82.41 M allocations: 5.142 GiB, 4.03% gc time, 95.08% compilation time)
300.684379 seconds (615.39 M allocations: 62.235 GiB, 3.44% gc time, 23.27% compilation time)
 10.994031 seconds (38.60 M allocations: 2.341 GiB, 4.99% gc time, 98.45% compilation time)
  7.422173 seconds (17.95 M allocations: 1.065 GiB, 4.46% gc time, 69.01% compilation time)

[KDr2]

First commit(5ac43d4) on Branch no-type-opt:

This commit has the least memory allocation, but it doesn't save much time cost.

[rikhuijzer]

First commit(5ac43d4) on Branch no-type-opt:

This commit has the least memory allocation, but it doesn't save much time cost.

Could be that your computer was more busy during the one run versus the other. What you can do to dive deeper into why things are going faster or slower is to put @time things at some places in the codebase. As far as I can tell, it's difficult to see these instruction constructors as stand-alone, so then running larger parts with a @time does help sometimes.

[rikhuijzer]

I don't understand everything. Some suggestions.

Good news about the benchmarking: TuringLang/Turing.jl#1787 is merged, so we can now see the allocations in the integration test.

[yebai]

@KDr2 Maybe we should introduce a few benchmark examples that do not depend on Turing. It allows us to better understand the performance bottleneck -- the Turing integration tests are helpful, but it involves many other parts, which make the runtime numbers hard to analyse.

[yebai]

thanks, @KDr2 - it looks good to me. I left some minor comments below.

I think we still need some benchmarks before merging. We need to ensure each Instruction in a TapedFunction is type-sable, whenever the original function f underlying TapedFunction is type-stabe.

[devmotion]

My main suggestion is to use Base.RefValue instead of Ref.

It is unclear to me if/how this affects performance. It would be good to perform some benchmarks (or (re-)post them, if they exist - I assumed the comments above are outdated).

[yebai]

many thanks @KDr2!