Skip to content

Add benchmarks #133

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Mar 25, 2022
Merged

Add benchmarks #133

merged 4 commits into from
Mar 25, 2022

Conversation

KDr2
Copy link
Member

@KDr2 KDr2 commented Mar 23, 2022

No description provided.

@KDr2 KDr2 linked an issue Mar 23, 2022 that may be closed by this pull request
@KDr2 KDr2 marked this pull request as ready for review March 23, 2022 07:44
@yebai
Copy link
Member

yebai commented Mar 23, 2022

Result for 318064d

benchmarking rosenbrock...
  Run Original Function:  998.907 μs (16 allocations: 6.10 MiB)
  Run TapedFunction:  23.594 ns (0 allocations: 0 bytes)
  Run TapedFunction (compiled):  38.710 ns (1 allocation: 48 bytes)
benchmarking ackley...
  Run Original Function:  2.027 ms (0 allocations: 0 bytes)
  Run TapedFunction:  23.494 ns (0 allocations: 0 bytes)
  Run TapedFunction (compiled):  40.625 ns (1 allocation: 48 bytes)
benchmarking matrix_test...
  Run Original Function:  154.501 μs (16 allocations: [469]
  Run TapedFunction:  22.066 ns (0 allocations: 0 bytes)
  Run TapedFunction (compiled):  38.005 ns (1 allocation: 48 bytes)
benchmarking neural_net...
  Run Original Function:  708.276 ns (4 allocations: 576 bytes)
  Run TapedFunction:  28.743 ns (0 allocations: 0 bytes)
  Run TapedFunction (compiled):  48.381 ns (1 allocation: 48 bytes)

@KDr2 A few (quick) observations:

1. The matrix_test is nearly ten times slower than the original function.
2. Compiled function always allocates (one) more and about 2x slower.

EDIT: it seems TapedFuntion is always faster!

EDIT 2: Perhaps the increased CI time of Turing.jl is from 1) the additional step of constructing TapedFunction (i.e. transforming the CodeInfo)... 2) the copy step of TapedFunction is suboptimal 3) integration between TapedTask and TapedFunction is suboptimal

@yebai yebai merged commit 2f561e4 into master Mar 25, 2022
@delete-merged-branch delete-merged-branch bot deleted the benchmark branch March 25, 2022 19:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add standalone benchmarks for TapedFunction
2 participants