Switch to LLVM's new pass manager

[kripken]

LLVM has switched their default to the new pass manager. This caused some regressions for us,

#13418 (comment)

and a big compile time regression on SIMD tests,

#13420

For now, we are staying on the old one,

#13426

Hopefully as LLVM stabilizes things over this dev cycle the regressions will go away. If not we may need to live with them.

[kripken]

Investigating the new pass manager on the emscripten benchmark suite, we do have a small regression of 20-30 bytes or so pretty much everywhere - likely the dlmalloc issue we saw before - but there are two large changes:

• Havlak is 13% larger (and no change in speed)
• SQLite is 19% smaller and 8% faster

The Havlak changes go away in -O2 or -Os - they are only present in -O3.

Overall, the dlmalloc issue doesn't look like a symptom of a larger problem. Not sure it's worth filing an issue on - we should probably just accept there are some wins and some losses with the new pass manager.

The SIMD compile time slowdown (8 seconds => 7 minutes) is more serious though, and I think we should block switching to the new pass manager on that, I filed https://bugs.llvm.org/show_bug.cgi?id=49102

[kripken]

Measuring now, almost 2 months later, and close to LLVM 12's release, the numbers are not much changed on code size:

• Most of the benchmark suite regresses by tiny amounts on -Os, less than 1%.
• The picture is more mixed with -O3. Most regress by tiny amounts, but a few change more substantially: fannkuch: -4%, Havlak: +13%, SQLite -19%.

No significant changes in speed.

The SIMD tests still show the same problems as before, on test_avx|sse1|sse2|sse4_1. We need to rewrite those it seems if we want to switch to the new pass manager.

Is there news on the LLVM side? If it's shipping on LLVM 12, then I assume they consider it stable, and we should probably switch to avoid running on an old config. Or has there been a change in plans there?

[dschuff]

A (seemingly?) fairly substantial change to LLVM's inlining heuristic was made in https://reviews.llvm.org/rGa8c2ba105d47
I wonder if it has any noticeable effects for us?

[kripken]

I did a quick check on havlak and fannkuch which had large code size changes in the past based on the pass manager, but I see no significant difference before and after that patch landed.