Propagate arity between compilation units #1594

hhugo · 2024-04-08T09:54:30Z

Fixes #550

Store immutability property of blocks
Try to recover some immutability info from debug events. (currently assuming modules are immutable.
Use this info to compute an approximated shape for registered globals.
Use shape info to compute function arity across compilation units
Shapes are currently written as special comments (//# shapes:) inside the generated js and can be read back if explicitly requested (with the --load-shape FILE.cma.js)

TODOs:

check the behavior with recursive modules
fix with effects
update global_flow ?
better mechanism for shape propagation ?-
store shape in a different file ? ~~yes~~ no, adding support in dune was too annoying.
should the "shape" file be given jsoo on the command line or should it look for it on the file system ? ~~both~~ loading shapes need to be explicit.
update dune rules with additional deps (Jsoo shapes ocaml/dune#10767)

hhugo · 2024-04-08T10:01:34Z

@vouillon, feel free to comment, review and even commit the branch.

compiler/lib/specialize.ml

compiler/lib/flow.ml

compiler/lib/shape.ml

hhugo · 2024-11-19T15:36:05Z

@vouillon, I think we could merge this before we have the support merged inside dune.
Would you want to give it another round of review ?

hhugo · 2024-11-22T08:59:25Z

It increases to the time to build the toplevel in whole program compilation by ~10%. But we don't benefit from shapes in such case.

vouillon · 2024-11-26T15:15:31Z

The following command spends a lot of time in Flow.the_shape_of (called from Specialize.function_arity).

js_of_ocaml `which ocamlc.byte`

compiler/tests-full/shapes.cma.expected.js

compiler/lib-wasm/link.ml

compiler/lib/shape.ml

- Propagate shape information through the flow analysis

This reverts commit 9c0692f8af4d6fafdf0fc428f820cc9753897cc4.

CHANGES: ## Features/Changes * Misc: drop support for OCaml 4.12 and bellow * Misc: switch to dune.3.19 * Misc: initial support for ocaml 5.4 (ocsigen/js_of_ocaml#2030, ocsigen/js_of_ocaml#2058) * Compiler: support for OCaml 4.14.3+trunk (ocsigen/js_of_ocaml#1844) * Compiler: add the `--empty-sourcemap` flag * Compiler: improve debug/sourcemap location of closures (ocsigen/js_of_ocaml#1947) * Compiler: optimize compilation of switches (ocsigen/js_of_ocaml#1921, ocsigen/js_of_ocaml#2057) * Compiler: evaluate statically more primitives (ocsigen/js_of_ocaml#1912, ocsigen/js_of_ocaml#1915, ocsigen/js_of_ocaml#1965, ocsigen/js_of_ocaml#1969) * Compiler: rewrote inlining pass (ocsigen/js_of_ocaml#1935, ocsigen/js_of_ocaml#2018, ocsigen/js_of_ocaml#2027) * Compiler: improve tailcall optimization (ocsigen/js_of_ocaml#1943) * Compiler: improve deadcode optimization (ocsigen/js_of_ocaml#1963, ocsigen/js_of_ocaml#1962, ocsigen/js_of_ocaml#1967) * Compiler: deadcode elimination of cyclic values (ocsigen/js_of_ocaml#1978) * Compiler: remove empty blocks (ocsigen/js_of_ocaml#1934) * Compiler: improve coloring optimization (ocsigen/js_of_ocaml#1971, ocsigen/js_of_ocaml#1984, ocsigen/js_of_ocaml#1986, ocsigen/js_of_ocaml#1989) * Compiler: faster constant sharing (ocsigen/js_of_ocaml#1988) * Compiler: faster js code generation (ocsigen/js_of_ocaml#1985) * Compiler: improve performance of Javascript linking * Compiler: more efficient code generation from bytecode (ocsigen/js_of_ocaml#1972) * Compiler: faster compilation by improving the scheduling of optimization passes (ocsigen/js_of_ocaml#1962, ocsigen/js_of_ocaml#2001, ocsigen/js_of_ocaml#2012, ocsigen/js_of_ocaml#2027) * Compiler: faster compilation by stopping sooner when optimizations become unproductive (ocsigen/js_of_ocaml#1939) * Compiler: Propagate arity between compilation units (ocsigen/js_of_ocaml#1594) * Compiler: Add flags to enable/disable warnings (ocsigen/js_of_ocaml#2052) * Compiler/wasm: directly write Wasm binary modules (ocsigen/js_of_ocaml#2000, ocsigen/js_of_ocaml#2003) * Compiler/wasm: faster wat output (ocsigen/js_of_ocaml#1992) * Compiler/wasm: use a Wasm text files preprocessor (ocsigen/js_of_ocaml#1822) * Compiler/wasm: optimize integer operations (ocsigen/js_of_ocaml#2032) * Compiler/wasm: use type analysis to remove some unnecessary uses of JavasScript strict equality (ocsigen/js_of_ocaml#2040) * Compiler/wasm: use more precise environment types (ocsigen/js_of_ocaml#2041) * Compiler/wasm: optimize calls to statically known function (ocsigen/js_of_ocaml#2044) * Runtime: use es6 class (ocsigen/js_of_ocaml#1840) * Runtime: support more Unix functions (ocsigen/js_of_ocaml#1829) * Runtime: remove polyfill for Map to simplify MlObjectTable implementation (ocsigen/js_of_ocaml#1846) * Runtime: refactor caml_xmlhttprequest_create implementation (ocsigen/js_of_ocaml#1846) * Runtime: update constant imports to use `node:fs` module (ocsigen/js_of_ocaml#1850) * Runtime: make Obj.dup work with floats and boxed numbers (ocsigen/js_of_ocaml#1871) * Runtime: delete BigStringReader, one should use UInt8ArrayReader instead * Runtime: less conversion during un-marshalling (ocsigen/js_of_ocaml#1889) * Runtime: use TextEncoder/TextDecoder for utf8-utf16 conversions * Runtime: use Dataview to convert between floats and bit representation * Runtime: optimize Str.search_forward/search_backward (ocsigen/js_of_ocaml#2056) * Runtime: deprecate caml_ba_create_from (ocsigen/js_of_ocaml#2056) * Runtime: check for unused variable in the runtime (ocsigen/js_of_ocaml#2056) * Runtime/wasm: implement BLAKE2b primitives for Wasm (ocsigen/js_of_ocaml#1873) * Runtime/wasm: support jsoo_env and keep track of backtrace status (ocsigen/js_of_ocaml#1881) * Runtime/wasm: support unmarshaling compressed data (ocsigen/js_of_ocaml#1898) * Runtime/wasm: make resuming a continuation more efficient in Wasm (ocsigen/js_of_ocaml#1892) * Runtime/wasm: use imported string constants for JavaScript strings (ocsigen/js_of_ocaml#2022) * Runtime/wasm: use DataView primitives to implement bigarrays (ocsigen/js_of_ocaml#1979) * Ppx: explicitly disallow polymorphic method (ocsigen/js_of_ocaml#1897) * Ppx: allow "function" in object literals (ocsigen/js_of_ocaml#1897) * Lib: add Dom_html.window.matchMedia & Dom_html.mediaQueryList (ocsigen/js_of_ocaml#2017) * Lib: make the Wasm version of Json.output work with native ints and JavaScript objects (ocsigen/js_of_ocaml#1872) ## Bug fixes * Compiler: fix stack overflow issues with double translation (ocsigen/js_of_ocaml#1869) * Compiler: minifier fix (ocsigen/js_of_ocaml#1867) * Compiler: fix shortvar with --enable es6 (AssignTarget was not properly handled) * Compiler: fix assert failure with double translation (ocsigen/js_of_ocaml#1870) * Compiler: fix path rewriting of Wasm source maps (ocsigen/js_of_ocaml#1882) * Compiler: fix global dead code in presence of dead tailcall (ocsigen/js_of_ocaml#2010) * Compiler/wasm: fix bound check for empty float array (ocsigen/js_of_ocaml#1904) * Runtime: fix path normalization (ocsigen/js_of_ocaml#1848) * Runtime: fix reading from the pseudo-filesystem (ocsigen/js_of_ocaml#1859) * Runtime: fix initialization of standard streams under Windows (ocsigen/js_of_ocaml#1849) * Runtime: fix Int64.of_string overflow check (ocsigen/js_of_ocaml#1874) * Runtime: fix caml_string_concat when not using JS strings (ocsigen/js_of_ocaml#1874) * Runtime: consistent bigarray hashing across all architectures (ocsigen/js_of_ocaml#1977) * Runtime: fix caml_utf8_of_utf16 bug in high surrogate case (ocsigen/js_of_ocaml#2008) * Runtime: fix method lookup (ocsigen/js_of_ocaml#2034, ocsigen/js_of_ocaml#2038, ocsigen/js_of_ocaml#2039) * Lib: fix Dom_html.Keyboard_code.of_event (ocsigen/js_of_ocaml#1878) * Tools: fix jsoo_mktop and jsoo_mkcmis (ocsigen/js_of_ocaml#1877) * Toplevel: fix for when use-js-strings is disabled (ocsigen/js_of_ocaml#1997)

CHANGES: ## Features/Changes * Misc: drop support for OCaml 4.12 and bellow * Misc: switch to dune.3.19 * Misc: initial support for ocaml 5.4 (ocsigen/js_of_ocaml#2030, ocsigen/js_of_ocaml#2058) * Compiler: support for OCaml 4.14.3+trunk (ocsigen/js_of_ocaml#1844) * Compiler: add the `--empty-sourcemap` flag * Compiler: improve debug/sourcemap location of closures (ocsigen/js_of_ocaml#1947) * Compiler: optimize compilation of switches (ocsigen/js_of_ocaml#1921, ocsigen/js_of_ocaml#2057) * Compiler: evaluate statically more primitives (ocsigen/js_of_ocaml#1912, ocsigen/js_of_ocaml#1915, ocsigen/js_of_ocaml#1965, ocsigen/js_of_ocaml#1969) * Compiler: rewrote inlining pass (ocsigen/js_of_ocaml#1935, ocsigen/js_of_ocaml#2018, ocsigen/js_of_ocaml#2027) * Compiler: improve tailcall optimization (ocsigen/js_of_ocaml#1943) * Compiler: improve deadcode optimization (ocsigen/js_of_ocaml#1963, ocsigen/js_of_ocaml#1962, ocsigen/js_of_ocaml#1967) * Compiler: deadcode elimination of cyclic values (ocsigen/js_of_ocaml#1978) * Compiler: remove empty blocks (ocsigen/js_of_ocaml#1934) * Compiler: improve coloring optimization (ocsigen/js_of_ocaml#1971, ocsigen/js_of_ocaml#1984, ocsigen/js_of_ocaml#1986, ocsigen/js_of_ocaml#1989) * Compiler: faster constant sharing (ocsigen/js_of_ocaml#1988) * Compiler: faster js code generation (ocsigen/js_of_ocaml#1985, ocsigen/js_of_ocaml#2066) * Compiler: improve performance of Javascript linking * Compiler: more efficient code generation from bytecode (ocsigen/js_of_ocaml#1972) * Compiler: faster compilation by improving the scheduling of optimization passes (ocsigen/js_of_ocaml#1962, ocsigen/js_of_ocaml#2001, ocsigen/js_of_ocaml#2012, ocsigen/js_of_ocaml#2027) * Compiler: faster compilation by stopping sooner when optimizations become unproductive (ocsigen/js_of_ocaml#1939) * Compiler: Propagate arity between compilation units (ocsigen/js_of_ocaml#1594) * Compiler: Add flags to enable/disable warnings (ocsigen/js_of_ocaml#2052) * Compiler/wasm: directly write Wasm binary modules (ocsigen/js_of_ocaml#2000, ocsigen/js_of_ocaml#2003) * Compiler/wasm: faster wat output (ocsigen/js_of_ocaml#1992) * Compiler/wasm: use a Wasm text files preprocessor (ocsigen/js_of_ocaml#1822) * Compiler/wasm: optimize integer operations (ocsigen/js_of_ocaml#2032) * Compiler/wasm: use type analysis to remove some unnecessary uses of JavasScript strict equality (ocsigen/js_of_ocaml#2040) * Compiler/wasm: use more precise environment types (ocsigen/js_of_ocaml#2041) * Compiler/wasm: optimize calls to statically known function (ocsigen/js_of_ocaml#2044) * Runtime: use es6 class (ocsigen/js_of_ocaml#1840) * Runtime: support more Unix functions (ocsigen/js_of_ocaml#1829) * Runtime: remove polyfill for Map to simplify MlObjectTable implementation (ocsigen/js_of_ocaml#1846) * Runtime: refactor caml_xmlhttprequest_create implementation (ocsigen/js_of_ocaml#1846) * Runtime: update constant imports to use `node:fs` module (ocsigen/js_of_ocaml#1850) * Runtime: make Obj.dup work with floats and boxed numbers (ocsigen/js_of_ocaml#1871) * Runtime: delete BigStringReader, one should use UInt8ArrayReader instead * Runtime: less conversion during un-marshalling (ocsigen/js_of_ocaml#1889) * Runtime: use TextEncoder/TextDecoder for utf8-utf16 conversions * Runtime: use Dataview to convert between floats and bit representation * Runtime: optimize Str.search_forward/search_backward (ocsigen/js_of_ocaml#2056) * Runtime: deprecate caml_ba_create_from (ocsigen/js_of_ocaml#2056) * Runtime: check for unused variable in the runtime (ocsigen/js_of_ocaml#2056) * Runtime/wasm: implement BLAKE2b primitives for Wasm (ocsigen/js_of_ocaml#1873) * Runtime/wasm: support jsoo_env and keep track of backtrace status (ocsigen/js_of_ocaml#1881) * Runtime/wasm: support unmarshaling compressed data (ocsigen/js_of_ocaml#1898) * Runtime/wasm: make resuming a continuation more efficient in Wasm (ocsigen/js_of_ocaml#1892) * Runtime/wasm: use imported string constants for JavaScript strings (ocsigen/js_of_ocaml#2022) * Runtime/wasm: use DataView primitives to implement bigarrays (ocsigen/js_of_ocaml#1979) * Ppx: explicitly disallow polymorphic method (ocsigen/js_of_ocaml#1897) * Ppx: allow "function" in object literals (ocsigen/js_of_ocaml#1897) * Lib: add Dom_html.window.matchMedia & Dom_html.mediaQueryList (ocsigen/js_of_ocaml#2017) * Lib: make the Wasm version of Json.output work with native ints and JavaScript objects (ocsigen/js_of_ocaml#1872) ## Bug fixes * Compiler: fix stack overflow issues with double translation (ocsigen/js_of_ocaml#1869) * Compiler: minifier fix (ocsigen/js_of_ocaml#1867) * Compiler: fix shortvar with --enable es6 (AssignTarget was not properly handled) * Compiler: fix assert failure with double translation (ocsigen/js_of_ocaml#1870) * Compiler: fix path rewriting of Wasm source maps (ocsigen/js_of_ocaml#1882) * Compiler: fix global dead code in presence of dead tailcall (ocsigen/js_of_ocaml#2010) * Compiler/wasm: fix bound check for empty float array (ocsigen/js_of_ocaml#1904) * Runtime: fix path normalization (ocsigen/js_of_ocaml#1848) * Runtime: fix reading from the pseudo-filesystem (ocsigen/js_of_ocaml#1859) * Runtime: fix initialization of standard streams under Windows (ocsigen/js_of_ocaml#1849) * Runtime: fix Int64.of_string overflow check (ocsigen/js_of_ocaml#1874) * Runtime: fix caml_string_concat when not using JS strings (ocsigen/js_of_ocaml#1874) * Runtime: consistent bigarray hashing across all architectures (ocsigen/js_of_ocaml#1977) * Runtime: fix caml_utf8_of_utf16 bug in high surrogate case (ocsigen/js_of_ocaml#2008) * Runtime: fix method lookup (ocsigen/js_of_ocaml#2034, ocsigen/js_of_ocaml#2038, ocsigen/js_of_ocaml#2039) * Lib: fix Dom_html.Keyboard_code.of_event (ocsigen/js_of_ocaml#1878) * Tools: fix jsoo_mktop and jsoo_mkcmis (ocsigen/js_of_ocaml#1877) * Toplevel: fix for when use-js-strings is disabled (ocsigen/js_of_ocaml#1997)

hhugo requested a review from vouillon April 8, 2024 09:59

hhugo force-pushed the shapes branch from b908b19 to ee229f5 Compare April 12, 2024 08:09