Tests fail on Windows with: Cannot decode build prefix map

[jonahbeckford]

For now, I'm just filing an issue even though I document the solution below. I'm in the middle of too many open PRs.

Expected Behavior

opam exec -- dune build "@merlin-from-subdir" should succeed.

Root Cause

I came across this testing out a new feature patch to Dune.

Consider

dune/test/blackbox-tests/test-cases/merlin/merlin-from-subdir.t/run.t

Lines 1 to 2 in a2a13a7

	$ ocamlc_where="$(ocamlc -where)"
	$ export BUILD_PATH_PREFIX_MAP="/OCAMLC_WHERE=$ocamlc_where:$BUILD_PATH_PREFIX_MAP"

Now ocamlc -where on native Windows will always have a colon. Here is mine: C:\Users\beckf\AppData\Local\Programs\DKMLNA~1\desktop\bc\lib\ocaml

But the build prefix path map specifications require that colons (and two other characters) are escaped. https://reproducible-builds.org/specs/build-path-prefix-map/ recommends that we:

Our recommended approach for a high-level language with string replace:
...
Encoding:

elem.replace("%", "%#").replace("=", "%+").replace(":", "%.")

So replacing ocamlc_where="$(ocamlc -where)" with ocamlc_where="$(ocamlc -where | sed 's/:/%./g')" fixes the problem. The other characters should be replaced as well.

Actual Behavior

Internal error, please report upstream including the contents of _build/log.
Description:
  ("Cannot decode build prefix map",
  { build_path_prefix_map =
      "/OCAMLC_WHERE=C:/Users/beckf/AppData/Local/Programs/DKMLNA~1/lib/ocaml:/workspace_root=Y%.\\source\\dune\\_build/.sandbox/6f6fa7cbeb79aea4c92f98caef489b21:$TESTCASE_ROOT=Y%.\\source\\dune\\_build/.sandbox/6f6fa7cbeb79aea4c92f98caef489b21/default/test/blackbox-tests/test-cases/merlin/merlin-from-subdir.t:$TMPDIR=C%.\\Users\\beckf\\AppData\\Local\\Temp\\dune_cram_6c1269_.merlin-from-subdir.t\\tmp"
  ; msg =
      "invalid key/value pair \"/Users/beckf/AppData/Local/Programs/DKMLNA~1/lib/ocaml\", no '=' separator"
  })
Raised at Stdune__Code_error.raise in file
  "otherlibs/stdune/src/code_error.ml", line 10, characters 30-62
Called from Dune_rules__Cram_exec.sanitize.(fun) in file
  "src/dune_rules/cram/cram_exec.ml", line 300, characters 10-236
Called from Stdlib__List.rev_map.rmap_f in file "list.ml", line 103,
  characters 22-25
Called from Stdune__List.map in file "otherlibs/stdune/src/list.ml", line 5,
  characters 19-33
Called from Dune_rules__Cram_exec.run in file
  "src/dune_rules/cram/cram_exec.ml", line 448, characters 18-62
Called from Fiber__Core.O.(>>|).(fun) in file "vendor/fiber/src/core.ml",
  line 253, characters 36-41
Called from Fiber__Scheduler.exec in file "vendor/fiber/src/scheduler.ml",
  line 76, characters 8-11
Re-raised at Stdune__Exn.raise_with_backtrace in file
  "otherlibs/stdune/src/exn.ml", line 38, characters 27-56
Called from Fiber__Scheduler.exec in file "vendor/fiber/src/scheduler.ml",
  line 76, characters 8-11
Re-raised at Stdune__Exn.raise_with_backtrace in file
  "otherlibs/stdune/src/exn.ml", line 38, characters 27-56
Called from Fiber__Scheduler.exec in file "vendor/fiber/src/scheduler.ml",
  line 76, characters 8-11
Re-raised at Stdune__Exn.raise_with_backtrace in file
  "otherlibs/stdune/src/exn.ml", line 38, characters 27-56
Called from Fiber__Scheduler.exec in file "vendor/fiber/src/scheduler.ml",
  line 76, characters 8-11
-> required by
   ("execute-action",
   { digest = digest "f482ce280e1ac19f343e249879b069ee"
   ; loc =
       { pos_fname = "<none>"
       ; start = { pos_lnum = 1; pos_bol = 0; pos_cnum = 0 }
       ; stop = { pos_lnum = 1; pos_bol = 0; pos_cnum = 0 }
       }
   })
-> required by ("<unnamed>", ())
-> required by
   ("build-alias",
   { dir = In_build_dir "default/test/blackbox-tests/test-cases/merlin"
   ; name = "merlin-from-subdir"
   })
-> required by ("toplevel", ())

I must not crash.  Uncertainty is the mind-killer. Exceptions are the
little-death that brings total obliteration.  I will fully express my cases.
Execution will pass over me and through me.  And when it has gone past, I
will unwind the stack along its path.  Where the cases are handled there will
be nothing.  Only I will remain.

[rgrinberg]

Thanks for the report. We should indeed fix this. I wonder if there's a way to do this purely in sh (or at least bash). Would be unfortunate to spin up sed just for this.

[nojb]

I wonder if there's a way to do this purely in sh (or at least bash).

${ocamlc_where//:/%.} should work in bash.

[gridbugs]

I came across this while cataloguing dune pkg issues on windows. Little repro case is here.