Hardware Float16 on A64fx

[milankl]

On arm's A64fx (i.e. aarch64) the official release (Julia 1.6) converts Float16s to Float32 for arithmetics, although A64fx natively supports Float16

julia> a,b = rand(Float16,2);

julia> @code_llvm a+b
;  @ float.jl:324 within `+'
define half @"julia_+_174"(half %0, half %1) {
top:
  %2 = fpext half %0 to float
  %3 = fpext half %1 to float
  %4 = fadd float %2, %3
  %5 = fptrunc float %4 to half
  ret half %5
}

So @vchuravy suggest to comment out

julia/src/aotcompile.cpp

Lines 622 to 623 in 91c297b

	PM->add(createDemoteFloat16Pass());
	PM->add(createGVNPass());

and build 1.6 from source. This worked fine, and now Float16s are not converted

julia> @code_llvm a+b
;  @ float.jl:324 within `+'
define half @"julia_+_174"(half %0, half %1) {
top:
  %2 = fadd half %0, %1
  ret half %2
}

and give a typical 4x speed-up on arithmetic operations compared to Float64. Unless subnormals are hit, see #40151

[milankl]

make testall of this build yields 2 errors, however, they are not related to Float16, so @vchuravy suggested to include a check for A64fx architecture to address this automatically in the future

Test Summary:                       |     Pass  Fail  Error  Broken     Total
  Overall                           | 38937450     1      1  352594  39290046
    unicode/utf8                    |       19                             19
    ...
    file                            |     1035     1                     1036
    ...
    Distributed                     |                     1                 1

The global RNG seed was 0x392db231c37d2b35344a0bbd2d5677dc.

Error in testset file:
Test Failed at /snx11273/home/ri-mkloewer/exjul/build/julia-1.6.0/test/file.jl:98
  Expression: dirname(t) == d
   Evaluated: "/var/tmp/pbs.4045.a64fx00" == "/var/tmp/pbs.4045.a64fx00/jl_7cuVre"
Error in testset Distributed:
Error During Test at /snx11273/home/ri-mkloewer/exjul/build/julia-1.6.0/test/testdefs.jl:21
  Got exception outside of a @test
  LoadError: Distributed test failed, cmd : `/snx11273/home/ri-mkloewer/exjul/build/julia-1.6.0/usr/bin/julia
-Cnative -J/snx11273/home/ri-mkloewer/exjul/build/julia-1.6.0/usr/lib/julia/sys.so -g1 --startup-file=no
--check-bounds=yes --startup-file=no --depwarn=error /snx11273/home/ri-mkloewer/exjul/build
/julia-1.6.0/usr/share/julia/stdlib/v1.6/Distributed/test/distributed_exec.jl`
  Stacktrace:
    [1] error(s::String)
      @ Base ./error.jl:33
    [2] top-level scope
      @ /snx11273~/exjul/build/julia-1.6.0/usr/share/julia/stdlib/v1.6/Distributed/test/runtests.jl:11
    ...
  in expression starting at /snx11273/home/ri-mkloewer/exjul/build/julia-1.6.0/usr/share/julia/stdlib/v1.6/Distributed/test/runtests.jl:10

[vchuravy]

After chatting with @maleadt today the necessary steps are likely:

1. Check in demote-float16 pass if the JIT target support Float16
2. Add support in multi-versioning for splitting functions based on the presence of Float16 operations.

@yuyichao Should one add a new JL_TARGET_CLONE_FLOAT16? Or is piggybacking on JL_TARGET_CLONE_MATH okay?

[yuyichao]

The cloning flag is a pretty rough heuristic mainly because it's hard to model what can be better optimized exactly and not because I didn't want to be more accurate......

In this case, I think it's fairly easy to detect the usage accurately so having a separate cloning flag should be fine. fp16fml and fullfp16 should probably trigger this condition.

On a related note both ARM and x86 now have bf16 extensions and LLVM also has support for it recently so there can probably be a flag for that too.

[gbaraldi]

So the solution to this is something similar to #43085 ?

[maleadt]

It doesn't need any intrinsics, just conditionally running the pass based on the target machine and integrating with multiversioning.