make ccall sigatomic (defer SIGINT handling)

[stevengj]

Currently, a ctrl-c in the REPL can interrupt a ccall to an external library, leaving that library in an inconsistent state and producing crashes or other problems later (see e.g. PyCall issue #15).

Julia's C API (julia.h) has SIGATOMIC macros to protect critical code regions from being interrupted by ctrl-c (SIGINT), but these are not easily accessible from Julia.

Arguably, every ccall should be made sigatomic by default, on the theory that C code in general is not interrupt-safe (with a separate interruptable_ccall for cases that are known to be re-entrant and interrupt-safe). Note that Python does the same thing. As noted below, the performance penalty for this is almost certainly negligible under ordinary circumstances.

[JeffBezanson]

see also #1468

[timholy]

In code that repeatedly calls Cairo, I've also seen issues with Ctrl-C.

[stevengj]

Making ccall sigatomic by default seems like the only safe choice. The overhead for this looks like it would be two additions, three loads, two stores, and two comparisons/branches in a typical call; this is negligible for all but the most trivial C function calls.

Moreover, Julia's JIT compiler need not generate the sigatomic checks at all unless jl_install_sigint_handler has been called, which (on a single core) only happens in the REPL by default, so this wouldn't affect non-interactive code.

(One of the comparisons could be eliminated in typical calls by changing the conditional from jl_defer_signal == 0 && jl_signal_pending != 0 to jl_signal_pending != 0 && jl_defer_signal == 0 since both jl_signal_pending and jl_defer_signal will usually be zero.)

[stevengj]

I'm labelling this a bug, since crashing generic C code after SIGINT is not a sane default.

[JeffBezanson]

This is reasonable. Unfortunately (as in #1468) we'd also like to be able to interrupt things like large matrix multiplies, but there doesn't seem to be much we can do about that on our end.

[kmsquire]

@JeffBezanson in #5399:

I'm kind of against this since it adds overhead and probably still won't make everything work.

It would be nice to measure the impact.

How about adding a safe_ccall or atomic_ccall function with the same signature as ccall, but wrapped in sigatomic?

Alternatively, ccall could be made safe by default and unsafe_ccall could be provided. Thoughts?

[StefanKarpinski]

Alternatively, ccall could be made safe by default and unsafe_ccall could be provided.

This seems like the better approach to me if we're going to do this since we would want to default to safe and most cases where someone is calling out to C are not really that performance critical.

[ivarne]

I also think safe-by-default suits Julia well. If we do not have other behavior to group together with interruptable ccalls in a unsafe_ccall, I think a name containing int(errupt) would have advantages.

[JeffBezanson]

I just don't think asynch interrupt is that important. Plus, you might want to break out of a long-running C routine.

This doesn't warrant a safe_ or unsafe_ prefix; it only refers to a very narrow issue of interactively interrupting things. With that approach, the very simplest and most innocent of C routines (e.g. libm functions) would have to be called with unsafe_ccall, which doesn't seem right.

We are already doing delayed binding of ccalls by checking a cached function pointer for NULL. This would add yet another thing.

[stevengj]

There's no point in allowing people to break out of long-running C routines if doing so has a high probability of creating crashes later; I doubt that most long-running C code is interrupt-safe.

I find it hard to believe that the overhead will be anything but negligible except for a very small number of extremely trivial C routines, and decorating these with unsafe_ccall seems like a small price to pay for not having ctrl-C be horribly dangerous.

[stevengj]

(I would tend to call it interruptable_ccall rather than "unsafe", since the former is more informative.)