-
Notifications
You must be signed in to change notification settings - Fork 474
Description
Description
The current description of DTRTRS claims that A check is made to verify that A is nonsingular.
. Initially I assumed that nonsingular meant "nonsingular within acceptable numerical precision", but after reading the code itself it has become evident that only exact singularity (as in 0.0d+0
) triggers an error.
I know it is too late to change this behavior, but was this a good design decision? I would think that at the very least, any subnormally small (< 2^−1022 for FP64) diagonal element is ought to trip the error exit. Perhaps something like tiny(0.0d+0)
would have been more appropriate.
This does not only affect *TRTRS, but many of the callers of it, which rely on *TRTRS for their rank-deficiency/singularity checking. For example, DGELS only errors out if the least squares matrix has an exact rank deficiency, even though it is reasonable to assume that being only a subnormal number away from rank deficiency will result in a loss of numerical precision in the computed linear coefficients.
Admittedly all of the above is purely academic. Even if I am correct, it would not be nice to the users if the behaviour of the function was changed, so I am proposing to change the documentation instead, making it immediately clear to the reader that these checks only detect exact singularity/rank-deficiency, and that near-singular cases are not caught.
Checklist
- [N/A] I've included a minimal example to reproduce the issue
- I'd be willing to make a PR to solve this issue