Skip to content

Documentation for *TRTRS should note it only detects exact singularity #918

@TiborGY

Description

@TiborGY

Description
The current description of DTRTRS claims that A check is made to verify that A is nonsingular.. Initially I assumed that nonsingular meant "nonsingular within acceptable numerical precision", but after reading the code itself it has become evident that only exact singularity (as in 0.0d+0) triggers an error.

I know it is too late to change this behavior, but was this a good design decision? I would think that at the very least, any subnormally small (< 2^−1022 for FP64) diagonal element is ought to trip the error exit. Perhaps something like tiny(0.0d+0) would have been more appropriate.
This does not only affect *TRTRS, but many of the callers of it, which rely on *TRTRS for their rank-deficiency/singularity checking. For example, DGELS only errors out if the least squares matrix has an exact rank deficiency, even though it is reasonable to assume that being only a subnormal number away from rank deficiency will result in a loss of numerical precision in the computed linear coefficients.

Admittedly all of the above is purely academic. Even if I am correct, it would not be nice to the users if the behaviour of the function was changed, so I am proposing to change the documentation instead, making it immediately clear to the reader that these checks only detect exact singularity/rank-deficiency, and that near-singular cases are not caught.

Checklist

  • [N/A] I've included a minimal example to reproduce the issue
  • I'd be willing to make a PR to solve this issue

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions