cf.eq, tolerance and subspace

[JonathanGregory]

Dear cf-python authors

https://ncas-cms.github.io/cf-python/function/cf.rtol.html#cf.rtol says

old = cf.rtol(1e-10)
>>> cf.rtol()
<CF Constant: 2.220446049250313e-16>
>>> cf.rtol(old)
<CF Constant: 1e-10>

I believe that those two returned stdin lines are exchanged; the first time, it should return 1e-10, the second time, the restored default value.

I suggest that it would be useful to provide atol and rtol parameters to cf.eq, and perhaps other condition functions, to set the tolerance just for that operation.

https://ncas-cms.github.io/cf-python/function/cf.eq.html#cf.eq explains the value parameter as "The query condition's value". It would be more informative to say something specific to the function, about it being the value that an equality test will use. About the obsolete exact parameter, it says "Use re.compile objects in value instead". Could you explain what this purpose would be?

In https://ncas-cms.github.io/cf-python/method/cf.Field.subspace.html#cf.Field.subspace it would be really helpful if the documentation of keyword parameters gave a link to a page which explains the construction of conditions comprehensively, and with examples. (I know there are some in the tutorial.)

Thanks for your consideration

Jonathan

[davidhassell]

Dear Jonathan,

I believe that those two returned stdin lines are exchanged; the first time, it should return 1e-10, the second time, the restored default value.

Thanks for spotting this. I'll fix the example to (the second 1e-10 was ok all along):

>>> old = cf.rtol(1e-10)
>>> cf.rtol()
<CF Constant: 1e-10>
>>> cf.rtol(old)
<CF Constant: 1e-10>
>>>cf.rtol()
<CF Constant: 2.220446049250313e-16>

This error is actually in cfdm, so will get fixed over at https://github.com/NCAS-CMS/cfdm.

I suggest that it would be useful to provide atol and rtol parameters to cf.eq, and perhaps other condition functions, to set the tolerance just for that operation.

I'd rather not add this to cf.eq, as it goes against the clarity of the == operator being used in all (numerical) cases. Would a new function cf.close suit you?

>>> q = cf.close(3)  # Uses the default rtol and atol to do `np.isclose(x, 3, rtol, atol)`
>>> q = cf.close(3, rtol=1e-4, atol=0.1)  # Does `np.isclose(x, 3, 1e-5, 0.1)`

You could also use a context manager:

>>> with cf.configuration(rtol=1e-4, atol=0.1):
...     q = cf.close(3)
...     print(q == 4)
...     print(q == 3.00000001)
...
False
True

https://ncas-cms.github.io/cf-python/function/cf.eq.html#cf.eq explains the value parameter as "The query condition's value". It would be more informative to say something specific to the function, about it being the value that an equality test will use. About the obsolete exact parameter, it says "Use re.compile objects in value instead". Could you explain what this purpose would be?

How about:

        value:
            The value to be used in the equality test. May be any object that can
            be operated on with ``==``. For testing against `str` types, *value* may
            also be a compiled regular expression (e.g. ``re.compile('^t')``), in which
            case equality with the regular expression is tested with `re.search`.

An example of testing with a regex:

>>> import cf, re
>>> q = cf.eq(re.compile('[a-m]'))
>>> q == 'moo'
>>> True
>>> q == 'zoo'
False

I'll put that in the examples (which currently contain the obsolete example of cf.eq('.*temp')).

In https://ncas-cms.github.io/cf-python/method/cf.Field.subspace.html#cf.Field.subspace it would be really helpful if the documentation of keyword parameters gave a link to a page which explains the construction of conditions comprehensively, and with examples. (I know there are some in the tutorial.)

Agreed. We'll create one.

[JonathanGregory]

Dear David

Thanks for your helpful reply.

I suggest that it would be useful to provide atol and rtol parameters to cf.eq, and perhaps other condition functions, to set the tolerance just for that operation.

You could also use a context manager with cf.configuration(rtol=1e-4, atol=0.1)

Thanks for telling me about with with statement. I didn't know about that---it's impossible to learn everything about python, it seems (although you might have done so). Yes, that's a possibility. It could still be one line, which is good, but it is a bit more typing than parameters, and less obvious (if others, like me, didn't know about with cf.configuration()). You could put it in the documentation.

I'd rather not add this to cf.eq, as it goes against the clarity of the == operator being used in all (numerical) cases. Would a new function cf.close suit you? q = cf.close(3, rtol=1e-4, atol=0.1)

Yes, that would be fine, but isclose is a better name, because it's like numpy, and because close looks like the verb (opposite to open), so confusing.

The reason I suggested keywords for cf.eq is that the same imprecise equality could affect other conditions as well, such as not equal, less than or equal to, greater than or equal to. This could be done by logical combinations of isclose with other tests, but that's longer and less obvious. You wouldn't want to provide fuzzy versions of all of them, I imagine.

        value:
            The value to be used in the equality test. May be any object that can
            be operated on with ``==``. For testing against `str` types, *value* may
            also be a compiled regular expression (e.g. ``re.compile('^t')``), in which
            case equality with the regular expression is tested with `re.search`.

Fine. For those who don't speak regex (personally, I love regex, but not everyone does, surprisingly), you could spell out, "e.g. re.compile('^t') for testing whether a string begins with t".

In https://ncas-cms.github.io/cf-python/method/cf.Field.subspace.html#cf.Field.subspace it would be really helpful if the documentation of keyword parameters gave a link to a page which explains the construction of conditions comprehensively, and with examples. (I know there are some in the tutorial.)

Agreed. We'll create one.

Splendid. Thanks

Jonathan

[davidhassell]

Implemented in v3.15.2