Precise sigmoid

[awni]

This changes the sigmoid computation to match PyTorch:

import mlx.core as mx
import torch
print(mx.sigmoid(mx.array(-8.0).astype(mx.float16)).item())
print(torch.sigmoid(torch.tensor(-8.0).to(torch.float16)).item())

It's not obviously more accurate. Here is a relative error plot for the old and the new (compared to double). You can see it's a bit worse close to zero but a quite a bit better from > 1 or so.

[angeloskath]

👍