Draft of smart pointers library for Nim

[cooldome]

This is early draft for user interface discussion. I will add tests and more doc once the interface is settled down.

Main design decisions:

• use converters instead pf ., .=, .()= templates. I have found user experience with converters a lot smoother. Error messages with dot related operators are terrible. Luckily, converters no longer need to copy thanks for var T, lent T return type support.
• active use of sink T and lent T
• no make_shared and similar. Nim doesn't have constructors, hence you can't create object in a uniform way. I don't think it is a problem thanks to sink T arguments, it is enough to have single way to create a smart pointer newXXXPtr[T](arg: sink T): Ptr[T]
• ConstPtr[T] is a distinct SharedPtr[T] is on purpose. User can make copy of smart pointer and pass it futher without a right to mutate the underlying object.

Open questions:

• how to make decision when to call getSharedAllocator() and when to call getLocalAllocator()?

[Araq]

I don't like the allocator here but I don't have a better idea either. Maybe a static[proc] type paramter instead?

[cooldome]

I thought about this idea. IMO, it is bad if 2 smart pointers will be treated as incompatible types, just because allocator was different.
I see the following options:

1. No support for different allocators ;(
2. There is only one allocator returned by getAllocator and it user responsibility to set it to thread local allocator, global allocator or custom allocator at start up time.
3. Do what I just did. While overhead seems noticeable, it is possibly less than one might think.
   There is 16 byte alignment requirement for all heap allocations on 64 bit platforms, hence you are likely to overallocate anyway hence there is a space for extra pointer.
4. Do what I just did, but do have mode were Allocator type is actually void and have zero byte size and alloc, dealloc are silently mapped to system's alloc and dealloc. Hence, those users who don't need customisation do not pay for it. Possible ABI compatibility problems due different defined flags used by different libraries. Problems are potentially solvable.

[Araq]

I have some ideas of solving this problem... I'm writing a blog post.

[cooldome]

Cool, I will wait for it then. Or if it is possible to describe it in one sentence please do so

[Araq]

Use a pool for the allocations, make the pool's address aligned so that you can bitmask the pointers to access the pool's header which contains the allocator.

[Araq]

You can perform this atomicDec in an else when you change the check to atomicRead(p.val[].atomicCounter) == 1. And == 0 would be better still, for this you need to store the value of RC-1.

[Araq]

The classical algorithm does the incRef first and then the decRef and doesn't require the dest.val != src.val check.

[cooldome]

I will come back on this one. I haven't seen this algo before hence I can't comment now.

[Araq]

If you store RC-1 here this becomes a 0.

[Araq]

assert here.

[Araq]

how to make decision when to call getSharedAllocator() and when to call getLocalAllocator()?

The default should be thread-safe, IMO. We need to have a faster threadsafe allocator. But I dislike that every unique and shared pointer has a field to the allocator, we need to do something about that.

[Araq]

Oh and also: If you use the optimized refcounting variant that I outlined there is little need for the UniquePtr variant. Though it's still nice because guarantees leak freedom as it cannot ever produce a cyclic data structure. And it helps in preventing race conditions too. I guess this means it's there to stay.

[dom96]

Sorry to be a buzzkill, but this gives me flashbacks of C++ too much. Do we have to adopt this into the stdlib?

[cooldome]

If you are using destructor based objects on heap these smart pointers are very helpful.
Destructor based objects are the only true multithreaded solution we have. For example, I am writing a library that C++ users expected to use and use it in the multithreaded environment. C++ will create object one thread, process on the other thread and serialise on another thread. I don't have a say in how objects jump around the threads. Destructors is the only option I have in this case.

I don't more clever idea than smart pointers. If anyone has please speak up.

If you are using ref object you don't need any of this.

[Araq]

@cooldome The point is that we should/could map ref to a variant of SharedPtr with --gc:destructors.

[cooldome]

@Araq:
I thought that ref and --gc:destructors can coexist without issues. If user explicitly specifies ref then he wants a garbage collector and he gets. In --gc:destructors builtin types like seq/string are not gc baaed hence no garbage collector unless user explicitly requested.

Your =trace function idea, can make it happen.

[Araq]

@cooldome ref is also used for closures (async!) and it's Nim's way of introducing polymorphism so we need a solution for them that works better in a "no GC please" mode.

[Araq]

Stalled PR, we are all busy with ARC, please reopen if you need it.