Possible JDK16 issue in 3.0.0: ThreadLocal not initialised?

[StrangeNoises]

So I just upgraded our project to JDK 16 and tried to actually run it. It failed with this:

Caused by: java.lang.NullPointerException: Cannot load from int array because the return value of "java.lang.ThreadLocal.get()" is null
	at com.github.benmanes.caffeine.cache.StripedBuffer$ThreadLocalProbe.get(StripedBuffer.java:346)
	at com.github.benmanes.caffeine.cache.StripedBuffer.expandOrRetry(StripedBuffer.java:204)
	at com.github.benmanes.caffeine.cache.StripedBuffer.offer(StripedBuffer.java:145)
	at com.github.benmanes.caffeine.cache.BoundedLocalCache.afterRead(BoundedLocalCache.java:1208)
	at com.github.benmanes.caffeine.cache.BoundedLocalCache.getIfPresent(BoundedLocalCache.java:1999)
	at com.github.benmanes.caffeine.cache.LocalManualCache.getIfPresent(LocalManualCache.java:56)

(The rest of the stack trace is in our code)

It looks like it's trying to read values from that ThreadLocal in StripedBuffer.ThreadLocalProbe before it's initialised.

I've been running 3.0.0 in JDK 15 for a while just fine, with jdk.unsupported absent, so this looks like it may be a JDK 16 thing. Don't know if relevant but they did do some deprecating stuff in ThreadGroup. OTOH only deprecating (for removal later), not actually removed.

[ben-manes]

Thanks. That is certainly surprising. As you saw from the code, it tries first to use VarHandles which worked fine in JDK15 for me. The thread local is supposed to be a last ditch approach, as the least optimized. I wonder why it had to fallback to that one...

Glancing at the code and I do see my mistake in the thread local version, so I apologize for that. I'll have a fix out by the end of the week.

[StrangeNoises]

hm, in that case we also need to figure out why it's not doing the VarHandles thing if, as you say, and seemed to me, it was fine in JDK15.

For my part I'll put in a full jdk rather than my linked server-jre to check at least it's still happy with jdk.unsupported on board.

Edit: Yes, it is.

[ben-manes]

JDK16 enables more restrictive access. Unfortunately that also disallows our ability to use VarHandles in this class, because we use Doug Lea's approach in java.util.concurrent.atomic.Striped64 to grow a dynamic array (in our case ring buffers, rather than counters). This relies on using a per-thread field to hash on, which the JDK does by inspecting the threadLocalProbe field in Thread.java. In JDK16, this is now no longer available due to module security.

I was hoping that it could be inspected by VarHandles rather than Unsafe, and that would punch through the module layer. Sadly that isn't true, so it fell back to the thread local version where I messed up the initialization order. I am not super thrilled at the thread local version as that won't be quite as fast, since Java's implementation is not known for its speed. I am not really sure if there is any better way since the JDK doesn't provide a facility like Striped64 which is a really scalable (though confusing) implementation.

Caused by: java.lang.IllegalAccessException: module java.base does not open java.lang to unnamed module @324e4822
    at java.base/java.lang.invoke.MethodHandles.privateLookupIn(MethodHandles.java:260)
    at com.github.benmanes.caffeine.cache.StripedBuffer$VarHandleProbe.<clinit>(StripedBuffer.java:323)
    ... 33 more

[ben-manes]

A pre-release of 3.0.1 will be published as a snapshot build by the CI, or available by jitpack. That should fix the initialization problem. I'd like to find a more optimal approach than thread local so I'll leave this open for a little bit to nag me, before I cut a release this week. Any ideas would be greatly appreciated!

For context, we use multiple ring buffers to record the entries that were read and then replay these events against the eviction policy later on, under a lock. This avoids every thread needing to lock in order to manipulate an LRU, where that manipulation is cheap but the frequency of locking is not. As a library, we don't know how many cache instances and how much concurrency an instance is subjected to. To be friendly, instead of pre-allocating a ring buffer array we let it grow on-demand. Because multiple threads might hash to the same array index, we use the probe to rehash so that under load every active thread is on an independent instance. This is how LongAdder works to have offer a scalable counter, where a sum() walks every cell to aggregate the value. Now that we cannot use threadLocalProbe to index against, we have to decide on something else which offers similar efficiencies. That is a bit of a puzzler.

[ben-manes]

The performance impact is tolerable and the below benchmark is single threaded throughput. The best case is to simply hash the thread's id. That is fixed, so multiple threads that index to the same array slot would suffer contention. Any dynamic mapping needs to reassign threads or purge stale thread mappings. For example if we had a concurrent WeakMap<Thread, Integer> (a cache!) then it would be more expensive to maintain than a thread local.

This is why Doug uses the probe to rehash with, thereby spreading the load evenly. In that case Unsafe and VarHandles perform equally. The thread local version drops significantly, but is still acceptable. On my laptop the SlotLookupBenchmark obtained the following results.

Benchmark	Throughput
threadIdHash	343,027,661 ops/s
varHandle	235,478,214 ops/s
unsafe	234,633,243 ops/s
threadLocal	151,262,359 ops/s

The impact to the cache on the multi-threaded GetPutBenchmark is -14M reads/s on my laptop. The raw throughput is still over 100M ops/s, whereas Guava is 25M ops/s when properly tuned (worse at defaults). Since we already exceed our performance requirements, this loss is frustrating but not application visible.

After chatting with Doug, he is kindly considering what a generic user-facing Striped64 class or equivalent might look like. I don't know if that will materialize, but it would be nice to regain that boost if only for my bruised ego. 🙂

[ben-manes]

Released 3.0.1. Thanks again for reporting this bug.