Only check for deadlocks in deadlock busting thread

[abellina]

1. This PR changes BUFN deadlock detection so it only happens in the deadlock busting thread, not in every free.
2. It removes a JNI->Java callback that was done, instead taking in the java values when needed (in the thread) as a parameter.
3. It also removes a static lock in ThreadStateRegistry

This is a performance improvement for the state machine and is part of #3905

[abellina]

build

[abellina]

build

[greptile-apps]

Greptile Summary

• Moves BUFN deadlock detection from critical path (every free/alloc) to dedicated watchdog thread for performance improvement
• Replaces expensive JNI->Java callback (isThreadBlocked) with Java->JNI parameter passing of blocked thread IDs
• Removes static synchronized lock in ThreadStateRegistry by switching to ConcurrentHashMap

Confidence Score: 4/5

• This PR is safe to merge with careful monitoring of deadlock detection behavior in production.
• Score reflects architectural improvement with one concern: BUFN thread wake-up logic removed from wake_next_highest_priority_blocked means progress now depends entirely on watchdog thread frequency, which could delay deadlock resolution if watchdog doesn't run frequently enough.
• Pay close attention to src/main/cpp/src/SparkResourceAdaptorJni.cpp - the removed BUFN wake-up logic in deallocation path changes deadlock resolution timing.

Important Files Changed

Filename	Overview
src/main/cpp/src/SparkResourceAdaptorJni.cpp	Removes JNI callback in critical path, delegates BUFN deadlock detection to watchdog thread, and simplifies wake logic to only handle THREAD_BLOCKED states
src/main/java/com/nvidia/spark/rapids/jni/ThreadStateRegistry.java	Replaces synchronized HashMap with ConcurrentHashMap, adds blockedThreadIds() method to collect blocked thread state

Sequence Diagram

sequenceDiagram
    participant DT as Deadlock Thread
    participant TSR as ThreadStateRegistry
    participant SRA as SparkResourceAdaptor
    participant Native as C++ spark_resource_adaptor
    participant WT as Worker Thread

    DT->>TSR: blockedThreadIds()
    TSR->>TSR: iterate knownThreads
    TSR->>TSR: check thread state for each
    TSR-->>DT: return blocked thread IDs array
    DT->>SRA: checkAndBreakDeadlocks()
    SRA->>Native: checkAndBreakDeadlocks(handle, blockedThreadIds)
    Native->>Native: check_and_break_deadlocks(java_blocked_thread_ids)
    Native->>Native: check_and_update_for_bufn(lock, java_blocked_thread_ids)
    Native->>Native: is_in_deadlock() with java thread state
    alt Deadlock detected
        Native->>Native: transition thread to BUFN_THROW or RUNNING
        Native->>Native: notify blocked threads
    end
    
    WT->>Native: do_deallocate()
    Native->>Native: dealloc_core()
    Native->>Native: wake_next_highest_priority_blocked()
    Note over Native: Only wakes THREAD_BLOCKED, not BUFN threads

Loading

[greptile-apps]

_{4 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile
React with 👍 or 👎 to share your feedback on this new summary format}

[revans2]

Also why is the cudf version changing? is there something needed by it to be different?

[revans2]

nit: Generally we have always had the logging follow a CSV/TSV like structure so that we can parse it and do post processing/analytics on it. Is the needed? Could this be moved to more of that format?

[abellina]

sure, I can change or remove. It is "just in case", I never like to be deadlocked without finding a way to debug it.

[abellina]

yeah let me check the cudf thing, it's not intended.

[greptile-apps]

_{4 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile
React with 👍 or 👎 to share your feedback on this new summary format}

[abellina]

build

[greptile-apps]

_{4 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile
React with 👍 or 👎 to share your feedback on this new summary format}

[abellina]

build

[greptile-apps]

_{4 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile
React with 👍 or 👎 to share your feedback on this new summary format}

[pxLi]

submodule sync is syncing the latest cudf commits (w/ more breaking changes) and corresponding pinned deps to release/25.12, the cudf ref updates in this change may cause commit roll back after merge. Can we revert the cudf refer update, and trigger CI later? thanks

Aborted the submodule run to allow this one to get merged first