TX_ENABLE_STACK_CHECKING hangs in endless loop

[OriginalGreenGuy]

Describe the bug
The application gets stuck in a do loop in the tx_thread_stack_analyze.c between lines 114 and 135. It is clear in the debugger that the loop will never complete. The loop is checking for the stack fill pattern 0xefefefef using a binary search after checking the pointer are not NULL.

Please also mention any information which could help others to understand
the problem you're facing:
STM32H743
Originally found in H7.3.1.0. Code has not changed since then. Last checked was H7.3.4.0
STM32CubeIDE V1.8.0
Found while executing in Debugger.

To Reproduce

1. Enable Stack Checking in ThreadX config.
2. After all the threads have been instantiated, and the application goes into idle mode waiting for commands from either the web server or the debug port a command is sent over the debug port to report on the status of some variables. This kicks off the debug thread which causes a stack check to occur. When the stack check process starts (code listed above) the end stack pointer is less than the start. This means, that the code above will never converge and endlessly loop.

Expected behavior
Code will hang in endless look in the code as stated in the description

Impact
Cannot check for stack issues using the Stack Check Feature.

Additional context
this was first noted in 2024-04-29 12:16 PM in the STM32 Forum Thread:
https://community.st.com/t5/stm32-mcus-embedded-software/azure-rtos-threadx-stack-checking-stuck-in-loop/m-p/831068#M66971
There were suggestions for work around, but functionality would match original intent.
This problem has gone unattended for more than a year.

[amgross]

From what I understand from the reference link here, the issue occur when analysing stack upon stack overflow.

The issue is that we are doing binary search between current stack pointer to the end of the stack without checking beforehand that there is no overflow which corrupts all the math.

Fix will be to ensure no overflow in before triggering the binary search

[OriginalGreenGuy]

There two issues. One where users were performing the search. In my case, I was not doing any searching but simply turning the feature and after the threads were created, I would pause the debugger to inspect memory. The hang would occur at random points without any stack overflow.