Bug: Neodymium Random is not Fixed for each test

[wurzelkuchen]

Quick Description

To rerun certain randomized tests, we introduced a seed, to be set for our random making a test suite reproducible. However, this seed is valid for all test runs per thread.

This has the downside, that we can only rerun the whole suite and if some behavior in another test changes, we are not able to rerun specific tests.

Tasks

• Reset the Random after every test to the configured seed, to make sure we can reproduce test cases independently from each other

Actual Behaviour

When I configure a seed and run a test suite the steps for a test might be different compared to running the test alone.

Expected Behaviour

When I configure a seed and get a result for a complete test suite, and rerun a single test I expect it to pull the same random values as in the suite.

[wurzelkuchen]

To refine the requirements here after we noticed some inconsistencies:

• if NO seed is configured, a new random seed needs to be configured for each test case.
• we need to make sure the new seed is printed into the log and the report if set (and used) but only once.