-
Notifications
You must be signed in to change notification settings - Fork 1.7k
C++: Reveal false negative in test case #17275
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This reveals that some tests were passing for the wrong reasons. See github#17275
This reveals that some tests were passing for the wrong reasons. See github#17275
This reveals that some tests were passing for the wrong reasons. See github#17275
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes look fine.
Is there any particular reason you want to change the tests, rather than just changing the expected results when you fix the issue? (with a note about why they've happened, I suppose)
... on a second look, I think some of these tests and the related models are a bit naive about collection content, which we do now have in CPP (mostly). I can perhaps find time to fix things a bit. |
Mainly I wanted to be confident that the fix is 100% correct, and it's just easier to get there when there are no suspicious-looking test changes, like losing a TP. I guess I could have described the issues in the PR, but I worry that the discussion could get a bit muddled in the weeds of a PR that affects multiple languages (Java also had a similar PR). |
This reveals that some tests were passing for the wrong reasons. See github#17275
Instead of modifying the existing tests, I think it would be better to add some tests. It's not clear to me that moving things around here means that we are still testing what we were testing before. |
The problem is that the test is going to fail once I merge #17262, and we obviously can't make it not-fail by adding more tests. |
Then the test results should be updated with #17262, and the test shouldn't be modified. |
1053434
to
16c2cf2
Compare
Sounds good to me. I've merged the shared PR. This PR now just updates the inline test annotation, but feel free to close and handle the test however you like. |
Thanks. |
Some of the sinks in this test were flagged for the wrong reason.
The flow into these sinks isn't working properly, but this was not revealed by the test since an alternate, spurious path exists for each of them. The spurious path goes through the implicit read at one of the prior sinks and takes a use-use step.
In one case the false negative can be revealed by swapping the order of the sinks, but in the other case I had to remove some of the sinks to avoid the interference.
This came up while trying to fix the issue with use-use flow after an implicit read, and this test started failing because the fix removes the spurious paths that the test was relying on.