Greentea Wifi testcase fixes #5568
Conversation
ONME-3266 Greentea testcase WIFI-SCAN fails arbitrarily ONME-3278 Greentea testcase WIFI-SET-CREDENTIAL doesn't try WEP with valid credentials ONME-3279 Greentea testcase WIFI-GET-RSSI uses arbitrary RSSI value limits ONME-3280 Greentea testcase WIFI-CONNECT-PARAMS-VALID-SECURE assumes WPA2
|
@SeppoTakalo , @jarlamsa, Please review |
There was a problem hiding this comment.
Otherwise correct changes but the connect_params_valid_secure() does not seem to make sense anymore.
It is not following the test plan anymore.
Unfortunately we need to have tradeoff and test only against the most common, WPA2, setup. We don't have resources enough to build access point for all.
|
|
||
| if(wifi->connect(MBED_CONF_APP_WIFI_SECURE_SSID, MBED_CONF_APP_WIFI_PASSWORD, NSAPI_SECURITY_WEP) == NSAPI_ERROR_OK) { | ||
| TEST_PASS(); | ||
| } |
There was a problem hiding this comment.
I don't think this works.
Our test environment does not have SSID that would allow you to connect with all these parameters.
You can only expect one to work.
Also, if first one works, how can we expect second connect() to work without disconnecting in between.
There was a problem hiding this comment.
Based on the fact that test case execution should abort immediately when TEST_PASS is called there shouldn't be an issue. WPA2 is tried first so this should behave the same in our environment as the previous solution.
/* It is not necessary for you to call PASS. A PASS condition is assumed if nothing fails.
* This method allows you to abort a test immediately with a PASS state, ignoring the remainder of the test. */
#define TEST_PASS() longjmp(Unity.AbortFrame, 1)
If connect() fails and is repeated with another security protocol this shouldn't be an issue as the failed try should only report error - which gets ignored. There shouldn't be any side-effects. Only if none of the methods succeeds we have an error situation. Lets also remember that these test cases are not only to be run in our test environment.
Also a thing to keep in mind is that WIFI-SET-CREDENTIAL test case checks that all the expected security protocols are supported.
There was a problem hiding this comment.
Our test environment does not have SSID that would allow you to connect with all these parameters.
Our test environment gets covered by the first if-clause, others are to prevent the need to modify the test case itself when someone else is using an access point with WEP/WPA/WPA_WPA2 Mixed mode.
There was a problem hiding this comment.
Actually, can you make if like:
if (...) {
TEST_PASS();
return;
}
just to make it more obvious.
There was a problem hiding this comment.
That's as simple as it can get. TEST_PASS replaced with return. test cases will be marked as passed if there is nothing which would mark them as failed.
|
What about ODIN target testing for this patch? |
Replace 'TEST_PASS' with 'return'
|
Not tried with ODIN as I don't have an access point with which to try at the moment. The ODIN driver won't accept WPA2 - it requires WPA/WPA2. @andreaslarssonublox mentioned in comments of #5582 that WPA2 isn't and won't be supported for now. |
|
Would someone please merge these changes - I would like to create a new PR but those new modifications are build on top of these. |
|
@0xc0170 These are ready to go in. |
|
@VeijoPesonen For future, please provide better headlines for a commit msg where possible, /morph build |
|
@0xc0170 yes, good point. |
Build : SUCCESSBuild number : 671 Triggering tests/morph test |
Exporter Build : SUCCESSBuild number : 315 |
|
"ci-morph-test — tests not started" - What is the trigger for these tests? |
|
/morph test |
Test : SUCCESSBuild number : 519 |
|
@0xc0170 Please merge in. |
Description
Greentea testcase WIFI-SCAN fails arbitrarily
Greentea testcase WIFI-SET-CREDENTIAL doesn't try WEP with valid credentials
Greentea testcase WIFI-GET-RSSI uses arbitrary RSSI value limits
Greentea testcase WIFI-CONNECT-PARAMS-VALID-SECURE assumes WPA2
Testcases in question either have unnecessary expectations about signal quality or make unnecessary assumptions about security features available. Testcases also assume that max password length is 64 characters when 802.11 says it's 63. We might have also access points with hidden SSIDs in vicinity but the scan test didn't like about those too much. This PR addresses all these issues.
Status
READY
Migrations
NO
Steps to test or reproduce