[greentea] RTL8195AM netsocket test failures

[yennster]

Description

Environment: Windows 10
Mbed CLI version: 1.8.0
Python version: 2.7.14
IAR version: 7.80.1.11864
ARMC5 version: 5060750
Mbed OS tag: mbed-os-5.10.0-rc2

Summary of netsocket test failures with ARMC5 and the RTL8195AM board:

mbedgt: test suite report:
+-----------------------+-------------------+-----------------------------+---------+--------------------+-------------+
| target                | platform_name     | test suite                  | result  | elapsed_time (sec) | copy_method |
+-----------------------+-------------------+-----------------------------+---------+--------------------+-------------+
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | OK      | 102.49             | default     |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TIMEOUT | 550.15             | default     |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | FAIL    | 523.8              | default     |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | FAIL    | 85.2               | default     |
+-----------------------+-------------------+-----------------------------+---------+--------------------+-------------+
mbedgt: test suite results: 2 FAIL / 1 OK / 1 TIMEOUT
mbedgt: test case report:
+-----------------------+-------------------+-----------------------------+---------------------------------------+--------+--------+---------+--------------------+
| target                | platform_name     | test suite                  | test case                             | passed | failed | result  | elapsed_time (sec) |
+-----------------------+-------------------+-----------------------------+---------------------------------------+--------+--------+---------+--------------------+
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS                      | 1      | 0      | OK      | 0.12               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_CACHE                | 1      | 0      | OK      | 0.21               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_CANCEL               | 1      | 0      | OK      | 5.1                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_EXTERNAL_EVENT_QUEUE | 1      | 0      | OK      | 2.32               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_INVALID_HOST         | 1      | 0      | OK      | 0.54               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_NON_ASYNC_AND_ASYNC  | 1      | 0      | OK      | 0.48               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_SIMULTANEOUS         | 1      | 0      | OK      | 0.57               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_SIMULTANEOUS_CACHE   | 1      | 0      | OK      | 0.28               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_SIMULTANEOUS_REPEAT  | 1      | 0      | OK      | 12.5               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_TIMEOUTS             | 1      | 0      | OK      | 2.29               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | SYNCHRONOUS_DNS                       | 1      | 0      | OK      | 0.03               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | SYNCHRONOUS_DNS_CACHE                 | 1      | 0      | OK      | 0.25               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | SYNCHRONOUS_DNS_INVALID               | 1      | 0      | OK      | 0.24               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | SYNCHRONOUS_DNS_MULTIPLE              | 1      | 0      | OK      | 0.14               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_CONNECT_INVALID             | 1      | 0      | OK      | 0.18               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST                    | 1      | 0      | OK      | 7.24               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_BURST              | 1      | 0      | OK      | 136.53             |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_BURST_NONBLOCK     | 1      | 0      | OK      | 156.99             |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_NONBLOCK           | 1      | 0      | OK      | 31.72              |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ENDPOINT_CLOSE              | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_OPEN_CLOSE_REPEAT           | 1      | 0      | OK      | 0.01               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_OPEN_LIMIT                  | 1      | 0      | OK      | 0.07               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_100K                   | 0      | 0      | ERROR   | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_100K_NONBLOCK          | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_TIMEOUT                | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_SEND_REPEAT                 | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_SEND_TIMEOUT                | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_THREAD_PER_SOCKET_SAFETY    | 1      | 0      | OK      | 5.15               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST                    | 1      | 0      | OK      | 3.93               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_BURST              | 0      | 1      | FAIL    | 437.52             |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_BURST_NONBLOCK     | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_NONBLOCK           | 1      | 0      | OK      | 5.98               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_OPEN_CLOSE_REPEAT           | 1      | 0      | OK      | 0.02               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_OPEN_LIMIT                  | 1      | 0      | OK      | 0.06               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_INVALID              | 1      | 0      | OK      | 0.03               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_REPEAT               | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_TIMEOUT              | 1      | 0      | OK      | 0.16               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT                          | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-DISCONNECT-REPEAT        | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-NOCREDENTIALS            | 0      | 1      | FAIL    | 15.03              |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-PARAMS-CHANNEL           | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-PARAMS-CHANNEL-FAIL      | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-PARAMS-NULL              | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-PARAMS-VALID-SECURE      | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-PARAMS-VALID-UNSECURE    | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-SECURE                   | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-SECURE-FAIL              | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONSTRUCTOR                      | 1      | 0      | OK      | 2.01               |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-GET-RSSI                         | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-SCAN                             | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-SCAN-NULL                        | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-SET-CHANNEL                      | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-SET-CREDENTIAL                   | 0      | 0      | SKIPPED | 0.0                |
+-----------------------+-------------------+-----------------------------+---------------------------------------+--------+--------+---------+--------------------+
mbedgt: test case results: 2 FAIL / 21 SKIPPED / 29 OK / 1 ERROR
mbedgt: completed in 1269.42 sec
mbedgt: exited with code 3
[mbed] ERROR: "mbedgt" returned error.
       Code: 3
       Path: "C:\Users\jenplu01\Repos\oob-5.10\network_test"
       Command: "mbedgt --test-spec .\BUILD\tests\REALTEK_RTL8195AM\ARM\test_spec.json -n mbed-os-tests-network-*,mbed-os-tests-netsocket*"
       Tip: You could retry the last command with "-v" flag for verbose output
---

Issue request type

[ ] Question
[ ] Enhancement
[X] Bug

[yennster]

IAR failures:

mbedgt: test suite report:
+-----------------------+-------------------+-----------------------------+---------+--------------------+-------------+
| target                | platform_name     | test suite                  | result  | elapsed_time (sec) | copy_method |
+-----------------------+-------------------+-----------------------------+---------+--------------------+-------------+
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | FAIL    | 89.51              | default     |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TIMEOUT | 550.23             | default     |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | FAIL    | 294.08             | default     |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | FAIL    | 87.61              | default     |
+-----------------------+-------------------+-----------------------------+---------+--------------------+-------------+
mbedgt: test suite results: 3 FAIL / 1 TIMEOUT
mbedgt: test case report:
+-----------------------+-------------------+-----------------------------+---------------------------------------+--------+--------+---------+--------------------+
| target                | platform_name     | test suite                  | test case                             | passed | failed | result  | elapsed_time (sec) |
+-----------------------+-------------------+-----------------------------+---------------------------------------+--------+--------+---------+--------------------+
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS                      | 1      | 0      | OK      | 0.12               |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_CACHE                | 0      | 1      | FAIL    | 2.6                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_CANCEL               | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_EXTERNAL_EVENT_QUEUE | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_INVALID_HOST         | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_NON_ASYNC_AND_ASYNC  | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_SIMULTANEOUS         | 1      | 0      | OK      | 0.62               |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_SIMULTANEOUS_CACHE   | 1      | 0      | OK      | 0.23               |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_SIMULTANEOUS_REPEAT  | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_TIMEOUTS             | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | SYNCHRONOUS_DNS                       | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | SYNCHRONOUS_DNS_CACHE                 | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | SYNCHRONOUS_DNS_INVALID               | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | SYNCHRONOUS_DNS_MULTIPLE              | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_CONNECT_INVALID             | 1      | 0      | OK      | 0.18               |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST                    | 1      | 0      | OK      | 7.85               |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_BURST              | 0      | 0      | ERROR   | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_BURST_NONBLOCK     | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_NONBLOCK           | 1      | 0      | OK      | 26.75              |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ENDPOINT_CLOSE              | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_OPEN_CLOSE_REPEAT           | 1      | 0      | OK      | 0.01               |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_OPEN_LIMIT                  | 1      | 0      | OK      | 0.09               |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_100K                   | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_100K_NONBLOCK          | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_TIMEOUT                | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_SEND_REPEAT                 | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_SEND_TIMEOUT                | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_THREAD_PER_SOCKET_SAFETY    | 1      | 0      | OK      | 5.57               |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST                    | 1      | 0      | OK      | 4.68               |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_BURST              | 0      | 1      | FAIL    | 196.47             |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_BURST_NONBLOCK     | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_NONBLOCK           | 1      | 0      | OK      | 6.18               |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_OPEN_CLOSE_REPEAT           | 1      | 0      | OK      | 0.01               |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_OPEN_LIMIT                  | 1      | 0      | OK      | 0.08               |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_INVALID              | 1      | 0      | OK      | 0.04               |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_REPEAT               | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_TIMEOUT              | 1      | 0      | OK      | 0.04               |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT                          | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-DISCONNECT-REPEAT        | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-NOCREDENTIALS            | 0      | 1      | FAIL    | 15.03              |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-PARAMS-CHANNEL           | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-PARAMS-CHANNEL-FAIL      | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-PARAMS-NULL              | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-PARAMS-VALID-SECURE      | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-PARAMS-VALID-UNSECURE    | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-SECURE                   | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-SECURE-FAIL              | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONSTRUCTOR                      | 1      | 0      | OK      | 2.17               |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-GET-RSSI                         | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-SCAN                             | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-SCAN-NULL                        | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-SET-CHANNEL                      | 0      | 0      | SKIPPED | 0.0                |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-SET-CREDENTIAL                   | 0      | 0      | SKIPPED | 0.0                |
+-----------------------+-------------------+-----------------------------+---------------------------------------+--------+--------+---------+--------------------+
mbedgt: test case results: 3 FAIL / 33 SKIPPED / 16 OK / 1 ERROR
mbedgt: completed in 1028.94 sec
mbedgt: exited with code 4
[mbed] ERROR: "mbedgt" returned error.
       Code: 4
       Path: "C:\Users\jenplu01\Repos\oob-5.10\network_test"
       Command: "mbedgt --test-spec .\BUILD\tests\REALTEK_RTL8195AM\IAR\test_spec.json -n mbed-os-tests-network-*,mbed-os-tests-netsocket*"
       Tip: You could retry the last command with "-v" flag for verbose output
---

[ashok-rao]

@SeppoTakalo FYI..

[cmonr]

@yennster Would you mind reconfirming that this is still an issue today? The Austin site had some network issues for most of the day yesterday, which created false failures with the netsocket test in CI.

The issues seem to have resolved themselves, so I'm curious if this is still an issue as well.

[MarceloSalazar]

It looks something for @samchuarm to investigate and work with Realtek.

[samchuarm]

Hi @ARMmbed/team-realtek @M-ichae-l , please help look into these GT netsocket test failures for RTL8195AM and address them as soon as you can. Thanks.

[samchuarm]

By the way, do we confirm the failures are not caused by the network issue happened in Austin site as @cmonr mentioned?

[SeppoTakalo]

This is the result from our CI

04:22:48 +---------------------------+-------------------+-----------------------------+---------+--------------------+-------------+
04:22:48 | target                    | platform_name     | test suite                  | result  | elapsed_time (sec) | copy_method |
04:22:48 +---------------------------+-------------------+-----------------------------+---------+--------------------+-------------+
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | TIMEOUT | 199.91             | default     |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TIMEOUT | 560.05             | default     |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | FAIL    | 510.15             | default     |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | FAIL    | 169.83             | default     |
04:22:48 +---------------------------+-------------------+-----------------------------+---------+--------------------+-------------+
04:22:48 mbedgt: test suite results: 2 FAIL / 2 TIMEOUT
04:22:48 mbedgt: test case report:
04:22:48 +---------------------------+-------------------+-----------------------------+------------------------------------+--------+--------+---------+--------------------+
04:22:48 | target                    | platform_name     | test suite                  | test case                          | passed | failed | result  | elapsed_time (sec) |
04:22:48 +---------------------------+-------------------+-----------------------------+------------------------------------+--------+--------+---------+--------------------+
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | mbed-os-tests-netsocket-dns        | 0      | 1      | ERROR   | 199.91             |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_CONNECT_INVALID          | 1      | 0      | OK      | 18.31              |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST                 | 1      | 0      | OK      | 13.33              |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_BURST           | 0      | 0      | ERROR   | 0.0                |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_BURST_NONBLOCK  | 0      | 0      | SKIPPED | 0.0                |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_NONBLOCK        | 1      | 0      | OK      | 30.07              |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ENDPOINT_CLOSE           | 0      | 0      | SKIPPED | 0.0                |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_OPEN_CLOSE_REPEAT        | 1      | 0      | OK      | 0.11               |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_OPEN_LIMIT               | 1      | 0      | OK      | 0.11               |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_100K                | 0      | 0      | SKIPPED | 0.0                |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_100K_NONBLOCK       | 0      | 0      | SKIPPED | 0.0                |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_TIMEOUT             | 0      | 0      | SKIPPED | 0.0                |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_SEND_REPEAT              | 0      | 0      | SKIPPED | 0.0                |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_SEND_TIMEOUT             | 0      | 0      | SKIPPED | 0.0                |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_THREAD_PER_SOCKET_SAFETY | 1      | 0      | OK      | 8.76               |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST                 | 1      | 0      | OK      | 2.18               |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_BURST           | 0      | 1      | FAIL    | 210.94             |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_BURST_NONBLOCK  | 1      | 0      | OK      | 195.44             |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_NONBLOCK        | 1      | 0      | OK      | 9.71               |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_OPEN_CLOSE_REPEAT        | 1      | 0      | OK      | 0.0                |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_OPEN_LIMIT               | 1      | 0      | OK      | 0.11               |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_INVALID           | 1      | 0      | OK      | 0.0                |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_REPEAT            | 1      | 0      | OK      | 0.21               |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_TIMEOUT           | 1      | 0      | OK      | 0.11               |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT                       | 1      | 0      | OK      | 2.53               |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-DISCONNECT-REPEAT     | 1      | 0      | OK      | 22.8               |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-NOCREDENTIALS         | 0      | 1      | FAIL    | 15.07              |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-PARAMS-CHANNEL        | 0      | 1      | FAIL    | 1.73               |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-PARAMS-CHANNEL-FAIL   | 0      | 1      | FAIL    | 1.22               |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-PARAMS-NULL           | 0      | 2      | FAIL    | 17.54              |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-PARAMS-VALID-SECURE   | 1      | 0      | OK      | 5.13               |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-PARAMS-VALID-UNSECURE | 0      | 0      | ERROR   | 0.0                |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-SECURE                | 1      | 0      | OK      | 2.49               |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONNECT-SECURE-FAIL           | 1      | 0      | OK      | 6.77               |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-CONSTRUCTOR                   | 1      | 0      | OK      | 2.29               |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-GET-RSSI                      | 1      | 0      | OK      | 4.14               |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-SCAN                          | 1      | 0      | OK      | 1.31               |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-SCAN-NULL                     | 1      | 0      | OK      | 1.3                |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-SET-CHANNEL                   | 0      | 3      | FAIL    | 0.0                |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi  | WIFI-SET-CREDENTIAL                | 0      | 1      | FAIL    | 0.11               |
04:22:48 +---------------------------+-------------------+-----------------------------+------------------------------------+--------+--------+---------+--------------------+
04:22:48 mbedgt: test case results: 7 FAIL / 7 SKIPPED / 23 OK / 3 ERROR
04:22:48 mbedgt: completed in 1440.12 sec

[samchuarm]

Thanks @SeppoTakalo . In that case, Hi @M-ichae-l , please help look into these failures. Thanks.

[VeijoPesonen]

PR #6292 is missing from new EMAC interface

[samchuarm]

Since the netsocket tests were fully passed when Realtek delivered the EMAC interface, it seems these failures are due to the 5.10 updates.

[M-ichae-l]

Hi,
Sorry for the late reply. I just finished testing the issue with 5.9 and 5.10-rc2. The tests are all passed from my side. As @samchuarm said the reason may be the 5.10 updates or setting error.
In addition, I have found that "RealtekInterface.json" from "~mbed-os\tools\test_configs" is missing after 5.9.