This is a test plan for Mbed OS Wi-Fi API.
The goal of this plan is to test the WiFiInterface class.
This test plan is written in a tool-agnostic way and does not specify how to write, build or run test cases.
The general test environment consists of DUTs, base stations, a network connection and a test server:
<ssid:unsecure>
.<ssid:secure>
, password referred as <pw:secure>
.<ch:secure>
and for unsecure on <ch:unsecure>
.<mac:secure>
and <mac:unsecure>
.mbed_app.json
NOTE: This document refers to an echo server because it is a requirement for running Socket API tests. The test cases defined in this document do not directly use it.
Please refer to the following table for priorities of test cases. Priorities are labeled as MUST and SHOULD. MUST means this is a requirement and therefore mandatory to pass the test. SHOULD means it is recommended to pass the test if the driver implements the feature in question.
Test case | Priority | ||
---|---|---|---|
1 | WIFI_CONSTRUCTOR | MUST | |
2 | WIFI_CONNECT_NOCREDENTIALS | MUST | |
3 | WIFI_SET_CREDENTIAL | MUST | |
4 | WIFI_SET_CHANNEL | SHOULD | |
5 | WIFI_GET_RSSI | SHOULD | |
6 | WIFI_CONNECT_PARAMS_NULL | MUST | |
8 | WIFI_CONNECT_PARAMS_VALID_SECURE | With security type: | |
NSAPI_SECURITY_WEP | SHOULD | ||
NSAPI_SECURITY_WPA | SHOULD | ||
NSAPI_SECURITY_WPA2 | SHOULD | ||
NSAPI_SECURITY_WPA_WPA2 | MUST | ||
NSAPI_SECURITY_WPA3_WPA2 | SHOULD | ||
NSAPI_SECURITY_WPA3 | SHOULD | ||
9 | WIFI_CONNECT_PARAMS_CHANNEL | SHOULD | |
10 | WIFI_CONNECT_PARAMS_CHANNEL_FAIL | SHOULD | |
11 | WIFI_CONNECT | MUST | |
12 | WIFI_CONNECT_DISCONNECT_NONBLOCK | SHOULD | |
13 | WIFI_CONNECT_SECURE | With security type: | |
NSAPI_SECURITY_WEP | SHOULD | ||
NSAPI_SECURITY_WPA | SHOULD | ||
NSAPI_SECURITY_WPA2 | SHOULD | ||
NSAPI_SECURITY_WPA_WPA2 | MUST | ||
NSAPI_SECURITY_WPA3_WPA2 | SHOULD | ||
NSAPI_SECURITY_WPA3 | SHOULD | ||
14 | WIFI_CONNECT_SECURE_FAIL | MUST | |
15 | WIFI_CONNECT_DISCONNECT_REPEAT | MUST | |
16 | WIFI_SCAN_NULL | SHOULD | |
17 | WIFI_SCAN | SHOULD |
For testing the board and driver, test against the Mbed OS master branch for the most recent, up-to-date test cases and drivers.
To create a build environment:
mbed new wifi_test cd wifi_test cd mbed-os git checkout master cd ..
Prepare an mbed_app.json
configuration file with all the required definitions provided. See template_mbed_app.txt file for the full list of necessary definitions.
Now build test binaries:
mbed test --compile -t <toolchain> -m <target> --app-config TESTS/network/wifi/template_mbed_app.txt -n mbed-os-tests-network-wifi
The RAAS user and password have to be set in the shell to access it:
mbedgt -g <target>:raas_client:ruka.mbedcloudtesting.com:8000 -n mbed-os-tests-network-wifi -V -v
Description:
Test that the constructor of the driver works.
Preconditions:
None.
Test step:
Construct the driver by calling the constructor.
Expected result:
Constructor returns.
Description:
Test WiFiInterface::connect()
without parameters. Don't set parameters with set_credentials()
.
This must be a first test to run after constructing the driver. No credentials should be given for the driver before this test.
Precondition:
Test the environment is set up as specified in the "Test Environment" chapter.
Test steps:
WiFiInterface::connect()
.disconnect()
.Expected result:
connect()
call returns NSAPI_ERROR_PARAMETER or NSAPI_ERROR_NO_SSID
.
Description:
This test case is to test whether the driver accepts valid credentials and rejects ones that are not valid.
Precondition:
Driver class is initialized.
Test step:
Call set_credentials()
with various parameters described in "Expected results".
Expected result:
Parameters | Password | Security | Expected return code |
---|---|---|---|
ssid=NULL | NULL | NSAPI_SECURITY_NONE | NSAPI_ERROR_PARAMETER |
ssid="OK" | NULL | NSAPI_SECURITY_WPA_WPA2 | NSAPI_ERROR_PARAMETER |
ssid="OK" | NULL | NSAPI_SECURITY_WPA3_WPA2 | NSAPI_ERROR_PARAMETER |
ssid="OK" | NULL | NSAPI_SECURITY_WEP | NSAPI_ERROR_PARAMETER because password is missing. |
ssid="OK" | NULL | NSAPI_SECURITY_NONE | NSAPI_ERROR_OK |
ssid="OK" | [any 64 character string] | NSAPI_SECURITY_WPA2 | NSAPI_ERROR_PARAMETER because password is too long |
ssid="OK" | [any 63 character string] | NSAPI_SECURITY_WPA2 | NSAPI_ERROR_OK |
ssid="OK" | "" | NSAPI_SECURITY_WPA_WPA2 | NSAPI_ERROR_PARAMETER |
ssid="OK" | "" | NSAPI_SECURITY_WPA3_WPA2 | NSAPI_ERROR_PARAMETER |
ssid="OK" | "" | NSAPI_SECURITY_NONE | NSAPI_ERROR_OK |
ssid="OK" | "12345678" | NSAPI_SECURITY_WPA_WPA2 | NSAPI_ERROR_OK |
ssid="OK" | "12345678" | NSAPI_SECURITY_WPA3_WPA2 | NSAPI_ERROR_OK |
ssid="OK" | "12345678" | NSAPI_SECURITY_WPA2 | NSAPI_ERROR_OK |
ssid="OK" | "12345678" | NSAPI_SECURITY_WPA3 | NSAPI_ERROR_OK |
ssid="OK" | "12345678" | NSAPI_SECURITY_WPA | NSAPI_ERROR_OK |
ssid="OK" | "12345678" | NSAPI_SECURITY_WEP | NSAPI_ERROR_OK or NSAPI_ERROR_UNSUPPORTED |
ssid="" | "" | NSAPI_SECURITY_NONE | NSAPI_ERROR_PARAMETER |
Description:
Test validity of WiFiInterface::set_channel()
.
When proper channel number is feed, it should return NSAPI_ERROR_OK
. API documentation is unclear what to report on error case, but we assume NSAPI_ERROR_PARAMETER
.
If the driver does not support setting channels for scan, it should report NSAPI_ERROR_UNSUPPORTED
for all of the channels.
Precondition:
Driver class is initialized.
Test step:
Call set_channel()
with various parameters described in "Expected results".
Expected result:
For 2.4 Ghz chips:
Channel | Expected return code |
---|---|
0 | NSAPI_ERROR_OK (0 = any channel) |
1 | NSAPI_ERROR_OK |
13 | NSAPI_ERROR_OK if supporting European frequencies |
NSAPI_ERROR_PARAMETER if configured for North America | |
15 | NSAPI_ERROR_PARAMETER |
because not a valid 2.4 Ghz channel |
For 5 Ghz chips:
Channel | Expected return code |
---|---|
3 | NSAPI_ERROR_PARAMETER |
because not a valid 5 Ghz channel number | |
36 | NSAPI_ERROR_OK |
169 | NSAPI_ERROR_PARAMETERs |
should not be allowed in any country |
Drivers not supporting
channel | Expected return code |
---|---|
NSAPI_ERROR_UNSUPPORTED should be returned for ALL channel numbers > 0. |
Description:
Test WiFiInterface::get_rssi()
API. When connected, it should return a valid RSSI value. When unconnected, it should return 0.
API is a bit unclear whether 0 can also mean that driver does not support reporting of RSSI.
Precondition:
Test environment is set up as specified in "Test Environment" chapter.
Test steps:
get_rssi()
, store the result.connect()
with valid SSID parameters. Wait for connection.get_rssi()
.Expected result:
The first call should return 0, and the second call should return a negative number between -10 and -100.
Description:
Test WiFiInterface::connect(ssid, pass, security, channel)
with NULL parameters.
Precondition:
Test environment is set up as specified in "Test Environment" chapter.
Test steps:
WiFiInterface::connect(NULL, NULL)
.WiFiInterface::connect("", "")
.Expected result:
Both connect()
calls return NSAPI_ERROR_PARAMETER.
Description:
Test WiFiInterface::connect(ssid, pass, security)
with valid parameters for secure network.
Preconditions:
Test environment is set up as specified in the "Test Environment" chapter.
Test steps:
WiFiInterface::connect( <ssid:secure>, <pw:secure>, NSAPI_SECURITY_WPA2)
.disconnect()
.Expected result:
connect()
call returns NSAPI_ERROR_OK.
Description:
Test WiFiInterface::connect(ssid, pass, security, channel)
with valid parameters for the secure network using the channel specified.
This test only applies to devices that support selecting the channel (passes WIFI-SET-CHANNEL
).
Precondition:
Test environment is set up as specified in the "Test Environment" chapter.
Test steps:
WiFiInterface::connect( <ssid:secure>, <pw:secure>, NSAPI_SECURITY_WPA2, <ch:secure>)
.disconnect()
.Expected result:
connect()
call returns NSAPI_ERROR_OK.
Description:
Test WiFiInterface::connect(ssid, pass, security, channel)
with valid parameters for the secure network using the channel specified. The channel specified must be wrong, so the connect call can fail.
This test only applies to devices that support selecting the channel (passes WIFI-SET-CHANNEL
).
Precondition:
Test environment is set up as specified in the "Test Environment" chapter.
Test steps:
WiFiInterface::connect( <ssid:secure>, <pw:secure>, NSAPI_SECURITY_WPA2, <ch:secure>)
.disconnect()
.Expected result:
The connect()
call returns NSAPI_ERROR_CONNECTION_TIMEOUT
or NSAPI_ERROR_NO_CONNECTION
.
Description:
Test WiFiInterface::connect()
without parameters. Use set_credentials()
for setting parameters. This checks that driver does not try to retrieve SSID from a location it's not controlling.
Preconditions:
Test steps:
Call WiFiInterface::set_credentials( <ssid:unsecure>, NULL)
.Call WiFiInterface::connect()
.disconnect()
.Call WiFiInterface::set_credentials( <ssid:unsecure>, "")
.Call WiFiInterface::connect()
.disconnect()
.Call WiFiInterface::set_credentials( <ssid:unsecure>, "")
.Call WiFiInterface::connect()
.disconnect()
.Expected result:
connect()
calls return NSAPI_ERROR_OK
.
Description:
Test WiFiInterface::connect()
and WiFiInterface::disconnect()
in non-blocking mode. It checks that driver can connect and disconnect in nonblocking mode.
Preconditions:
Test steps:
Call WiFiInterface::set_credentials( <ssid:secure>, NULL)
.Call WiFiInterface::set_blocking(false)
Call WiFiInterface::connect()
.Cal WiFiInterface::set_credentials(const char *ssid, const char *pass, nsapi_security_t security)
Call WiFiInterface::connect()
.disconnect()
disconnect()
Call WiFiInterface::set_blocking(true)
Expected result:
set_blocking(false)
call returns NSAPI_ERROR_UNSUPPORTED
and skips test case.connect()
call returns NSAPI_ERROR_OK
.set_credentials(...)
call returns NSAPI_ERROR_BUSY
.connect()
call returns NSAPI_ERROR_BUSY
or NSAPI_ERROR_IS_CONNECTED
.NSAPI_STATUS_CONNECTING
and NSAPI_STATUS_CONNECTED
.disconnect()
call returns NSAPI_ERROR_OK
.disconnect()
call returns NSAPI_ERROR_BUSY
or NSAPI_ERROR_IS_CONNECTED
.NSAPI_STATUS_DISCONNECTED
.Description:
Test WiFiInterface::connect()
without parameters. Use secure settings for set_credentials
.
Precondition:
The test environment is set up as specified in the "Test Environment" chapter.
Test steps:
WiFiInterface::set_credentials( <ssid:secure>, <pw:secure>, NSAPI_SECURITY_WPA2)
.WiFiInterface::connect()
.disconnect()
.Expected result:
The connect()
call returns NSAPI_ERROR_OK
.
Description:
Test WiFiInterface::connect()
failing with the wrong password.
Precondition:
The test environment is set up as specified in the "Test Environment" chapter.
Test steps:
WiFiInterface::set_credentials( <ssid:secure>, <any 8 character string, not pw:secure>, NSAPI_SECURITY_WPA2)
.WiFiInterface::connect()
.disconnect()
.Expected result:
connect()
call returns NSAPI_ERROR_AUTH_FAILUR
, NSAPI_ERROR_CONNECTION_TIMEOUT
or NSAPI_ERROR_NO_CONNECTION
.
Description:
Test WiFiInterface::connect()
- disconnect()
repetition works.
Precondition:
The test environment is set up as specified in the "Test Environment" chapter.
Test steps:
WiFiInterface::set_credentials( <ssid:secure>, NULL)
.WiFiInterface::connect()
.disconnect()
.Expected result:
Each connect()
call returns NSAPI_ERROR_OK
.
Each disconnect()
returns NSAPI_ERROR_OK
.
Description:
Call WiFiInterface::scan()
with null parameters to get the number of networks available.
Precondition:
The test environment is set up as specified in the "Test Environment" chapter.
Test steps:
WiFiInterface::scan(NULL, 0)
;.Expected result:
scan()
returns a positive number that is higher or equal to two. (>=2).
Description:
Call WiFiInterface::scan()
with a valid accesspoint list allocated.
Preconditions:
Test steps:
WiFiInterface::scan(<array>, 10)
;Expected result:
scan()
returns a positive number that is higher or equal to two. (>=2).
Scan elements from array indices between 0 and the number returned:
get_ssid()
for each element. <ssid:secure>
and <ssid:unsecure>
must be found.get_rssid()
for each element. The value must be between -10 and -100.For those known networks, test that the following return values are found:
get_ssid() | get_bssid() | get_security() | get_channel() |
---|---|---|---|
<ssid:unsecure> |
<mac:unsecure> |
NSAPI_SECURITY_NONE | <ch:unsecure> |
<ssid:secure> |
<mac:secure> |
NSAPI_SECURITY_WPA2 | <ch:secure> |