Use Case Four - Iterating through devices using roles

Writing general-purpose test definitions

How to test that the test definition assumptions are valid.

Requirements

• Multi-Node job with at least two devices with usable network interfaces.
• Run operations on each device with a specified role.
• Scale the tests as the group size changes.

Background

YAML test definition files are commonly reused between jobs and scalable tests need to cope with changes in the size of the MultiNode job. Tests should use iteration whenever any kind of comparative performance, saturation or load measurements are required. This allows someone to later amend the JSON to increase the number of clients per server with the valid expectation that all clients will operate equally and that the test definition will not fail when it finds six clients instead of one or two.

It is particularly important that test writers should use iteration whenever any kind of comparative performance, saturation testing or load measurements are required. These are just the kind of jobs where someone will later modify the JSON to increase the number of clients per server, expecting all clients to operate equally and for the YAML to not care that it suddenly has six clients rather than one or two.

Recommendations

Test the defined roles

Always test that any role required by the YAML does exist in the MultiNode job.:

- lava-test-case check-server-role --shell lava-group server
- lava-test-case check-slave-role --shell lava-group slave

This test will fail (lava-group returns non-zero) if the specified role has not been defined for this MultiNode job.

Iterate over each role

#!/bin/sh
for role in `lava-role list`; do
    echo $role
done

Iterate over each device with a specified role

Using set -e will allow the script to fail if the requested role is not defined:

#!/bin/sh
set -e
for device in `lava-group backend`; do
    echo $device
done

Iterate over all devices by role

#!/bin/sh
for role in `lava-role list`; do
    if [ "${role}" = "exception" ]; then
        continue
    fi
    for device in `lava-group $role`; do
        echo $device
    done
done

Clearly indicate test definitions with fixed group sizes.

If the test definition assumes that the group is a particular size or a particular group composition, make this clear to other test writers. To make it obvious when test writers are editing JSON, rename the test definition YAML file to include the word static. To make it obvious to test writers are viewing result bundles and job lists, include the same indication in the test definition name.

Creating an alias in /etc/hosts based on the role

lava-network has optional support via the alias-hosts command to create an alias for each device based on the role of that device. The device names are unique and will vary on each run but are guaranteed to be usable as hostnames. The alias only uses the alphanumeric characters within the role - [a-z0-9] - and appends a two digit count suffix for each device assigned that role.

If the device fails to return a fully qualified hostname, lava-network uses the unqualified hostname (which is the same as the device name). In this example, staging-kvm02 was the only device in the group to return a value for hostname -f:

10.1.1.2      staging-kvm01   slave01
10.1.1.6      staging-kvm02.localdomain       box01
10.1.1.2      staging-kvm03   slave02
10.1.1.3      staging-kvm04   slave03

Caution

Resist the temptation to hardcode aliases into scripts

YAML and the scripts called by the YAML should not make assumptions about the group size or group constitution as those are defined in the JSON. The same YAML file should be to be usable in multiple groups of varying sizes. e.g. if the test definition relies on two roles, server and client, the YAML and the scripts called from the YAML must not fail if there is no server03 or client7 - equally the YAML must still test server08 and client12 if those exist.