Quality Assurance

icon for testing

Test as a Service

Nard offer a regression testing cloud service to fulfill product requirements. As part of your nightly builds the final SD-card image can be uploaded to our TaaS where an automated audit will take place. If all tests pass one can be pretty sure the binary image work as expected.

The audit is accomplished on real hardware. A Raspberry Compute Module is connected via custom built electronics to a dedicated server for thorough monitoring and control during the tests. Uploaded SD-card images fully replace the Raspberry flash content and thus the system can always be recovered should it need to. Don't worry if you blunder and brick the device – Nard TaaS will heal automatically. The service is fully integrated into the build system and issued with a simple command:

$ make skeleton
$ make test
Nard TaaS is online and unoccupied; uploading image to http://www ...
#########################################################     79.6%

--------- Unpacking ------------
MD5 of audit.tar.gz: 9e4b9ac5ae65ce839918c5d7856c3073
Unpacking finished

--------- Flashing ------------
Uploading image to Compute Module...
Success, resetting Compute Module into normal operation...

--------- Boot console (short) ------------
Uncompressing Linux... done, booting the kernel.
[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 3.10.19 (rln@foo) (gcc version 4.4.5 ...
... snipp ...

--------- Tests summary ------------
Executing test-003-heartbeat-led.sh... OK
Executing test-005-ping.sh... OK
Executing test-009-node-name.sh... OK
Executing test-048-ssh-root-key.sh... OK
Executing test-051-ssh-host-key.sh... OK
Executing test-053-scp-file-transfer.sh... OK
Executing test-056-inetd-enable-many-ssh.sh... OK
Executing test-060-set-time-ntp.sh... OK
Executing test-065-syslog.sh... OK
Executing test-068-automount.sh... OK
Executing test-080-gdb-hello-world.sh... OK
Executing test-090-mount-usb-mem-stick.sh... OK
Executing test-170-sqlite.sh... OK
Executing test-190-openssl.sh... OK
Executing test-195-wget-https.sh... OK
Executing test-300-python.sh... OK
Executing test-310-i2c.sh... OK
Executing test-320-samba.sh... OK
Executing test-330-perl.sh... OK
Executing test-380-rrdtool.sh... OK
Executing test-800-sdcard-removal.sh... OK
Executing test-810-usb-hotplug.sh... OK
Audit success, all tests passed.

Had the audit failed make test would have exited with an error instead. This is perfect for Continuous Integration (CI) with Jenkins or even a daily scheduled cron job. It's a good habit to audit your upgrades before sending them away to any remotely located device.

Sequencing

Specifying what to test is done with a configuration that accompanies the product recipe. (Remember that recipes inherit from each other and "add up".) Here is a simplified example:

taas config

This will test that SSH, I2C and SD-card hotplugging works in your image. Any product building "on top" of the board raspi_x_revx will do those tests too. Here is complete list of what can be specified in taasSeqs:

heartbeat The slowly flashing green LED, indicating watchdog is running
net IPv4 and discoverable with make scan and broadcast ping
ssh SSH and SCP login with the key generated during build
syslog Sets clock with NTP and verify system log
automount Mounts SD-card only when in use
sys Misc basics, such as correct file ownerships
gdb cross-compilig with flags for debugability of everything (incl. syslibs)
usb Hotplug and mount a USB memorystick
sqlite Creates a SQLite database
openssl Creates an encrypted connection to a webserver
wget-https Downloads from a webserver with TLS encryption
python2 Basic Python 2.7 language
i2c I2C communication and related drivers
samba Windows shares exporting and mounting
perl Basic Perl language
minidlna The MiniDLNA daemon (UPnP)
rrdtool Data logging and statistics graphing
sdcard Hotswap the SD-card

If you need widened tests for a custom product; contact the authors.

Login

Access to the online test service is restricted by a password and a firewall. Everybody who donates will however get access automatically. Send a mail to the authors if you need assistance.

Disclaimer

The service is provided "as is" without warranty of any kind. It is used internally by Nard developers and when idle made available publicly. Please use it responsibly since it's a shared resource!


icon version control

Version control

Functionality and bugfixes are first added into the project GIT development branch. Occasionally, when it becomes stable, it's merged into the master branch and a release is made. Detailed source code history is available in the repository:

$ git clone http://www.arbetsmyra.dyndns.org/nard/download/nard.git

icon log

Changelog

A source code modification summary is published in the Changelog at each stable release.




icon arrow prev