Samsung/js-remote-test

Name: js-remote-test

Owner: Samsung

Description: This repository is includes the automatic test results of iotjs

Created: 2017-03-14 09:15:12.0

Updated: 2017-11-10 02:20:32.0

Pushed: 2018-01-15 14:31:21.0

Homepage: null

Size: 945

Language: Python

GitHub Committers

UserMost Recent Commit# Commits

Other Committers

UserEmailMost Recent Commit# Commits

README

Remote testing environment

The main goal of the project is to run IoT.js and JerryScript tests on low footprint devices such as STM32F4-Discovery reference board. Since the size of the volatile memory (SRAM) can be very limited, the testing happens remotely. This means that the main control flow (test selection, output processing) happens on the host side and just the test execution happens on the device side. The communication is established over serial port or SSH.

The following table shows the supported devices and applications:

| | IoT.js | JerryScript | | :—: | :—: | :—: | | STM32F4-Discovery | 🗸 | 🗸 | | Raspberry Pi 2 | 🗸 | 🗸 | | Artik053 | 🗸 | 🗸 |

In the first step, all the dependencies should be installed:

(Note: Ubuntu 14.04 LTS is not supported, please use 16.04 LTS)

stall missing packages
sh install-deps.sh

wnload subprojects
sh init.sh


Set up STM32F4-Discovery board to test

In case of the stm32f4-discovery devices the communication happens over the serial port. That is why a miniusb (flashing) and a microusb (serial communication) cables are needed. There are prepared NuttX configuration files in the config folder that contain all the settings that is required.

Flash the device without root permission

In order to flash the binary onto the board without the system requires root permission, copy the 49-stlinkv2.rules to /etc/udev/rules.d/.

do cp projects/stlink/etc/udev/rules.d/49-stlinkv2.rules /etc/udev/rules.d/

(Source: https://github.com/texane/stlink/blob/master/doc/compiling.md#permissions-with-udev)

Read/write serial without root permission

Since the board is restarted at every test, the serial device id could change (from /dev/ttyACM0 to /dev/ttyACM1) during testing. To eliminate this side effect, a permanent serial id should be created for the device:

http://hintshop.ludvig.co.nz/show/persistent-names-usb-serial-devices/

You need to complete the previously created udev rule with MODE="0666" to allow the member of others (like us) to read/write the serial device without root permission, because it is required normally.

So your udev ruleset should look like this:

YSTEM=="tty", ATTRS{idVendor}=="<your vendor id>", ATTRS{idProduct}=="<your product id>", MODE="0666", SYMLINK+="STM32F4"

Now you can refer to the device as /dev/STM32F4.


Set up Raspberry Pi 2 to test

Raspberry devices should have a UNIX based operating system, like Raspbian Jessie. Since the communication happens over SSH, an ssh server should be installed on the device:

aspberrypi $ sudo apt-get install openssh-server

After that, Raspberry devices can handle ssh connections. In order to avoid the authentication every time, you should create an ssh key (on your desktop) and share the public key with the device:

@desktop $ ssh-keygen -t rsa
@desktop $ ssh-copy-id pi@address

Since Raspberry devices have much more resources than microcontrollers, it is possible to use other programs to get more precise information from the tested application (iotjs, jerry). Such a program is the Freya tool of Valgrind, that monitors the memory management and provides information about the memory usage.


Set up Artik053 board to test

In case of the artik053 devices, the communication happens over the serial port. You only need a microusb cable in order to test. To handle the device without root permission, please follow the STM32F4-Discovery flash and read/write sections.


Start testrunner

On the host side, it is enough to run the driver.py file.

thon driver.py
Testrunner parameters:
p
fines the application to test {iotjs, jerryscript}.

anch
fines the application branch that should be tested.

ildtype
fines the buildtype for the projects {release, debug}. Just for debugging.

mmit
fines the application commit that should be tested.

vice
fines the target device {stm32f4dis, rpi2, artik053}.

blic
blish the test results to the web projects.
tps://samsung.github.io/iotjs-test-results/
tps://jerryscript-project.github.io/jerryscript-test-results/
t the required environment variables (FIREBASE_USER, FIREBASE_PWD).

$ export FIREBASE_USER=“your_user_email” $ export FIREBASE_PWD=“your_user_password”

mote-path
en the operating system is provided (e.g. in case of Raspberry),
u should define, which folder contains the helper scripts to test.

meout
fines a time (in seconds) when to restart the device.

communication:

ername
fines the username for the Raspberry Pi target.

dress
(v4) address of the device.

al communication:

rt
fines the serial device id (e.g. /dev/ttyACM0)

ud
fines the baud rate (default: 115200)
Examples to run tests
thon driver.py --device stm32f4dis --app iotjs --port /dev/ttyACM0 --baud 115200
thon driver.py --device stm32f4dis --app jerryscript --port /dev/ttyACM0 --baud 115200
thon driver.py --device rpi2 --app iotjs --address a.b.c.d --username pi --remote-path /home/pi/testrunner
thon driver.py --device rpi2 --app jerryscript --address a.b.c.d --username pi --remote-path /home/pi/testrunner
thon driver.py --device artik053 --app iotjs --port /dev/ttyUSB1 --baud 115200
thon driver.py --device artik053 --app jerryscript --port /dev/ttyUSB1 --baud 115200

All the results are written into JSON files that are found in a results folder. Name of the output files are datetime with the following format:

%mm-%ddT%HH.%MM.%SSZ.json      e.g. 2017-04-15T10.02.33Z.json

Every JSON file contain information about the test results (status, output, memory usage), environments (used hashes, commit messages) and the main section sizes of the application (iotjs or jerryscript) binary. These sizes are based on a stripped release, rpi2 build.


This work is supported by the National Institutes of Health's National Center for Advancing Translational Sciences, Grant Number U24TR002306. This work is solely the responsibility of the creators and does not necessarily represent the official views of the National Institutes of Health.