Name: results-collection
Owner: World Wide Web Consortium
Owner: web-platform-tests
Description: Results dashboard for web-platform-tests
Created: 2017-04-07 19:03:45.0
Updated: 2018-05-15 17:57:47.0
Pushed: 2018-05-17 22:53:05.0
Homepage: http://builds.wpt.fyi/
Size: 8889
Language: Python
GitHub Committers
User | Most Recent Commit | # Commits |
---|
Other Committers
User | Most Recent Commit | # Commits |
---|
A dashboard of cross-browser results for web-platform-tests.
It consists of 3 parts:
You'll need Docker. With Docker installed, build the base image and development image, and start a development server instance:
er build -t wptd-base -f Dockerfile.base .
er build -t wptd-dev -f Dockerfile.dev .
il/docker-dev/run.sh
This starts a Docker instance named wptd-dev-instance
.
In one terminal, start the web server:
il/docker-dev/web_server.sh
This will build dependencies and start the Google App Engine development server inside wptd-dev-instance
.
You'll also need to populate the app datastore with some initial data, using util/populate_dev_data.py.
il/populate_dev_data.py \
--server localhost:9999 \
--sdk-root /path/to/google-cloud-sdk \
--creds ~/Downloads/wptdashboard-creds-file.json
Further instructions on using the populate util can be found with the –help flag.
See CONTRIBUTING.md for more information on local development.
We run the tests in the development environment with a Python script run/run.py
which is a thin wrapper around WPT's wpt run
. If you're triaging test failures, use wpt run
.
Ensure that the Docker development image is running (./util/docker-dev/run.sh
). To run a directory of WPT, pass the platform ID and a test path to run/run.py
on the development server:
er exec -it -u $(id -u $USER):$(id -g $USER) wptd-dev-instance \
run/run.py firefox-56.0-linux --path battery-status
config['build_path']
.gcloud
in the wptdashboard
project.All test result data is public. There are two types of gzipped JSON data files we store: test run summary files, and individual test result files.
These are of the pattern: {sha[0:10]}/{platform_id}-summary.json.gz
sha[0:10]
: the first 10 characters of the WPT commit hash that run was tested againstplatform_id
: the key of the platform configuration in webapp/browsers.json
Example: https://storage.googleapis.com/wptd/791e95323d/firefox-56.0-linux-summary.json.gz
(Note that wptd
is the bucket name)
Structure:
An object where the key is the test file name and the value is a list of the type
[number passing subtests, total number subtests]
.
"/test/file/name1.html": [0, 1],
"/test/file/name2.html": [5, 10]
These are of the pattern: {sha[0:10]}/{platform_id}/{test_file_path}
sha[0:10]
: the first 10 characters of the WPT commit hash that run was tested againstplatform_id
: the key of the platform configuration in webapp/browsers.json
test_file_path
: the full WPT path of the test fileExample: https://storage.googleapis.com/wptd/b12daf6ead/safari-10-macos-10.12-sauce/IndexedDB/abort-in-initial-upgradeneeded.html
Structure:
"test": "/test/file/name.html",
"status": "OK",
"message": "The failure message, if exists",
"subtests": [
{
"status": "FAIL",
"name": "The subtest name",
"message": "The failure message, if exists"
}
]
There is no public API for TestRuns, so if you need to access only the most recent results, looking at the main page will give you the latest test SHAs. If you need to access earlier results, an exhaustive search is the only way to do that (see issue #73 and #43).
src/third_party/WebKit/LayoutTests/imported/wpt
testing/web-platform/tests
LayoutTests/imported/w3c/web-platform-tests
Try out http://w3c-test.org/html/semantics/forms/the-input-element/checkbox.html
This doesn't work with some HTTPS tests. Also be advised that the server is not intended for frequent large-scale test runs.
These are the keys in webapp/browsers.json
. They're used to identify a tuple (browser name, browser version, os name, os version).