mozilla-services/switchboard-experiments

Name: switchboard-experiments

Owner: Mozilla Services

Description: null

Created: 2016-01-07 21:39:34.0

Updated: 2016-12-05 22:06:54.0

Pushed: 2017-01-23 14:19:21.0

Homepage: null

Size: 46

Language: null

GitHub Committers

UserMost Recent Commit# Commits

Other Committers

UserEmailMost Recent Commit# Commits

README

switchboard-experiments

travis-ci

This configuration is oudated! The experiments configuration is now hosted on Kinto (Firefox 50+). For more information see the Switchboard wiki page.

This repository contains the JSON file that configures the switchboard expermients that are active on Firefox for Android.

Any changes to this file require the approval of a Firefox for Android peer, such as @leibovic, @liuche, or @mfinkle. Additionally, changes that affect release branches (i.e. Aurora/Beta/Release), must have approval from our product and release management teams.

Deployments
Experiment Definitions

Experiment names are defined in the client in Experiments.java and in the Experiments object in browser.js;

UI experiments:

Onboarding experiments are unique because we use local logic to determine whether a client is in an experiment. We do this because we must know if the experiment is active at startup, and we cannot wait to contact the Switchboard server. Given this fact, changes to experiments.json will not affect onboarding experiments. Those experiments are maintained in the client codebase.

Experiment names should not be reused. Becuase we have one config for all clients, we do not have a way to guarantee which version of an experiment is active. So instead, we use new experiment names.

experiments.json Format

The format of experiments.json is a set of experiment name keys, each of which maps to an object with match and buckets keys. These keys determine if the experiment is active or not for a given client.

For example:


xperiment-name": {
"match": {
  "appId": "org.mozilla.fennec"
},
"buckets": {
  "min": "0",
  "max": "50"
}

nother-experiment-name": {
"match": {
  "lang": "eng"
},
"buckets": {
  "min": "0",
  "max": "50"
}
match Key

The match key is a JSON object that contains keys that map to string values. Each key/value pair is a regular expression match requirement for that experiment. Regular expressions are matched by the node backend, and follow this format. Note: this means that if you want an exact string match, as opposed to a RegExp that will match strings that contain your specified string, you must use string delimiters (^ and $ for start and end of string respectively). All key/value pairs must be satisfied for the experiment to be considered a match.

Supported keys are specified by the client, as parameters on the server request. Here is a list of keys that are currently supported in Firefox for Android:

buckets Key

The buckets key is a JSON object that contains two keys, min and max. min and max should be strings containing integer values, 0 <= min <= max <= 100. The bounds are [min, max).

The experiment “experiment-name” below will trigger for buckets 0-49:

eriment-name": {
atch": {},
uckets": {
"min": "0",
"max": "50"

Switchboard is currently configured to have 100 buckets (0-99), but the max value is a non-inclusive upper bound, meaning that unless a value of 100 or higher is used as the max, bucket 99 will not be included. There is no check for out of bounds values, so “disabled” experiments can have min/max buckets keys that are either out of bounds or have min == max.

Making Client Changes

To use Switchboard to expose a new feature to a portion of Firefox users, use a code snippet like this in the client:

SwitchBoard.isInExperiment(this, Experiments.YOUR_EXPERIMENT_NAME)) {
 Do something interesting.

You should define your experiment in Experiments.java, and then add the same experiment name definition to experiments.json. All new experiment names must be documented in this repo.

To force enable a feature for testing on the client, you can use the switchboard-experiments add-on. Note: Support for this has only landed in Firefox 49, but may be uplifted to 47 or 48.

Making Config Changes

You can use the stage branch to test your config changes. Once you push a change to stage, you can start Fennec with an intent extra to specify a custom Switchboard server host:

adb shell am start --es "switchboard-host" "switchboard.stage.mozaws.net" <package-name>

Support for this intent extra is currently on Nightly only, but once this is in all release channels, we should use this to test config changes against all versions of the client.

This work is supported by the National Institutes of Health's National Center for Advancing Translational Sciences, Grant Number U24TR002306. This work is solely the responsibility of the creators and does not necessarily represent the official views of the National Institutes of Health.