Name: testr
Owner: Zapier
Description: Tools for Bayesian A/B testing
Forked from: ayakubovich/testr
Created: 2017-02-09 20:39:13.0
Updated: 2017-02-09 20:39:14.0
Pushed: 2016-04-09 22:20:13.0
Size: 91
Language: R
GitHub Committers
User | Most Recent Commit | # Commits |
---|
Other Committers
User | Most Recent Commit | # Commits |
---|
testr
is an R package for Bayesian design and analysis of randomized experiments (A/B tests). It uses an adaptive design (i.e. optional stopping rules), allowing tests to end as early as possible [5](5).
Two probability models are currently supported:
testr
provides a convenient API to these models, and extends them in several ways:
Install the package using devtools:
stall.packages('devtools')
ools::install_github('ayakubovich/testr')
ary(testr)
tr
To run a simple test for conversion with a Beta-Binomial model:
_binomial_ab_test(y = c(100,115), n = c(1000, 1000)) # flat prior
_conversion_prior(expected_conversion_rate = .1, alpha0 = 2) # try tweaking alpha0 to decrase variance
_binomial_ab_test(y = c(100,115), n = c(1000, 1000), expected_conversion_rate = .1, alpha0 = 2) # informative prior
To run a test for revenue with a zero-inflated lognormal model:
mulate some data from an A/B/C test:
10000
ta <- rbinom(n, 1, conversion) * rlnorm(n, meanlog = 0, sdlog = 0.2)
ta <- rbinom(n, 1, conversion) * rlnorm(n, meanlog = 0.08, sdlog = 0.2)
ta <- rbinom(n, 1, conversion) * rlnorm(n, meanlog = 0.15, sdlog = 0.2)
<- data.frame(ab_group = rep(c('A', 'B', 'C'), each = n), ltv = c(A_data, B_data, C_data))
ot the priors and run the test:
_revenue_prior(expected_conversion_rate = 0.12, alpha0 = 15, expected_revenue_converted_users = 1.5, v0 = 73, k0 = 100, s_sq0 = 1.2) # specify prior
lognormal_ab_test(data, expected_conversion_rate = 0.65, alpha0 = 15, expected_revenue_converted_users = 1.5, v0 = 73, k0 = 100, s_sq0 = 1.2)
(l)
testr
currently allows for more than two groups in a test, but does not take multiple comparisons into account.