Premium problem126. A/B Conversion Test

Medium Locked

Variant A had n_a visitors and c_a conversions; variant B had n_b and c_b. Return (difference, p_value) where the difference is p^B−p^A\hat{p}_B - \hat{p}_A and the p-value comes from a two-sided two-proportion z-test.

Pool the rate for the standard error, since the null hypothesis says both variants share one true rate:

p^=cA+cBnA+nB,SE=p^(1−p^)(1nA+1nB)\hat{p} = \frac{c_A + c_B}{n_A + n_B}, \qquad SE = \sqrt{\hat{p}(1-\hat{p})\left(\tfrac{1}{n_A}+\tfrac{1}{n_B}\right)}

Input

n_a = 1000
c_a = 100
n_b = 1000
c_b = 130

Output

(0.03, 0.03548845046647473)

Premium problem

This one's part of Premium. Unlock the full Probability track plus every other premium problem on the site.

Implement solve(...)