Train-mode batch norm. Given X (n, d), gamma (d,), beta (d,),
running_mean (d,), running_var (d,), momentum and eps, return the tuple
(out, new_running_mean, new_running_var):
out = gamma * (X - mu) / sqrt(var + eps) + betamu and var are the per-feature batch mean and biased variancemomentum * old + (1 - momentum) * batchInput
X =
[[1. 2.]
[3. 6.]]
gamma = [1. 2.]
beta = [0. 1.]
running_mean = [0. 0.]
running_var = [1. 1.]
momentum = 0.9
eps = 1e-05
Output
(array([[-0.999995 , -0.9999975],
[ 0.999995 , 2.9999975]]), array([0.2, 0.4]), array([1. , 1.3]))