Problema
Weights that start too big or too small make deep networks hard to train. Two standard schemes pick the scale from the layer size, where fan_in is the number of inputs and fan_out the number of outputs:
| method | returns |
|---|---|
"xavier_uniform" | limit √(6 / (fan_in + fan_out)), weights drawn from [-limit, limit] |
"xavier_normal" | standard deviation √(2 / (fan_in + fan_out)) |
"he_uniform" | limit √(6 / fan_in) |
"he_normal" | standard deviation √(2 / fan_in) |
Xavier (Glorot) suits sigmoid and tanh layers; He suits ReLU layers.
Implement init_bound(method, fan_in, fan_out).
Input
method: One of "xavier_uniform", "xavier_normal", "he_uniform" or "he_normal".
fan_in, fan_out: The layer's number of inputs and outputs.
Output
Return the limit or standard deviation as a decimal number.
Restrições
1 ≤ fan_in, fan_out ≤ 100,000
Exemplos
Entrada:
fan_in = 3
method = "xavier_uniform"
fan_out = 3
Saída esperada:
expected = 1.0Entrada:
long long fan_in = 3;
std::string method = "xavier_uniform";
long long fan_out = 3;
Saída esperada:
double expected = 1.0;Entrada:
long fan_in = 3;
string method = "xavier_uniform";
long fan_out = 3;
Saída esperada:
double expected = 1.0;Explicação
√(6 / (3 + 3)) = √1 = 1, so weights are drawn from [-1, 1].
Entrada:
fan_in = 50
method = "he_normal"
fan_out = 10
Saída esperada:
expected = 0.2Entrada:
long long fan_in = 50;
std::string method = "he_normal";
long long fan_out = 10;
Saída esperada:
double expected = 0.2;Entrada:
long fan_in = 50;
string method = "he_normal";
long fan_out = 10;
Saída esperada:
double expected = 0.2;Explicação
He only uses fan_in: √(2 / 50) = √0.04 = 0.2.
Como abordar
Tente sozinho primeiro e depois abra uma dica de cada vez.
Dica 1
Each method only depends on the layer sizes and whether you need a bound or a spread.
Dica 2
Use fan_in alone for He methods, and fan_in plus fan_out for Xavier methods.
Dica 3
Do not mix up uniform and normal formulas, since one returns a limit and the other returns a standard deviation.
Complexidade esperada
O(1) time, O(1) extra space
Referências
Este desafio foi adaptado para o CodeUtility. Siga o enunciado e as restrições acima.