Problema
Weights that start too big or too small make deep networks hard to train. Two standard schemes pick the scale from the layer size, where fan_in is the number of inputs and fan_out the number of outputs:
| method | returns |
|---|---|
"xavier_uniform" | limit √(6 / (fan_in + fan_out)), weights drawn from [-limit, limit] |
"xavier_normal" | standard deviation √(2 / (fan_in + fan_out)) |
"he_uniform" | limit √(6 / fan_in) |
"he_normal" | standard deviation √(2 / fan_in) |
Xavier (Glorot) suits sigmoid and tanh layers; He suits ReLU layers.
Implement init_bound(method, fan_in, fan_out).
Input
method: One of "xavier_uniform", "xavier_normal", "he_uniform" or "he_normal".
fan_in, fan_out: The layer's number of inputs and outputs.
Output
Return the limit or standard deviation as a decimal number.
Vincoli
1 ≤ fan_in, fan_out ≤ 100,000
Esempi
Input:
fan_in = 3
method = "xavier_uniform"
fan_out = 3
Output atteso:
expected = 1.0Input:
long long fan_in = 3;
std::string method = "xavier_uniform";
long long fan_out = 3;
Output atteso:
double expected = 1.0;Input:
long fan_in = 3;
string method = "xavier_uniform";
long fan_out = 3;
Output atteso:
double expected = 1.0;Spiegazione
√(6 / (3 + 3)) = √1 = 1, so weights are drawn from [-1, 1].
Input:
fan_in = 50
method = "he_normal"
fan_out = 10
Output atteso:
expected = 0.2Input:
long long fan_in = 50;
std::string method = "he_normal";
long long fan_out = 10;
Output atteso:
double expected = 0.2;Input:
long fan_in = 50;
string method = "he_normal";
long fan_out = 10;
Output atteso:
double expected = 0.2;Spiegazione
He only uses fan_in: √(2 / 50) = √0.04 = 0.2.
Come affrontarlo
Prova prima da solo, poi apri un suggerimento alla volta.
Suggerimento 1
Each method only depends on the layer sizes and whether you need a bound or a spread.
Suggerimento 2
Use fan_in alone for He methods, and fan_in plus fan_out for Xavier methods.
Suggerimento 3
Do not mix up uniform and normal formulas, since one returns a limit and the other returns a standard deviation.
Complessità attesa
O(1) time, O(1) extra space
Riferimenti
Questa sfida è adattata per CodeUtility. Segui la traccia e i vincoli sopra.