問題
Weights that start too big or too small make deep networks hard to train. Two standard schemes pick the scale from the layer size, where fan_in is the number of inputs and fan_out the number of outputs:
| method | returns |
|---|---|
"xavier_uniform" | limit √(6 / (fan_in + fan_out)), weights drawn from [-limit, limit] |
"xavier_normal" | standard deviation √(2 / (fan_in + fan_out)) |
"he_uniform" | limit √(6 / fan_in) |
"he_normal" | standard deviation √(2 / fan_in) |
Xavier (Glorot) suits sigmoid and tanh layers; He suits ReLU layers.
Implement init_bound(method, fan_in, fan_out).
Input
method: One of "xavier_uniform", "xavier_normal", "he_uniform" or "he_normal".
fan_in, fan_out: The layer's number of inputs and outputs.
Output
Return the limit or standard deviation as a decimal number.
制約
1 ≤ fan_in, fan_out ≤ 100,000
例
入力：
fan_in = 3
method = "xavier_uniform"
fan_out = 3
期待される出力：
expected = 1.0入力：
long long fan_in = 3;
std::string method = "xavier_uniform";
long long fan_out = 3;
期待される出力：
double expected = 1.0;入力：
long fan_in = 3;
string method = "xavier_uniform";
long fan_out = 3;
期待される出力：
double expected = 1.0;解説
√(6 / (3 + 3)) = √1 = 1, so weights are drawn from [-1, 1].
入力：
fan_in = 50
method = "he_normal"
fan_out = 10
期待される出力：
expected = 0.2入力：
long long fan_in = 50;
std::string method = "he_normal";
long long fan_out = 10;
期待される出力：
double expected = 0.2;入力：
long fan_in = 50;
string method = "he_normal";
long fan_out = 10;
期待される出力：
double expected = 0.2;解説
He only uses fan_in: √(2 / 50) = √0.04 = 0.2.
解き方のヒント
まずは自分で考えてから、ヒントを 1 つずつ開いてみましょう。
ヒント 1
Each method only depends on the layer sizes and whether you need a bound or a spread.
ヒント 2
Use fan_in alone for He methods, and fan_in plus fan_out for Xavier methods.
ヒント 3
Do not mix up uniform and normal formulas, since one returns a limit and the other returns a standard deviation.
目安の計算量
O(1) time, O(1) extra space
参考資料
この課題はCodeUtility用に調整されています。上記の説明と制約に従ってください。