Условие
Weights that start too big or too small make deep networks hard to train. Two standard schemes pick the scale from the layer size, where fan_in is the number of inputs and fan_out the number of outputs:
| method | returns |
|---|---|
"xavier_uniform" | limit √(6 / (fan_in + fan_out)), weights drawn from [-limit, limit] |
"xavier_normal" | standard deviation √(2 / (fan_in + fan_out)) |
"he_uniform" | limit √(6 / fan_in) |
"he_normal" | standard deviation √(2 / fan_in) |
Xavier (Glorot) suits sigmoid and tanh layers; He suits ReLU layers.
Implement init_bound(method, fan_in, fan_out).
Input
method: One of "xavier_uniform", "xavier_normal", "he_uniform" or "he_normal".
fan_in, fan_out: The layer's number of inputs and outputs.
Output
Return the limit or standard deviation as a decimal number.
Ограничения
1 ≤ fan_in, fan_out ≤ 100,000
Примеры
Ввод:
fan_in = 3
method = "xavier_uniform"
fan_out = 3
Ожидаемый вывод:
expected = 1.0Ввод:
long long fan_in = 3;
std::string method = "xavier_uniform";
long long fan_out = 3;
Ожидаемый вывод:
double expected = 1.0;Ввод:
long fan_in = 3;
string method = "xavier_uniform";
long fan_out = 3;
Ожидаемый вывод:
double expected = 1.0;Пояснение
√(6 / (3 + 3)) = √1 = 1, so weights are drawn from [-1, 1].
Ввод:
fan_in = 50
method = "he_normal"
fan_out = 10
Ожидаемый вывод:
expected = 0.2Ввод:
long long fan_in = 50;
std::string method = "he_normal";
long long fan_out = 10;
Ожидаемый вывод:
double expected = 0.2;Ввод:
long fan_in = 50;
string method = "he_normal";
long fan_out = 10;
Ожидаемый вывод:
double expected = 0.2;Пояснение
He only uses fan_in: √(2 / 50) = √0.04 = 0.2.
Как подступиться
Сначала попробуйте сами, затем открывайте подсказки по одной.
Подсказка 1
Each method only depends on the layer sizes and whether you need a bound or a spread.
Подсказка 2
Use fan_in alone for He methods, and fan_in plus fan_out for Xavier methods.
Подсказка 3
Do not mix up uniform and normal formulas, since one returns a limit and the other returns a standard deviation.
Ожидаемая сложность
O(1) time, O(1) extra space
Ссылки
Это задание адаптировано для CodeUtility. Следуйте условию и ограничениям выше.